diff --git a/README.md b/README.md index 7be5fc7f47d5db027d120b8024982df93db95b74..4390000ebbda4f736ce11a64a0ab861d244d87f2 100644 --- a/README.md +++ b/README.md @@ -1,3 +1,137 @@ ---- -license: mit ---- +--- +language: +- en +library_name: transformers +tags: +- glm +- MOE +- pruning +- compression +license: mit +name: cerebras/GLM-4.7-REAP-268B-A32B +description: > + This model was obtained by uniformly pruning 25% of experts in GLM-4.7 using the REAP method. +readme: > + https://huggingface.co/cerebras/GLM-4.7-REAP-268B-A32B/main/README.md +license_link: https://huggingface.co/zai-org/GLM-4.7/blob/main/LICENSE +pipeline_tag: text-generation +base_model: +- zai-org/GLM-4.7 +--- + +

+ ๐“Œณ REAP๐“Œณ the Experts: Why Pruning Prevails for One-Shot MoE Compression
+ REAP +

+ +# GLM-4.7-REAP-268B-A32B + +## โœจ Highlights + +Introducing **GLM-4.7-REAP-268B-A32B**, a **memory-efficient compressed variant** of GLM-4.7 that maintains near-identical performance while being **25% lighter**. + +This model was created using **REAP (Router-weighted Expert Activation Pruning)**, a novel expert pruning method that selectively removes redundant experts while preserving the router's independent control over remaining experts. Key features include: + +- **Near-Lossless Performance**: Maintains almost identical accuracy on code generation, agentic coding, and function calling tasks compared to the full 355B model +- **25% Memory Reduction**: Compressed from 355B to 268B parameters, significantly lowering deployment costs and memory requirements +- **Preserved Capabilities**: Retains all core functionalities including code generation, agentic workflows, repository-scale understanding, and function calling +- **Drop-in Compatibility**: Works with vanilla vLLM - no source modifications or custom patches required +- **Optimized for Real-World Use**: Particularly effective for resource-constrained environments, local deployments, and academic research + +**For downstream low-bit quantization, we suggest using the [BF16 variant](https://huggingface.co/cerebras/GLM-4.7-REAP-268B-A32B).** + +--- +## ๐Ÿ“‹ Model Overview + +**GLM-4.7-REAP-268B-A32B** has the following specifications: + +- **Base Model**: GLM-4.7 +- **Compression Method**: REAP (Router-weighted Expert Activation Pruning) +- **Compression Ratio**: 25% expert pruning +- **Type**: Sparse Mixture-of-Experts (SMoE) Causal Language Model +- **Number of Parameters**: 268B total, 32B activated per token +- **Number of Layers**: 92 +- **Number of Attention Heads (GQA)**: 96 for Q and 8 for KV +- **Number of Experts**: 120 (uniformly pruned from 160) +- **Number of Activated Experts**: 8 per token +- **Context Length**: 202,752 tokens +- **License**: MIT + +--- + +## ๐Ÿ“Š Evaluations + +TBD for BF16 model. [Evalulation results available for the FP8 variant](https://huggingface.co/cerebras/GLM-4.7-REAP-268B-A32B-FP8#%F0%9F%93%8A-evaluations). + +For more details on the evaluation setup, refer to the [REAP arXiv preprint](https://arxiv.org/abs/2510.13999). + +--- + +## ๐Ÿš€ Deployment + +You can deploy the model directly using the **latest vLLM** (v0.11.0), no source modifications or custom patches required. + +```bash +vllm serve cerebras/GLM-4.7-REAP-268B-A32B \ + --tensor-parallel-size 8 \ + --tool-call-parser glm45 \ + --enable-auto-tool-choice \ + --enable-expert-parallel +``` + +If you encounter insufficient memory when running this model, you might need to set a lower value for `--max-num-seqs` flag (e.g. set to 64). + + +## ๐Ÿงฉ Model Creation + +This checkpoint was created by applying the **REAP (Router-weighted Expert Activation Pruning)** method uniformly across all Mixture-of-Experts (MoE) blocks of **GLM-4.7**, with a **25% pruning rate**. + +### How REAP Works + +REAP selects experts to prune based on a novel **saliency criterion** that considers both: +- **Router gate values**: How frequently and strongly the router activates each expert +- **Expert activation norms**: The magnitude of each expert's output contributions + +This dual consideration ensures that experts contributing minimally to the layer's output are pruned, while preserving those that play critical roles in the model's computations. + +### Key Advantages + +- **One-Shot Compression**: No fine-tuning required after pruning - the model is immediately ready for deployment +- **Preserved Router Control**: Unlike expert merging methods, REAP maintains the router's independent, input-dependent control over remaining experts, avoiding "functional subspace collapse" +- **Generative Task Superiority**: REAP significantly outperforms expert merging approaches on generative benchmarks (code generation, creative writing, mathematical reasoning) while maintaining competitive performance on discriminative tasks + +### Calibration + +The model was calibrated using a diverse mixture of domain-specific datasets including: +- Code generation samples ([evol-codealpaca](https://huggingface.co/datasets/theblackcat102/evol-codealpaca-v1)) +- Function calling examples ([xlam-function-calling](https://huggingface.co/datasets/Salesforce/xlam-function-calling-60k)) +- Agentic multi-turn trajectories ([SWE-smith-trajectories](https://huggingface.co/datasets/SWE-bench/SWE-smith-trajectories)) + +๐Ÿ“š For more details, refer to the following resources: + +- [๐Ÿงพ arXiv Preprint](https://arxiv.org/abs/2510.13999) +- [๐Ÿงพ REAP Blog](https://www.cerebras.ai/blog/reap) +- [๐Ÿ’ป REAP Codebase (GitHub)](https://github.com/CerebrasResearch/reap) + +--- + +## โš–๏ธ License + +This model is derived from +**[`zai-org/GLM-4.7`](https://huggingface.co/zai-org/GLM-4.7)** +and distributed under the **MIT license**. + +--- + +## ๐Ÿงพ Citation + +If you use this checkpoint, please cite the REAP paper: + +```bibtex +@article{lasby-reap, + title={REAP the Experts: Why Pruning Prevails for One-Shot MoE compression}, + author={Lasby, Mike and Lazarevich, Ivan and Sinnadurai, Nish and Lie, Sean and Ioannou, Yani and Thangarasa, Vithursan}, + journal={arXiv preprint arXiv:2510.13999}, + year={2025} +} +``` \ No newline at end of file diff --git a/chat_template.jinja b/chat_template.jinja new file mode 100644 index 0000000000000000000000000000000000000000..2ab98ef068d62829d17c5ade1827b9f013fa2bbf --- /dev/null +++ b/chat_template.jinja @@ -0,0 +1,86 @@ +[gMASK] +{%- if tools -%} +<|system|> +# Tools + +You may call one or more functions to assist with the user query. + +You are provided with function signatures within XML tags: + +{% for tool in tools %} +{{ tool | tojson(ensure_ascii=False) }} +{% endfor %} + + +For each function call, output the function name and arguments within the following XML format: +{function-name}{arg-key-1}{arg-value-1}{arg-key-2}{arg-value-2}...{%- endif -%} +{%- macro visible_text(content) -%} + {%- if content is string -%} + {{- content }} + {%- elif content is iterable and content is not mapping -%} + {%- for item in content -%} + {%- if item is mapping and item.type == 'text' -%} + {{- item.text }} + {%- elif item is string -%} + {{- item }} + {%- endif -%} + {%- endfor -%} + {%- else -%} + {{- content }} + {%- endif -%} +{%- endmacro -%} +{%- set ns = namespace(last_user_index=-1) %} +{%- for m in messages %} + {%- if m.role == 'user' %} + {% set ns.last_user_index = loop.index0 -%} + {%- endif %} +{%- endfor %} +{% for m in messages %} +{%- if m.role == 'user' -%}<|user|>{{ visible_text(m.content) }} +{%- elif m.role == 'assistant' -%} +<|assistant|> +{%- set reasoning_content = '' %} +{%- set content = visible_text(m.content) %} +{%- if m.reasoning_content is string %} + {%- set reasoning_content = m.reasoning_content %} +{%- else %} + {%- if '' in content %} + {%- set reasoning_content = content.split('')[0].rstrip('\n').split('')[-1].lstrip('\n') %} + {%- set content = content.split('')[-1].lstrip('\n') %} + {%- endif %} +{%- endif %} +{%- if ((clear_thinking is defined and not clear_thinking) or loop.index0 > ns.last_user_index) and reasoning_content -%} +{{ '' + reasoning_content.strip() + ''}} +{%- else -%} +{{ '' }} +{%- endif -%} +{%- if content.strip() -%} +{{ content.strip() }} +{%- endif -%} +{% if m.tool_calls %} +{% for tc in m.tool_calls %} +{%- if tc.function %} + {%- set tc = tc.function %} +{%- endif %} +{{- '' + tc.name -}} +{% set _args = tc.arguments %}{% for k, v in _args.items() %}{{ k }}{{ v | tojson(ensure_ascii=False) if v is not string else v }}{% endfor %}{% endfor %} +{% endif %} +{%- elif m.role == 'tool' -%} +{%- if m.content is string -%} +{%- if loop.first or (messages[loop.index0 - 1].role != "tool") %} + {{- '<|observation|>' }} +{%- endif %} +{{- '' }} +{{- m.content }} +{{- '' }} +{%- else -%} +<|observation|>{% for tr in m.content %} +{{ tr.output if tr.output is defined else tr }}{% endfor -%} +{% endif -%} +{%- elif m.role == 'system' -%} +<|system|>{{ visible_text(m.content) }} +{%- endif -%} +{%- endfor -%} +{%- if add_generation_prompt -%} + <|assistant|>{{- '' if (enable_thinking is defined and not enable_thinking) else '' -}} +{%- endif -%} \ No newline at end of file diff --git a/config.json b/config.json new file mode 100644 index 0000000000000000000000000000000000000000..97624f55f955bc001ef4ff42e932b862c35a92d4 --- /dev/null +++ b/config.json @@ -0,0 +1,43 @@ +{ + "architectures": [ + "Glm4MoeForCausalLM" + ], + "attention_bias": true, + "attention_dropout": 0.0, + "eos_token_id": [ + 151329, + 151336, + 151338 + ], + "first_k_dense_replace": 3, + "head_dim": 128, + "hidden_act": "silu", + "hidden_size": 5120, + "initializer_range": 0.02, + "intermediate_size": 12288, + "max_position_embeddings": 202752, + "model_type": "glm4_moe", + "moe_intermediate_size": 1536, + "n_group": 1, + "n_routed_experts": 120, + "n_shared_experts": 1, + "norm_topk_prob": true, + "num_attention_heads": 96, + "num_experts_per_tok": 8, + "num_hidden_layers": 92, + "num_key_value_heads": 8, + "num_nextn_predict_layers": 0, + "pad_token_id": 151329, + "partial_rotary_factor": 0.5, + "rms_norm_eps": 1e-05, + "rope_scaling": null, + "rope_theta": 1000000, + "routed_scaling_factor": 2.5, + "tie_word_embeddings": false, + "topk_group": 1, + "torch_dtype": "bfloat16", + "transformers_version": "4.55.0", + "use_cache": true, + "use_qk_norm": true, + "vocab_size": 151552 +} diff --git a/model-00001-of-00101.safetensors b/model-00001-of-00101.safetensors new file mode 100644 index 0000000000000000000000000000000000000000..2a6038ff6b757526f801c79dd0bf5cc08b50d7a1 --- /dev/null +++ b/model-00001-of-00101.safetensors @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:ede6c6a0e9de3b7f67e6256ba08911318e8cb11fbef6b858817607f0c0ac554a +size 5363662896 diff --git a/model-00002-of-00101.safetensors b/model-00002-of-00101.safetensors new file mode 100644 index 0000000000000000000000000000000000000000..fc61745a499003186a08e52a16583e3e6b0275af --- /dev/null +++ b/model-00002-of-00101.safetensors @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:d6239e07c8745f600cc8fa2aecdadc6e34e296b0957dbb74ea00b5cce76607fa +size 5354300984 diff --git a/model-00003-of-00101.safetensors b/model-00003-of-00101.safetensors new file mode 100644 index 0000000000000000000000000000000000000000..283895b54c8640ae40687cdee30ff80f914e8bc4 --- /dev/null +++ b/model-00003-of-00101.safetensors @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:60fcad3686cea0788d58ef8f6618a4679dd1b32e6b9cf2356e93f3096da1650b +size 5354300984 diff --git a/model-00004-of-00101.safetensors b/model-00004-of-00101.safetensors new file mode 100644 index 0000000000000000000000000000000000000000..1b6c80c3bbbcd0c7f452b1441c3f0404a8fc2cee --- /dev/null +++ b/model-00004-of-00101.safetensors @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:d2e88d666058abebeeec96ffe2437ef3960af68343f04c2c0e90d6a5af0bb306 +size 5364738088 diff --git a/model-00005-of-00101.safetensors b/model-00005-of-00101.safetensors new file mode 100644 index 0000000000000000000000000000000000000000..5331307bb7b9ddf820ec73b4ef79940fd2b1a5a2 --- /dev/null +++ b/model-00005-of-00101.safetensors @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:098d018a0099575ddbdc401454cfc0b815ef9f37d1124094aed8fb44994576a8 +size 5353071440 diff --git a/model-00006-of-00101.safetensors b/model-00006-of-00101.safetensors new file mode 100644 index 0000000000000000000000000000000000000000..7246d3bb9c058477d09a192096a22b88843eb2e5 --- /dev/null +++ b/model-00006-of-00101.safetensors @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:1cf2073694fa982cb5db204ffea925ce450f7381fa6fa470eb040c5d94dcc3df +size 5354300952 diff --git a/model-00007-of-00101.safetensors b/model-00007-of-00101.safetensors new file mode 100644 index 0000000000000000000000000000000000000000..f146fdb7b001cd48e8713ca1cb770924658dc2b8 --- /dev/null +++ b/model-00007-of-00101.safetensors @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:9e3db218d091de80f1a566c9bf5dfd64e8a94633205bc04c02681bad28e8b5b3 +size 5354300976 diff --git a/model-00008-of-00101.safetensors b/model-00008-of-00101.safetensors new file mode 100644 index 0000000000000000000000000000000000000000..b6491120443ab775508bf0d7a1bf76c6bd693202 --- /dev/null +++ b/model-00008-of-00101.safetensors @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:7f020df07f5bc0524732077099dcedbbd472e451755cbb748000c5791062099b +size 5354300984 diff --git a/model-00009-of-00101.safetensors b/model-00009-of-00101.safetensors new file mode 100644 index 0000000000000000000000000000000000000000..7f71f0f3fbbc11a15827d8bf3d83bae558b95463 --- /dev/null +++ b/model-00009-of-00101.safetensors @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:4c7f3421b15ccebcc80598af09e4b2fab8bfbc87301f9dd90d96a0eee740c4b1 +size 5354301160 diff --git a/model-00010-of-00101.safetensors b/model-00010-of-00101.safetensors new file mode 100644 index 0000000000000000000000000000000000000000..e1a339dd93e40602bc62a65a01256e64de71dec0 --- /dev/null +++ b/model-00010-of-00101.safetensors @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:3e604cf2774246980a317b54958ee1af61116d6c3c2343e46d4643537ac50098 +size 5354301312 diff --git a/model-00011-of-00101.safetensors b/model-00011-of-00101.safetensors new file mode 100644 index 0000000000000000000000000000000000000000..264bf1dbd14fe81780050f93c08e57c5ae824d74 --- /dev/null +++ b/model-00011-of-00101.safetensors @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:1bc7efe2718953ef60b829890788ab79a2a46f52ef9809a63138a818422c3429 +size 5354301320 diff --git a/model-00012-of-00101.safetensors b/model-00012-of-00101.safetensors new file mode 100644 index 0000000000000000000000000000000000000000..a8538d1d5427f08b9164d0e25176e6af2145a771 --- /dev/null +++ b/model-00012-of-00101.safetensors @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:35bc379a16b9d2928a0ebb59f2a9b7bbc09aae93531e4419c9c15416f76e4719 +size 5354301312 diff --git a/model-00013-of-00101.safetensors b/model-00013-of-00101.safetensors new file mode 100644 index 0000000000000000000000000000000000000000..4c23f5baa119709353ad5deadc50fa26ba6e5743 --- /dev/null +++ b/model-00013-of-00101.safetensors @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:5cc32d4e390abc16cb7adf17b2abd613d6133a64c702cf5961af900eebf834d4 +size 5354301344 diff --git a/model-00014-of-00101.safetensors b/model-00014-of-00101.safetensors new file mode 100644 index 0000000000000000000000000000000000000000..4f7822cba013fc092cd81bf214738dd92acd8ef9 --- /dev/null +++ b/model-00014-of-00101.safetensors @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:d833c7367750a4c669bc9c186c0f40c1a74e09b9d6d22dd96d4c8a483022410f +size 5363508888 diff --git a/model-00015-of-00101.safetensors b/model-00015-of-00101.safetensors new file mode 100644 index 0000000000000000000000000000000000000000..aea24b21126498b1066fa9ab9c647a6be9650969 --- /dev/null +++ b/model-00015-of-00101.safetensors @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:f46dbc7b61184d2842edf3b60898d8b097d44d0068897da6198a15b80979b9c6 +size 5354301272 diff --git a/model-00016-of-00101.safetensors b/model-00016-of-00101.safetensors new file mode 100644 index 0000000000000000000000000000000000000000..1ac3194d47fb518b77cf710fa31a5f6b45bd2cc8 --- /dev/null +++ b/model-00016-of-00101.safetensors @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:1a3a53658f561a1d13effa6204cc108b6ef72a438fe03ac9cb9cfb4126890a45 +size 5354301304 diff --git a/model-00017-of-00101.safetensors b/model-00017-of-00101.safetensors new file mode 100644 index 0000000000000000000000000000000000000000..cfd9aa4a35fc7ef6961c1c180ab5bd5448be5924 --- /dev/null +++ b/model-00017-of-00101.safetensors @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:0340ebbd48bef6c01da36223594db9cb4d85616fd9edd5b013d91902b6670cdc +size 5354301320 diff --git a/model-00018-of-00101.safetensors b/model-00018-of-00101.safetensors new file mode 100644 index 0000000000000000000000000000000000000000..1cfc00d4a0427de368c42a11701214156e7b1338 --- /dev/null +++ b/model-00018-of-00101.safetensors @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:23be87625478ed570833863db47aedf016db87514d12500b732c750d726c1e48 +size 5354301312 diff --git a/model-00019-of-00101.safetensors b/model-00019-of-00101.safetensors new file mode 100644 index 0000000000000000000000000000000000000000..5c6b9e959e35a0015f885599146a7014008b7a97 --- /dev/null +++ b/model-00019-of-00101.safetensors @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:c25ca828202d993c71bcec36792d149824a598797cfc499afd3c7f065acbae59 +size 5354301312 diff --git a/model-00020-of-00101.safetensors b/model-00020-of-00101.safetensors new file mode 100644 index 0000000000000000000000000000000000000000..fd67e99f049c695af8a6f5f18653fa69fcb71746 --- /dev/null +++ b/model-00020-of-00101.safetensors @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:7f2f48ee4b803cc500912d401201a592b5da7eb14e709f321e4703927580afab +size 5354301320 diff --git a/model-00021-of-00101.safetensors b/model-00021-of-00101.safetensors new file mode 100644 index 0000000000000000000000000000000000000000..30aa681e0b4324192112f3051a03ae1bfea3b9d4 --- /dev/null +++ b/model-00021-of-00101.safetensors @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:0681d7894d9cfaec8f1e150afd25bae19b508900d1cadd23dc00b875e1944f0d +size 5354301312 diff --git a/model-00022-of-00101.safetensors b/model-00022-of-00101.safetensors new file mode 100644 index 0000000000000000000000000000000000000000..48513cabeecb6b86ea440f43f5ed7b6169a813d4 --- /dev/null +++ b/model-00022-of-00101.safetensors @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:6f78b9ca0308e3de0a96adcf01ab8e3105fb2867dd0f909d0bc02cf854c2a24d +size 5354301320 diff --git a/model-00023-of-00101.safetensors b/model-00023-of-00101.safetensors new file mode 100644 index 0000000000000000000000000000000000000000..0fba451ef12e0e3f76433d68efa1e554a65385cf --- /dev/null +++ b/model-00023-of-00101.safetensors @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:bbc793310c41b9f92137c520341f8650b1b3703e798d46db7223cbe38378658d +size 5349030376 diff --git a/model-00024-of-00101.safetensors b/model-00024-of-00101.safetensors new file mode 100644 index 0000000000000000000000000000000000000000..a5413c0eb894206332f5a3eca2209a79d94f3362 --- /dev/null +++ b/model-00024-of-00101.safetensors @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:e3f9fc40d43cd9b1c79945205feb1b1ff54c2b1d8d1bb50c94e6483d028e4ba2 +size 5353051064 diff --git a/model-00025-of-00101.safetensors b/model-00025-of-00101.safetensors new file mode 100644 index 0000000000000000000000000000000000000000..f184f9b7d2d44e2f26c90f43bdad3ede35e8c947 --- /dev/null +++ b/model-00025-of-00101.safetensors @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:88a46be217e6539c4a6c1b6521f6ddab78e4d4bec5a80fea53735403864a9a98 +size 5354301288 diff --git a/model-00026-of-00101.safetensors b/model-00026-of-00101.safetensors new file mode 100644 index 0000000000000000000000000000000000000000..f7e9c404331c460a5edf464037dc63d642fa8dbf --- /dev/null +++ b/model-00026-of-00101.safetensors @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:9eeb271e88637a083c34ccd1fb5d77fbe663d8a59ebbfcac9fbbc01f1b6d728a +size 5354301312 diff --git a/model-00027-of-00101.safetensors b/model-00027-of-00101.safetensors new file mode 100644 index 0000000000000000000000000000000000000000..7710bc12e11da1e5178474c2d55376ce6e9c4bdb --- /dev/null +++ b/model-00027-of-00101.safetensors @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:b5d7a146c292183e29db53d2b00d586ab6f8407e1323aaaf0da708d5cc64f7f1 +size 5354301312 diff --git a/model-00028-of-00101.safetensors b/model-00028-of-00101.safetensors new file mode 100644 index 0000000000000000000000000000000000000000..6de95033e1d6de4e9598c9374b249c101349b91e --- /dev/null +++ b/model-00028-of-00101.safetensors @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:91043910206957b011adf4f41312c8135119ac7d8aabd3cab55d3378f576f6ed +size 5354301320 diff --git a/model-00029-of-00101.safetensors b/model-00029-of-00101.safetensors new file mode 100644 index 0000000000000000000000000000000000000000..861a92677e20f188de6d8cb1ebceba3d9f68bbbd --- /dev/null +++ b/model-00029-of-00101.safetensors @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:9ecd6d24cabf8716cd7fac3faec581b754388cfd95d99914f31bcecd813ab389 +size 5354301312 diff --git a/model-00030-of-00101.safetensors b/model-00030-of-00101.safetensors new file mode 100644 index 0000000000000000000000000000000000000000..d32b487a865a8ea4de4edff44deaeb9db9c1456e --- /dev/null +++ b/model-00030-of-00101.safetensors @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:52a59db18bae58e6d1047f8cbc9691d2fdbff8181563d800a4cf77410b4b711a +size 5354301312 diff --git a/model-00031-of-00101.safetensors b/model-00031-of-00101.safetensors new file mode 100644 index 0000000000000000000000000000000000000000..3239203a3b7407bd2321e70d2fdfdc493a7a10d0 --- /dev/null +++ b/model-00031-of-00101.safetensors @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:2d656d01405b685dff98cc32b803915e54270625404babb2d37fd79a5001cca4 +size 5354301320 diff --git a/model-00032-of-00101.safetensors b/model-00032-of-00101.safetensors new file mode 100644 index 0000000000000000000000000000000000000000..d4e3d15fc4dc3fb063598a83c05b0470e807aab0 --- /dev/null +++ b/model-00032-of-00101.safetensors @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:9287bf6c56662d8a4c7a2efafbf6f720cbbe9c302affec19d274b0f566cc8b6c +size 5354301344 diff --git a/model-00033-of-00101.safetensors b/model-00033-of-00101.safetensors new file mode 100644 index 0000000000000000000000000000000000000000..6aee751f4e7867024c10e1d752dc13e712a3e630 --- /dev/null +++ b/model-00033-of-00101.safetensors @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:74d4383d62d189b33e92925b36a5536feb06300803db4efbe5cb8953a515ce5d +size 5363508888 diff --git a/model-00034-of-00101.safetensors b/model-00034-of-00101.safetensors new file mode 100644 index 0000000000000000000000000000000000000000..9207bee2133b5261b8977497e5e793bf9ab7cd64 --- /dev/null +++ b/model-00034-of-00101.safetensors @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:07d6773470b989b7637570036884dab91f03b57a3e2be9255df7735ddacc6003 +size 5354301272 diff --git a/model-00035-of-00101.safetensors b/model-00035-of-00101.safetensors new file mode 100644 index 0000000000000000000000000000000000000000..71806f52b6e478308b70d992a8298ae90668d314 --- /dev/null +++ b/model-00035-of-00101.safetensors @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:3dd45debdcbf8d94803847a8f70de910370f7a26028d3801aa3a435666d9c01c +size 5354301304 diff --git a/model-00036-of-00101.safetensors b/model-00036-of-00101.safetensors new file mode 100644 index 0000000000000000000000000000000000000000..a75a1d118f6621ab022b76a8bfc71f6d6f6a093c --- /dev/null +++ b/model-00036-of-00101.safetensors @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:cd80b39bdfcc88b4eedb8dad8806894aa7d66c5936486ddda1e560f2fc3cdf64 +size 5354301312 diff --git a/model-00037-of-00101.safetensors b/model-00037-of-00101.safetensors new file mode 100644 index 0000000000000000000000000000000000000000..ef6058b1d3039070fe30e8d4e6e71856bf81eb4f --- /dev/null +++ b/model-00037-of-00101.safetensors @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:5acf3e1720737b527a3e621988bda073ae8724122905bf02ce4bb5570467ea4b +size 5354301320 diff --git a/model-00038-of-00101.safetensors b/model-00038-of-00101.safetensors new file mode 100644 index 0000000000000000000000000000000000000000..11f52938169b9b3cd1936e65edc9bd57cf0b1a3d --- /dev/null +++ b/model-00038-of-00101.safetensors @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:3e1ba43051565a9896c6fb283282df21d09e4a8a2ede3d0a5cae70a28e9544ed +size 5354301312 diff --git a/model-00039-of-00101.safetensors b/model-00039-of-00101.safetensors new file mode 100644 index 0000000000000000000000000000000000000000..4cf30a759d436ce84e1d385ddf19b8955258ff44 --- /dev/null +++ b/model-00039-of-00101.safetensors @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:01ed5a3c07f07f842863a693ecd82c52b39d61546b884ef43a41039f82563d45 +size 5354301312 diff --git a/model-00040-of-00101.safetensors b/model-00040-of-00101.safetensors new file mode 100644 index 0000000000000000000000000000000000000000..d462feb5d88e387b1b6e0955a17fbefb0a75d583 --- /dev/null +++ b/model-00040-of-00101.safetensors @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:f8fa3222b4e3e1bd4eea2863ffec0ff1fda7f8f3250c97a2db0466a59881d717 +size 5354301320 diff --git a/model-00041-of-00101.safetensors b/model-00041-of-00101.safetensors new file mode 100644 index 0000000000000000000000000000000000000000..d417efc50cfdade8976bee6d0751e6cf5f3e84ff --- /dev/null +++ b/model-00041-of-00101.safetensors @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:377d568d9db28a1e17c87386acd2d2cd7309eaa6ba5890a9ad1ae9e672813e1b +size 5354301320 diff --git a/model-00042-of-00101.safetensors b/model-00042-of-00101.safetensors new file mode 100644 index 0000000000000000000000000000000000000000..a9c3a7ecde282235e686fef41cf76d1f7a3b9262 --- /dev/null +++ b/model-00042-of-00101.safetensors @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:2cb2db48589d7893d1f3d988988799198639dfd716a6101661d5268d9d1b875a +size 5333301608 diff --git a/model-00043-of-00101.safetensors b/model-00043-of-00101.safetensors new file mode 100644 index 0000000000000000000000000000000000000000..1029e9a348c83b39d79214a6531a1fdc2b807198 --- /dev/null +++ b/model-00043-of-00101.safetensors @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:7e550ff1abc1873fcbbb8cf124b0d3eb131cc7a069c827280cf983bd096a6395 +size 5353051064 diff --git a/model-00044-of-00101.safetensors b/model-00044-of-00101.safetensors new file mode 100644 index 0000000000000000000000000000000000000000..ddb1af2eb500deeb62cf2b49baebc96576bda2ed --- /dev/null +++ b/model-00044-of-00101.safetensors @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:98d8d24e5958e8ace266dc7dcdf7bfb9b02d191ec5b7c008b3689622560d039b +size 5354301288 diff --git a/model-00045-of-00101.safetensors b/model-00045-of-00101.safetensors new file mode 100644 index 0000000000000000000000000000000000000000..4763069f4dc470805932fbe5a88b8f42b8fb5a4b --- /dev/null +++ b/model-00045-of-00101.safetensors @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:09a7687e61faaebb4db6db2befc09bfd9e1f840a0f02d708f62e7ba7add5d134 +size 5354301312 diff --git a/model-00046-of-00101.safetensors b/model-00046-of-00101.safetensors new file mode 100644 index 0000000000000000000000000000000000000000..608f4851a8a132b5efeacf39d813f9bc1053b3f2 --- /dev/null +++ b/model-00046-of-00101.safetensors @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:d7aa431bd41d1309ff91d185976dea69c057f957021a418096dc6c3cdfdbca58 +size 5354301312 diff --git a/model-00047-of-00101.safetensors b/model-00047-of-00101.safetensors new file mode 100644 index 0000000000000000000000000000000000000000..00db7b0930e0b9d19204012d476db90d8ef11996 --- /dev/null +++ b/model-00047-of-00101.safetensors @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:3ea18aaf3058d203df0e3f365221c5c69d4d3820ee50b2e9728dda13ec898fe9 +size 5354301320 diff --git a/model-00048-of-00101.safetensors b/model-00048-of-00101.safetensors new file mode 100644 index 0000000000000000000000000000000000000000..9f234b18760f55d7d653fdb4a1744830a751ea64 --- /dev/null +++ b/model-00048-of-00101.safetensors @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:016dc7744851427cd855125e5b59220b0400f4cccec49f7ef955f067ecebf58e +size 5354301312 diff --git a/model-00049-of-00101.safetensors b/model-00049-of-00101.safetensors new file mode 100644 index 0000000000000000000000000000000000000000..03f8d523f802c70d0958e8f35da39e331f5e398f --- /dev/null +++ b/model-00049-of-00101.safetensors @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:77f7100076942049e9869cc361d8f57b464469041fe9f360716c0a6973e3cf50 +size 5354301312 diff --git a/model-00050-of-00101.safetensors b/model-00050-of-00101.safetensors new file mode 100644 index 0000000000000000000000000000000000000000..f1a1188b9c609f0569272580131a33be814b4e59 --- /dev/null +++ b/model-00050-of-00101.safetensors @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:bf5c5faf24c325f57772dbadb7e5f5172610caa5e0ad76e0d1e5bf2e6588daf1 +size 5354301320 diff --git a/model-00051-of-00101.safetensors b/model-00051-of-00101.safetensors new file mode 100644 index 0000000000000000000000000000000000000000..3dc10a95da95a87adcf1636bef8603f988f44671 --- /dev/null +++ b/model-00051-of-00101.safetensors @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:c6cf3eb53282f5ff9c43d6c75549b8a9c6007ab7d1db9315485d52325ff08553 +size 5354301344 diff --git a/model-00052-of-00101.safetensors b/model-00052-of-00101.safetensors new file mode 100644 index 0000000000000000000000000000000000000000..499b419bffe3b611aad9c65dade4786e350fc191 --- /dev/null +++ b/model-00052-of-00101.safetensors @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:ed22882f4e4cd773de849929aef700da3a373269db374130716d9922e750094a +size 5363508888 diff --git a/model-00053-of-00101.safetensors b/model-00053-of-00101.safetensors new file mode 100644 index 0000000000000000000000000000000000000000..373be282929f5451c42261ae1dac188235d0727f --- /dev/null +++ b/model-00053-of-00101.safetensors @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:85222f7001f29494f2d994df2ad378bef6d65e68f7a11814a037e08e36c78c44 +size 5354301272 diff --git a/model-00054-of-00101.safetensors b/model-00054-of-00101.safetensors new file mode 100644 index 0000000000000000000000000000000000000000..aa873da909398087074769bb1af2ed6a2eb118ca --- /dev/null +++ b/model-00054-of-00101.safetensors @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:7bad97de60e1cf9f4964ede995bad13b393c2fe50645b3bb0de8518f3f45b428 +size 5354301304 diff --git a/model-00055-of-00101.safetensors b/model-00055-of-00101.safetensors new file mode 100644 index 0000000000000000000000000000000000000000..f053666785d7b4a6dc27acbc55c2f8d5290e44dd --- /dev/null +++ b/model-00055-of-00101.safetensors @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:f80e234fc9d80b794d66707e1104c716bd66650d3497f28441d1081a5c942ee4 +size 5354301312 diff --git a/model-00056-of-00101.safetensors b/model-00056-of-00101.safetensors new file mode 100644 index 0000000000000000000000000000000000000000..3ba4b1a5c92843ed960930167f27e9c37b4021f5 --- /dev/null +++ b/model-00056-of-00101.safetensors @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:5d47c71797aa6bea411ab1b5bf1bccb5e8f739f9bfa1638e5c59f877b01a10a9 +size 5354301320 diff --git a/model-00057-of-00101.safetensors b/model-00057-of-00101.safetensors new file mode 100644 index 0000000000000000000000000000000000000000..9244a37b591bee0033695e9359ecd2d53a9e792f --- /dev/null +++ b/model-00057-of-00101.safetensors @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:e5a52a7e67a6f41a1c48a11eab673907b43ace32d193d16bc47d643b48bafd5e +size 5354301312 diff --git a/model-00058-of-00101.safetensors b/model-00058-of-00101.safetensors new file mode 100644 index 0000000000000000000000000000000000000000..83348179328db31d1f44b806a3048621a0cb02e0 --- /dev/null +++ b/model-00058-of-00101.safetensors @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:8b45a5e2e3f4ef30219f644eb77b65aa7405d949166d2caa4090173239ce2ad5 +size 5354301312 diff --git a/model-00059-of-00101.safetensors b/model-00059-of-00101.safetensors new file mode 100644 index 0000000000000000000000000000000000000000..815161b38e8dff7e45700dcf68e2037ea2041477 --- /dev/null +++ b/model-00059-of-00101.safetensors @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:30b4e3946ba9bb42d70aa2a447608af844d197711e64fc57e1fff9b43ea63c6c +size 5354301320 diff --git a/model-00060-of-00101.safetensors b/model-00060-of-00101.safetensors new file mode 100644 index 0000000000000000000000000000000000000000..6ad753741079badbf0095885f130e69becff9743 --- /dev/null +++ b/model-00060-of-00101.safetensors @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:5fc511380a015c9e24aa1c4c2e8cfe0cd00c8e243a2426d235342e5df3ae5a40 +size 5354301320 diff --git a/model-00061-of-00101.safetensors b/model-00061-of-00101.safetensors new file mode 100644 index 0000000000000000000000000000000000000000..1dcff5de7694e9c4ed9f14d101102d0f7ffb0b84 --- /dev/null +++ b/model-00061-of-00101.safetensors @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:e46000ab0218871c3682c451f78f33cbd1398c479823ccdbf3aae5f43f0d7e20 +size 5333301608 diff --git a/model-00062-of-00101.safetensors b/model-00062-of-00101.safetensors new file mode 100644 index 0000000000000000000000000000000000000000..61c402c5fb91d76d774056c1878af20ec72f1eb7 --- /dev/null +++ b/model-00062-of-00101.safetensors @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:6669185f392aab3603d2366592ff32f179c180069bf9abd1a6e1f10604e11345 +size 5353051064 diff --git a/model-00063-of-00101.safetensors b/model-00063-of-00101.safetensors new file mode 100644 index 0000000000000000000000000000000000000000..16371bb0297a57189d02a58a29afb9e6a139db30 --- /dev/null +++ b/model-00063-of-00101.safetensors @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:25334db3d3a4bf6fc01679150aee1299074d44d2eb5e6392f247506a04aeae01 +size 5354301288 diff --git a/model-00064-of-00101.safetensors b/model-00064-of-00101.safetensors new file mode 100644 index 0000000000000000000000000000000000000000..10980739e2ea7d275bc4a8f20d6d2f2d35647fbb --- /dev/null +++ b/model-00064-of-00101.safetensors @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:fb98baef3bfa339168beb04dc3e56554c26029577f95c88d2ebf082c0dda58ed +size 5354301312 diff --git a/model-00065-of-00101.safetensors b/model-00065-of-00101.safetensors new file mode 100644 index 0000000000000000000000000000000000000000..ec66ab3e3d90a70c5c0f9aa79a95240895a2b901 --- /dev/null +++ b/model-00065-of-00101.safetensors @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:0371410d620f6826b42b03b17c4664f8aa7ff9a8f3a74bb7df6e6b1dc506ffa9 +size 5354301312 diff --git a/model-00066-of-00101.safetensors b/model-00066-of-00101.safetensors new file mode 100644 index 0000000000000000000000000000000000000000..c0d8a1a68afe71e54847bb6206d763199ff4e399 --- /dev/null +++ b/model-00066-of-00101.safetensors @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:d205f1866df9148273479664e66bbc05629b3872830c41d8a96e4edafc09f62a +size 5354301320 diff --git a/model-00067-of-00101.safetensors b/model-00067-of-00101.safetensors new file mode 100644 index 0000000000000000000000000000000000000000..53e7723934df00abe0153e179e5aaac52149767b --- /dev/null +++ b/model-00067-of-00101.safetensors @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:303e099055a792d2b6a4c53d1996a155e2a3093a989d6ecc6e8812f13d60b534 +size 5354301312 diff --git a/model-00068-of-00101.safetensors b/model-00068-of-00101.safetensors new file mode 100644 index 0000000000000000000000000000000000000000..0968c90fcc9649c61693cfaeef89859b4d765d3f --- /dev/null +++ b/model-00068-of-00101.safetensors @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:2dd6d61d9e4b1322b741bc16e53fc98b7d6da4002a5e684aff049222a5d981de +size 5354301312 diff --git a/model-00069-of-00101.safetensors b/model-00069-of-00101.safetensors new file mode 100644 index 0000000000000000000000000000000000000000..bc2ce22a545bbc236e4388723ef93910058dda10 --- /dev/null +++ b/model-00069-of-00101.safetensors @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:41e788a7cce79885437e31b355000d9759b5cc16926a416588192fa8deb57e89 +size 5354301320 diff --git a/model-00070-of-00101.safetensors b/model-00070-of-00101.safetensors new file mode 100644 index 0000000000000000000000000000000000000000..00d1b3f78c635b54a325baa36d07f2a28179ebd6 --- /dev/null +++ b/model-00070-of-00101.safetensors @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:d02eb8096df8676fd2e87d23db3940aa03393c1b2bc08f84fa399e82a2f376fb +size 5354301344 diff --git a/model-00071-of-00101.safetensors b/model-00071-of-00101.safetensors new file mode 100644 index 0000000000000000000000000000000000000000..335fc228635ff93399fd545ee98f29b1c1a8e878 --- /dev/null +++ b/model-00071-of-00101.safetensors @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:8902c2ff3042363b3a557d8b23381588a535747d3364e178bc599c21d64c37c6 +size 5363508888 diff --git a/model-00072-of-00101.safetensors b/model-00072-of-00101.safetensors new file mode 100644 index 0000000000000000000000000000000000000000..be61079558914343821d6aee7e489ebfc1909aaa --- /dev/null +++ b/model-00072-of-00101.safetensors @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:6662e68de45126998ad2c115e479e9b18029701b136e2c20d5451f17002281cd +size 5354301272 diff --git a/model-00073-of-00101.safetensors b/model-00073-of-00101.safetensors new file mode 100644 index 0000000000000000000000000000000000000000..afb12721d45846d4841eb535ed5bb2e246aefcf5 --- /dev/null +++ b/model-00073-of-00101.safetensors @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:395742e488cc92c12d8ae248c7c3fc03901ca0322c0042aecf8495d22f4057a7 +size 5354301304 diff --git a/model-00074-of-00101.safetensors b/model-00074-of-00101.safetensors new file mode 100644 index 0000000000000000000000000000000000000000..745e0b2b8a61e0fea28a124cdb1d1147afc8ff99 --- /dev/null +++ b/model-00074-of-00101.safetensors @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:5ebb8db6ad31f3634843a7ff052de9f8a345ecea8798576d2a364f1ba388b147 +size 5354301312 diff --git a/model-00075-of-00101.safetensors b/model-00075-of-00101.safetensors new file mode 100644 index 0000000000000000000000000000000000000000..a165a867395b227ab65951769930de9c8bb69212 --- /dev/null +++ b/model-00075-of-00101.safetensors @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:7de05de5ad7d4372a0d41bd944ed71adb6cbd830243c84e3c05cac5bb97136d9 +size 5354301320 diff --git a/model-00076-of-00101.safetensors b/model-00076-of-00101.safetensors new file mode 100644 index 0000000000000000000000000000000000000000..9db7a1c67a79c5081d7226bc3a395676faea4f69 --- /dev/null +++ b/model-00076-of-00101.safetensors @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:8cebbb8ec8f0b8331dd7be5cc70f4385c3c1a3675322cea019fc3f8dced69368 +size 5354301312 diff --git a/model-00077-of-00101.safetensors b/model-00077-of-00101.safetensors new file mode 100644 index 0000000000000000000000000000000000000000..06dcd0189f44b7e07a002631258c80148f1e603c --- /dev/null +++ b/model-00077-of-00101.safetensors @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:8196e5476d8a9184d7d612108c952511451a4a62a2a29c1f1b8e743e402e7903 +size 5354301312 diff --git a/model-00078-of-00101.safetensors b/model-00078-of-00101.safetensors new file mode 100644 index 0000000000000000000000000000000000000000..053fd01f64dab08d31a9b9c6deb01dfb72a22d51 --- /dev/null +++ b/model-00078-of-00101.safetensors @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:26b2fd774dee00564f0bccdae12879d02a235bab4be76a74a7be1d43b32a0faf +size 5354301320 diff --git a/model-00079-of-00101.safetensors b/model-00079-of-00101.safetensors new file mode 100644 index 0000000000000000000000000000000000000000..91634f33609785758c0c749f82bd48fe49e403f6 --- /dev/null +++ b/model-00079-of-00101.safetensors @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:c1bd691de3481fc71100f1607d4f054ef288e931e2fd0b70c08b02c18317a101 +size 5354301320 diff --git a/model-00080-of-00101.safetensors b/model-00080-of-00101.safetensors new file mode 100644 index 0000000000000000000000000000000000000000..39431cda4097ad3bd5fb5709f1e212787bab5560 --- /dev/null +++ b/model-00080-of-00101.safetensors @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:272f39a5252aaa6d2b3964cf19b966083db80f3aff35ee79b2ef9ca26cabf01a +size 5333301608 diff --git a/model-00081-of-00101.safetensors b/model-00081-of-00101.safetensors new file mode 100644 index 0000000000000000000000000000000000000000..c366babf2ba9c2706945027ae7a9f2fe839d88e5 --- /dev/null +++ b/model-00081-of-00101.safetensors @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:dc811f5caa9dc50ad0f9bd217df15ead145b7997c400c836df2aa32ae8ae79b9 +size 5353051064 diff --git a/model-00082-of-00101.safetensors b/model-00082-of-00101.safetensors new file mode 100644 index 0000000000000000000000000000000000000000..5021fb311fd4718ea26246611ebf3b0e3bf0cdba --- /dev/null +++ b/model-00082-of-00101.safetensors @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:4999cead5a71c4f05b5826046cffd5141ce1da760aa4ee71b84670be0dc2a7b3 +size 5354301288 diff --git a/model-00083-of-00101.safetensors b/model-00083-of-00101.safetensors new file mode 100644 index 0000000000000000000000000000000000000000..47edcc636eec24cf76649af3c3230ce06ccc769b --- /dev/null +++ b/model-00083-of-00101.safetensors @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:d90e89bd5e1a237af5a5cec2f67474e156e7484d8b83c9d6b7997f1b5e06d512 +size 5354301312 diff --git a/model-00084-of-00101.safetensors b/model-00084-of-00101.safetensors new file mode 100644 index 0000000000000000000000000000000000000000..0df55d11487051e4240345e38d8d29c33b0ed7d9 --- /dev/null +++ b/model-00084-of-00101.safetensors @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:4174d5ce844fe8796343fd7970261b5ad788efca493dbc146ae5e1b973c59652 +size 5354301312 diff --git a/model-00085-of-00101.safetensors b/model-00085-of-00101.safetensors new file mode 100644 index 0000000000000000000000000000000000000000..9abcbb2aef123d30edbccf6a691d61b6607b8954 --- /dev/null +++ b/model-00085-of-00101.safetensors @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:db91e04bcb71b68e1329a08479cd0780574278390608d0cef5970720c743e4dd +size 5354301320 diff --git a/model-00086-of-00101.safetensors b/model-00086-of-00101.safetensors new file mode 100644 index 0000000000000000000000000000000000000000..540bb4f4cbe8f1ffb398ae1e90eedb13a05b8084 --- /dev/null +++ b/model-00086-of-00101.safetensors @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:4bf28f8b2fea8fbc37bc805b82b6ec7bde11fd67d6a076ccb3a6904b1b4f2ab8 +size 5354301312 diff --git a/model-00087-of-00101.safetensors b/model-00087-of-00101.safetensors new file mode 100644 index 0000000000000000000000000000000000000000..78a24172e25affadc3420ebfee546c3e2d780542 --- /dev/null +++ b/model-00087-of-00101.safetensors @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:605ce632924bc60b7c4cc46b8a8565292c7cd8ef9e444fa40e43ad2679ab01b4 +size 5354301312 diff --git a/model-00088-of-00101.safetensors b/model-00088-of-00101.safetensors new file mode 100644 index 0000000000000000000000000000000000000000..bc9574b087809ac70680a0dea2e3f412b4f96acc --- /dev/null +++ b/model-00088-of-00101.safetensors @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:fe8e714ecd21a8e258d7bff04306218cfd4cc40aed29be7a8c858c6a85a95a08 +size 5354301320 diff --git a/model-00089-of-00101.safetensors b/model-00089-of-00101.safetensors new file mode 100644 index 0000000000000000000000000000000000000000..6d5020e4c320111afa78e3138adefbfd4b0bafe6 --- /dev/null +++ b/model-00089-of-00101.safetensors @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:4322a372fb9565364c81b627ba74385f89d6b4e3e0da540a424515faaa23543d +size 5354301344 diff --git a/model-00090-of-00101.safetensors b/model-00090-of-00101.safetensors new file mode 100644 index 0000000000000000000000000000000000000000..51e8e67bbf5abe966fa2af31c8e7509d8a7f2265 --- /dev/null +++ b/model-00090-of-00101.safetensors @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:80ef4201fe7efd3749a0fcc524e3beab7aa0764237be1e08f60b4ec6fb39b6e7 +size 5363508888 diff --git a/model-00091-of-00101.safetensors b/model-00091-of-00101.safetensors new file mode 100644 index 0000000000000000000000000000000000000000..458f7dec9a274425edbda258d4b3980d112cf71d --- /dev/null +++ b/model-00091-of-00101.safetensors @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:59e6277810c0ba904fd7e0b20b7fa3020bafecbcbc98d6c30a1c2e55f35861dd +size 5354301272 diff --git a/model-00092-of-00101.safetensors b/model-00092-of-00101.safetensors new file mode 100644 index 0000000000000000000000000000000000000000..d81e85e0927fe89f989ba147041c9d0eab3a00db --- /dev/null +++ b/model-00092-of-00101.safetensors @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:2d829d87ef08d5ce62e8d98cf02f6ff821980bd6c530c69f89734020f6181e01 +size 5354301304 diff --git a/model-00093-of-00101.safetensors b/model-00093-of-00101.safetensors new file mode 100644 index 0000000000000000000000000000000000000000..6c1e04ef76e2871c94c91f1050ae5bdeb5401aaf --- /dev/null +++ b/model-00093-of-00101.safetensors @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:5187d869ce220ec5a6042f5ac4fc375f094fbec531f6f34ecce85fd4e6a664dc +size 5354301312 diff --git a/model-00094-of-00101.safetensors b/model-00094-of-00101.safetensors new file mode 100644 index 0000000000000000000000000000000000000000..10e373bcd0df0e15edad959a05966bce497bdf47 --- /dev/null +++ b/model-00094-of-00101.safetensors @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:2ffaa37e16d437be79180ef5ace39da0f99237ee8b326f7cdc41cc983850914e +size 5354301320 diff --git a/model-00095-of-00101.safetensors b/model-00095-of-00101.safetensors new file mode 100644 index 0000000000000000000000000000000000000000..47d34112ecdd4d931e7f773be89c6cd87ea14bd3 --- /dev/null +++ b/model-00095-of-00101.safetensors @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:09059a7a6736705522b39c2fb3f49c72a766258341dcfafec6f00305778b06f3 +size 5354301312 diff --git a/model-00096-of-00101.safetensors b/model-00096-of-00101.safetensors new file mode 100644 index 0000000000000000000000000000000000000000..089cd9d7134857cd6c962c3bb5bed3c720e57294 --- /dev/null +++ b/model-00096-of-00101.safetensors @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:274b262829c3bcacb7fef0da9319d4df74b3743b3e728078477d0195552ff705 +size 5354301312 diff --git a/model-00097-of-00101.safetensors b/model-00097-of-00101.safetensors new file mode 100644 index 0000000000000000000000000000000000000000..1b288805aaa0e28297a0e3f793fa683c3281d9ab --- /dev/null +++ b/model-00097-of-00101.safetensors @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:f33b2944b925cdafbbe6dd5851d4d718f7b44991106ec0e749ab5de71dd52ebf +size 5354301320 diff --git a/model-00098-of-00101.safetensors b/model-00098-of-00101.safetensors new file mode 100644 index 0000000000000000000000000000000000000000..1ab2ee356b39412425fbfa3b61ef917793fd6abf --- /dev/null +++ b/model-00098-of-00101.safetensors @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:9784411dcc6e0de785d4c7de6ffd8c10f9cb743c167536991a96f4c6199a8b48 +size 5354301320 diff --git a/model-00099-of-00101.safetensors b/model-00099-of-00101.safetensors new file mode 100644 index 0000000000000000000000000000000000000000..53bf6562601c16ca048eeef27f6f875113133d2c --- /dev/null +++ b/model-00099-of-00101.safetensors @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:181a66c76400d2c2cc5188e0b4773ccb4c9fd8b5ed5b3a266562b1ab09dad135 +size 5333301608 diff --git a/model-00100-of-00101.safetensors b/model-00100-of-00101.safetensors new file mode 100644 index 0000000000000000000000000000000000000000..bdc35260667c26937cdbdbac826244146404192e --- /dev/null +++ b/model-00100-of-00101.safetensors @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:e5583549aac4d2df3a9c713ec03b28bb90e85fcfd00ac5136f6656b87345a469 +size 5353051064 diff --git a/model-00101-of-00101.safetensors b/model-00101-of-00101.safetensors new file mode 100644 index 0000000000000000000000000000000000000000..374a201f1459646295f2e3f32b50d3b0e7def1c2 --- /dev/null +++ b/model-00101-of-00101.safetensors @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:d6d7a638b13754450ea195b62f0453e0dc004d906c53e2428eca4eba25ec49b4 +size 2182303808 diff --git a/model.safetensors.index.json b/model.safetensors.index.json new file mode 100644 index 0000000000000000000000000000000000000000..b15c5622d14f0222f2145bcc4ba52dcc602e5495 --- /dev/null +++ b/model.safetensors.index.json @@ -0,0 +1,33516 @@ +{ + "metadata": { + "total_size": 537577342688 + }, + "weight_map": { + "model.embed_tokens.weight": "model-00001-of-00101.safetensors", + "model.layers.0.self_attn.q_proj.weight": "model-00001-of-00101.safetensors", + "model.layers.0.self_attn.q_proj.bias": "model-00001-of-00101.safetensors", + "model.layers.0.self_attn.k_proj.weight": "model-00001-of-00101.safetensors", + "model.layers.0.self_attn.k_proj.bias": "model-00001-of-00101.safetensors", + "model.layers.0.self_attn.v_proj.weight": "model-00001-of-00101.safetensors", + "model.layers.0.self_attn.v_proj.bias": "model-00001-of-00101.safetensors", + "model.layers.0.self_attn.o_proj.weight": "model-00001-of-00101.safetensors", + "model.layers.0.self_attn.q_norm.weight": "model-00001-of-00101.safetensors", + "model.layers.0.self_attn.k_norm.weight": "model-00001-of-00101.safetensors", + "model.layers.0.mlp.gate_proj.weight": "model-00001-of-00101.safetensors", + "model.layers.0.mlp.up_proj.weight": "model-00001-of-00101.safetensors", + "model.layers.0.mlp.down_proj.weight": "model-00001-of-00101.safetensors", + "model.layers.0.input_layernorm.weight": "model-00001-of-00101.safetensors", + "model.layers.0.post_attention_layernorm.weight": "model-00001-of-00101.safetensors", + "model.layers.1.self_attn.q_proj.weight": "model-00001-of-00101.safetensors", + "model.layers.1.self_attn.q_proj.bias": "model-00001-of-00101.safetensors", + "model.layers.1.self_attn.k_proj.weight": "model-00001-of-00101.safetensors", + "model.layers.1.self_attn.k_proj.bias": "model-00001-of-00101.safetensors", + "model.layers.1.self_attn.v_proj.weight": "model-00001-of-00101.safetensors", + "model.layers.1.self_attn.v_proj.bias": "model-00001-of-00101.safetensors", + "model.layers.1.self_attn.o_proj.weight": "model-00001-of-00101.safetensors", + "model.layers.1.self_attn.q_norm.weight": "model-00001-of-00101.safetensors", + "model.layers.1.self_attn.k_norm.weight": "model-00001-of-00101.safetensors", + "model.layers.1.mlp.gate_proj.weight": "model-00001-of-00101.safetensors", + "model.layers.1.mlp.up_proj.weight": "model-00001-of-00101.safetensors", + "model.layers.1.mlp.down_proj.weight": "model-00001-of-00101.safetensors", + "model.layers.1.input_layernorm.weight": "model-00001-of-00101.safetensors", + "model.layers.1.post_attention_layernorm.weight": "model-00001-of-00101.safetensors", + "model.layers.2.self_attn.q_proj.weight": "model-00001-of-00101.safetensors", + "model.layers.2.self_attn.q_proj.bias": "model-00001-of-00101.safetensors", + "model.layers.2.self_attn.k_proj.weight": "model-00001-of-00101.safetensors", + "model.layers.2.self_attn.k_proj.bias": "model-00001-of-00101.safetensors", + "model.layers.2.self_attn.v_proj.weight": "model-00001-of-00101.safetensors", + "model.layers.2.self_attn.v_proj.bias": "model-00001-of-00101.safetensors", + "model.layers.2.self_attn.o_proj.weight": "model-00001-of-00101.safetensors", + "model.layers.2.self_attn.q_norm.weight": "model-00001-of-00101.safetensors", + "model.layers.2.self_attn.k_norm.weight": "model-00001-of-00101.safetensors", + "model.layers.2.mlp.gate_proj.weight": "model-00001-of-00101.safetensors", + "model.layers.2.mlp.up_proj.weight": "model-00001-of-00101.safetensors", + "model.layers.2.mlp.down_proj.weight": "model-00001-of-00101.safetensors", + "model.layers.2.input_layernorm.weight": "model-00001-of-00101.safetensors", + "model.layers.2.post_attention_layernorm.weight": "model-00001-of-00101.safetensors", + "model.layers.3.self_attn.q_proj.weight": "model-00001-of-00101.safetensors", + "model.layers.3.self_attn.q_proj.bias": "model-00001-of-00101.safetensors", + "model.layers.3.self_attn.k_proj.weight": "model-00001-of-00101.safetensors", + "model.layers.3.self_attn.k_proj.bias": "model-00001-of-00101.safetensors", + "model.layers.3.self_attn.v_proj.weight": "model-00001-of-00101.safetensors", + "model.layers.3.self_attn.v_proj.bias": "model-00001-of-00101.safetensors", + "model.layers.3.self_attn.o_proj.weight": "model-00001-of-00101.safetensors", + "model.layers.3.self_attn.q_norm.weight": "model-00001-of-00101.safetensors", + "model.layers.3.self_attn.k_norm.weight": "model-00001-of-00101.safetensors", + "model.layers.3.mlp.experts.0.gate_proj.weight": "model-00001-of-00101.safetensors", + "model.layers.3.mlp.experts.0.up_proj.weight": "model-00001-of-00101.safetensors", + "model.layers.3.mlp.experts.0.down_proj.weight": "model-00001-of-00101.safetensors", + "model.layers.3.mlp.experts.1.gate_proj.weight": "model-00001-of-00101.safetensors", + "model.layers.3.mlp.experts.1.up_proj.weight": "model-00001-of-00101.safetensors", + "model.layers.3.mlp.experts.1.down_proj.weight": "model-00001-of-00101.safetensors", + "model.layers.3.mlp.experts.2.gate_proj.weight": "model-00001-of-00101.safetensors", + "model.layers.3.mlp.experts.2.up_proj.weight": "model-00001-of-00101.safetensors", + "model.layers.3.mlp.experts.2.down_proj.weight": "model-00001-of-00101.safetensors", + "model.layers.3.mlp.experts.3.gate_proj.weight": "model-00001-of-00101.safetensors", + "model.layers.3.mlp.experts.3.up_proj.weight": "model-00001-of-00101.safetensors", + "model.layers.3.mlp.experts.3.down_proj.weight": "model-00001-of-00101.safetensors", + "model.layers.3.mlp.experts.4.gate_proj.weight": "model-00001-of-00101.safetensors", + "model.layers.3.mlp.experts.4.up_proj.weight": "model-00001-of-00101.safetensors", + "model.layers.3.mlp.experts.4.down_proj.weight": "model-00001-of-00101.safetensors", + "model.layers.3.mlp.experts.5.gate_proj.weight": "model-00001-of-00101.safetensors", + "model.layers.3.mlp.experts.5.up_proj.weight": "model-00001-of-00101.safetensors", + "model.layers.3.mlp.experts.5.down_proj.weight": "model-00001-of-00101.safetensors", + "model.layers.3.mlp.experts.6.gate_proj.weight": "model-00001-of-00101.safetensors", + "model.layers.3.mlp.experts.6.up_proj.weight": "model-00001-of-00101.safetensors", + "model.layers.3.mlp.experts.6.down_proj.weight": "model-00001-of-00101.safetensors", + "model.layers.3.mlp.experts.7.gate_proj.weight": "model-00001-of-00101.safetensors", + "model.layers.3.mlp.experts.7.up_proj.weight": "model-00001-of-00101.safetensors", + "model.layers.3.mlp.experts.7.down_proj.weight": "model-00001-of-00101.safetensors", + "model.layers.3.mlp.experts.8.gate_proj.weight": "model-00001-of-00101.safetensors", + "model.layers.3.mlp.experts.8.up_proj.weight": "model-00001-of-00101.safetensors", + "model.layers.3.mlp.experts.8.down_proj.weight": "model-00001-of-00101.safetensors", + "model.layers.3.mlp.experts.9.gate_proj.weight": "model-00001-of-00101.safetensors", + "model.layers.3.mlp.experts.9.up_proj.weight": "model-00001-of-00101.safetensors", + "model.layers.3.mlp.experts.9.down_proj.weight": "model-00001-of-00101.safetensors", + "model.layers.3.mlp.experts.10.gate_proj.weight": "model-00001-of-00101.safetensors", + "model.layers.3.mlp.experts.10.up_proj.weight": "model-00001-of-00101.safetensors", + "model.layers.3.mlp.experts.10.down_proj.weight": "model-00001-of-00101.safetensors", + "model.layers.3.mlp.experts.11.gate_proj.weight": "model-00001-of-00101.safetensors", + "model.layers.3.mlp.experts.11.up_proj.weight": "model-00001-of-00101.safetensors", + "model.layers.3.mlp.experts.11.down_proj.weight": "model-00001-of-00101.safetensors", + "model.layers.3.mlp.experts.12.gate_proj.weight": "model-00001-of-00101.safetensors", + "model.layers.3.mlp.experts.12.up_proj.weight": "model-00001-of-00101.safetensors", + "model.layers.3.mlp.experts.12.down_proj.weight": "model-00001-of-00101.safetensors", + "model.layers.3.mlp.experts.13.gate_proj.weight": "model-00001-of-00101.safetensors", + "model.layers.3.mlp.experts.13.up_proj.weight": "model-00001-of-00101.safetensors", + "model.layers.3.mlp.experts.13.down_proj.weight": "model-00001-of-00101.safetensors", + "model.layers.3.mlp.experts.14.gate_proj.weight": "model-00001-of-00101.safetensors", + "model.layers.3.mlp.experts.14.up_proj.weight": "model-00001-of-00101.safetensors", + "model.layers.3.mlp.experts.14.down_proj.weight": "model-00001-of-00101.safetensors", + "model.layers.3.mlp.experts.15.gate_proj.weight": "model-00001-of-00101.safetensors", + "model.layers.3.mlp.experts.15.up_proj.weight": "model-00001-of-00101.safetensors", + "model.layers.3.mlp.experts.15.down_proj.weight": "model-00001-of-00101.safetensors", + "model.layers.3.mlp.experts.16.gate_proj.weight": "model-00001-of-00101.safetensors", + "model.layers.3.mlp.experts.16.up_proj.weight": "model-00001-of-00101.safetensors", + "model.layers.3.mlp.experts.16.down_proj.weight": "model-00001-of-00101.safetensors", + "model.layers.3.mlp.experts.17.gate_proj.weight": "model-00001-of-00101.safetensors", + "model.layers.3.mlp.experts.17.up_proj.weight": "model-00001-of-00101.safetensors", + "model.layers.3.mlp.experts.17.down_proj.weight": "model-00001-of-00101.safetensors", + "model.layers.3.mlp.experts.18.gate_proj.weight": "model-00001-of-00101.safetensors", + "model.layers.3.mlp.experts.18.up_proj.weight": "model-00001-of-00101.safetensors", + "model.layers.3.mlp.experts.18.down_proj.weight": "model-00001-of-00101.safetensors", + "model.layers.3.mlp.experts.19.gate_proj.weight": "model-00001-of-00101.safetensors", + "model.layers.3.mlp.experts.19.up_proj.weight": "model-00001-of-00101.safetensors", + "model.layers.3.mlp.experts.19.down_proj.weight": "model-00001-of-00101.safetensors", + "model.layers.3.mlp.experts.20.gate_proj.weight": "model-00001-of-00101.safetensors", + "model.layers.3.mlp.experts.20.up_proj.weight": "model-00001-of-00101.safetensors", + "model.layers.3.mlp.experts.20.down_proj.weight": "model-00001-of-00101.safetensors", + "model.layers.3.mlp.experts.21.gate_proj.weight": "model-00001-of-00101.safetensors", + "model.layers.3.mlp.experts.21.up_proj.weight": "model-00001-of-00101.safetensors", + "model.layers.3.mlp.experts.21.down_proj.weight": "model-00001-of-00101.safetensors", + "model.layers.3.mlp.experts.22.gate_proj.weight": "model-00001-of-00101.safetensors", + "model.layers.3.mlp.experts.22.up_proj.weight": "model-00001-of-00101.safetensors", + "model.layers.3.mlp.experts.22.down_proj.weight": "model-00001-of-00101.safetensors", + "model.layers.3.mlp.experts.23.gate_proj.weight": "model-00001-of-00101.safetensors", + "model.layers.3.mlp.experts.23.up_proj.weight": "model-00001-of-00101.safetensors", + "model.layers.3.mlp.experts.23.down_proj.weight": "model-00001-of-00101.safetensors", + "model.layers.3.mlp.experts.24.gate_proj.weight": "model-00001-of-00101.safetensors", + "model.layers.3.mlp.experts.24.up_proj.weight": "model-00001-of-00101.safetensors", + "model.layers.3.mlp.experts.24.down_proj.weight": "model-00001-of-00101.safetensors", + "model.layers.3.mlp.experts.25.gate_proj.weight": "model-00001-of-00101.safetensors", + "model.layers.3.mlp.experts.25.up_proj.weight": "model-00001-of-00101.safetensors", + "model.layers.3.mlp.experts.25.down_proj.weight": "model-00001-of-00101.safetensors", + "model.layers.3.mlp.experts.26.gate_proj.weight": "model-00001-of-00101.safetensors", + "model.layers.3.mlp.experts.26.up_proj.weight": "model-00001-of-00101.safetensors", + "model.layers.3.mlp.experts.26.down_proj.weight": "model-00001-of-00101.safetensors", + "model.layers.3.mlp.experts.27.gate_proj.weight": "model-00001-of-00101.safetensors", + "model.layers.3.mlp.experts.27.up_proj.weight": "model-00001-of-00101.safetensors", + "model.layers.3.mlp.experts.27.down_proj.weight": "model-00001-of-00101.safetensors", + "model.layers.3.mlp.experts.28.gate_proj.weight": "model-00001-of-00101.safetensors", + "model.layers.3.mlp.experts.28.up_proj.weight": "model-00001-of-00101.safetensors", + "model.layers.3.mlp.experts.28.down_proj.weight": "model-00001-of-00101.safetensors", + "model.layers.3.mlp.experts.29.gate_proj.weight": "model-00001-of-00101.safetensors", + "model.layers.3.mlp.experts.29.up_proj.weight": "model-00001-of-00101.safetensors", + "model.layers.3.mlp.experts.29.down_proj.weight": "model-00001-of-00101.safetensors", + "model.layers.3.mlp.experts.30.gate_proj.weight": "model-00001-of-00101.safetensors", + "model.layers.3.mlp.experts.30.up_proj.weight": "model-00001-of-00101.safetensors", + "model.layers.3.mlp.experts.30.down_proj.weight": "model-00001-of-00101.safetensors", + "model.layers.3.mlp.experts.31.gate_proj.weight": "model-00001-of-00101.safetensors", + "model.layers.3.mlp.experts.31.up_proj.weight": "model-00001-of-00101.safetensors", + "model.layers.3.mlp.experts.31.down_proj.weight": "model-00001-of-00101.safetensors", + "model.layers.3.mlp.experts.32.gate_proj.weight": "model-00001-of-00101.safetensors", + "model.layers.3.mlp.experts.32.up_proj.weight": "model-00001-of-00101.safetensors", + "model.layers.3.mlp.experts.32.down_proj.weight": "model-00001-of-00101.safetensors", + "model.layers.3.mlp.experts.33.gate_proj.weight": "model-00001-of-00101.safetensors", + "model.layers.3.mlp.experts.33.up_proj.weight": "model-00001-of-00101.safetensors", + "model.layers.3.mlp.experts.33.down_proj.weight": "model-00002-of-00101.safetensors", + "model.layers.3.mlp.experts.34.gate_proj.weight": "model-00002-of-00101.safetensors", + "model.layers.3.mlp.experts.34.up_proj.weight": "model-00002-of-00101.safetensors", + "model.layers.3.mlp.experts.34.down_proj.weight": "model-00002-of-00101.safetensors", + "model.layers.3.mlp.experts.35.gate_proj.weight": "model-00002-of-00101.safetensors", + "model.layers.3.mlp.experts.35.up_proj.weight": "model-00002-of-00101.safetensors", + "model.layers.3.mlp.experts.35.down_proj.weight": "model-00002-of-00101.safetensors", + "model.layers.3.mlp.experts.36.gate_proj.weight": "model-00002-of-00101.safetensors", + "model.layers.3.mlp.experts.36.up_proj.weight": "model-00002-of-00101.safetensors", + "model.layers.3.mlp.experts.36.down_proj.weight": "model-00002-of-00101.safetensors", + "model.layers.3.mlp.experts.37.gate_proj.weight": "model-00002-of-00101.safetensors", + "model.layers.3.mlp.experts.37.up_proj.weight": "model-00002-of-00101.safetensors", + "model.layers.3.mlp.experts.37.down_proj.weight": "model-00002-of-00101.safetensors", + "model.layers.3.mlp.experts.38.gate_proj.weight": "model-00002-of-00101.safetensors", + "model.layers.3.mlp.experts.38.up_proj.weight": "model-00002-of-00101.safetensors", + "model.layers.3.mlp.experts.38.down_proj.weight": "model-00002-of-00101.safetensors", + "model.layers.3.mlp.experts.39.gate_proj.weight": "model-00002-of-00101.safetensors", + "model.layers.3.mlp.experts.39.up_proj.weight": "model-00002-of-00101.safetensors", + "model.layers.3.mlp.experts.39.down_proj.weight": "model-00002-of-00101.safetensors", + "model.layers.3.mlp.experts.40.gate_proj.weight": "model-00002-of-00101.safetensors", + "model.layers.3.mlp.experts.40.up_proj.weight": "model-00002-of-00101.safetensors", + "model.layers.3.mlp.experts.40.down_proj.weight": "model-00002-of-00101.safetensors", + "model.layers.3.mlp.experts.41.gate_proj.weight": "model-00002-of-00101.safetensors", + "model.layers.3.mlp.experts.41.up_proj.weight": "model-00002-of-00101.safetensors", + "model.layers.3.mlp.experts.41.down_proj.weight": "model-00002-of-00101.safetensors", + "model.layers.3.mlp.experts.42.gate_proj.weight": "model-00002-of-00101.safetensors", + "model.layers.3.mlp.experts.42.up_proj.weight": "model-00002-of-00101.safetensors", + "model.layers.3.mlp.experts.42.down_proj.weight": "model-00002-of-00101.safetensors", + "model.layers.3.mlp.experts.43.gate_proj.weight": "model-00002-of-00101.safetensors", + "model.layers.3.mlp.experts.43.up_proj.weight": "model-00002-of-00101.safetensors", + "model.layers.3.mlp.experts.43.down_proj.weight": "model-00002-of-00101.safetensors", + "model.layers.3.mlp.experts.44.gate_proj.weight": "model-00002-of-00101.safetensors", + "model.layers.3.mlp.experts.44.up_proj.weight": "model-00002-of-00101.safetensors", + "model.layers.3.mlp.experts.44.down_proj.weight": "model-00002-of-00101.safetensors", + "model.layers.3.mlp.experts.45.gate_proj.weight": "model-00002-of-00101.safetensors", + "model.layers.3.mlp.experts.45.up_proj.weight": "model-00002-of-00101.safetensors", + "model.layers.3.mlp.experts.45.down_proj.weight": "model-00002-of-00101.safetensors", + "model.layers.3.mlp.experts.46.gate_proj.weight": "model-00002-of-00101.safetensors", + "model.layers.3.mlp.experts.46.up_proj.weight": "model-00002-of-00101.safetensors", + "model.layers.3.mlp.experts.46.down_proj.weight": "model-00002-of-00101.safetensors", + "model.layers.3.mlp.experts.47.gate_proj.weight": "model-00002-of-00101.safetensors", + "model.layers.3.mlp.experts.47.up_proj.weight": "model-00002-of-00101.safetensors", + "model.layers.3.mlp.experts.47.down_proj.weight": "model-00002-of-00101.safetensors", + "model.layers.3.mlp.experts.48.gate_proj.weight": "model-00002-of-00101.safetensors", + "model.layers.3.mlp.experts.48.up_proj.weight": "model-00002-of-00101.safetensors", + "model.layers.3.mlp.experts.48.down_proj.weight": "model-00002-of-00101.safetensors", + "model.layers.3.mlp.experts.49.gate_proj.weight": "model-00002-of-00101.safetensors", + "model.layers.3.mlp.experts.49.up_proj.weight": "model-00002-of-00101.safetensors", + "model.layers.3.mlp.experts.49.down_proj.weight": "model-00002-of-00101.safetensors", + "model.layers.3.mlp.experts.50.gate_proj.weight": "model-00002-of-00101.safetensors", + "model.layers.3.mlp.experts.50.up_proj.weight": "model-00002-of-00101.safetensors", + "model.layers.3.mlp.experts.50.down_proj.weight": "model-00002-of-00101.safetensors", + "model.layers.3.mlp.experts.51.gate_proj.weight": "model-00002-of-00101.safetensors", + "model.layers.3.mlp.experts.51.up_proj.weight": "model-00002-of-00101.safetensors", + "model.layers.3.mlp.experts.51.down_proj.weight": "model-00002-of-00101.safetensors", + "model.layers.3.mlp.experts.52.gate_proj.weight": "model-00002-of-00101.safetensors", + "model.layers.3.mlp.experts.52.up_proj.weight": "model-00002-of-00101.safetensors", + "model.layers.3.mlp.experts.52.down_proj.weight": "model-00002-of-00101.safetensors", + "model.layers.3.mlp.experts.53.gate_proj.weight": "model-00002-of-00101.safetensors", + "model.layers.3.mlp.experts.53.up_proj.weight": "model-00002-of-00101.safetensors", + "model.layers.3.mlp.experts.53.down_proj.weight": "model-00002-of-00101.safetensors", + "model.layers.3.mlp.experts.54.gate_proj.weight": "model-00002-of-00101.safetensors", + "model.layers.3.mlp.experts.54.up_proj.weight": "model-00002-of-00101.safetensors", + "model.layers.3.mlp.experts.54.down_proj.weight": "model-00002-of-00101.safetensors", + "model.layers.3.mlp.experts.55.gate_proj.weight": "model-00002-of-00101.safetensors", + "model.layers.3.mlp.experts.55.up_proj.weight": "model-00002-of-00101.safetensors", + "model.layers.3.mlp.experts.55.down_proj.weight": "model-00002-of-00101.safetensors", + "model.layers.3.mlp.experts.56.gate_proj.weight": "model-00002-of-00101.safetensors", + "model.layers.3.mlp.experts.56.up_proj.weight": "model-00002-of-00101.safetensors", + "model.layers.3.mlp.experts.56.down_proj.weight": "model-00002-of-00101.safetensors", + "model.layers.3.mlp.experts.57.gate_proj.weight": "model-00002-of-00101.safetensors", + "model.layers.3.mlp.experts.57.up_proj.weight": "model-00002-of-00101.safetensors", + "model.layers.3.mlp.experts.57.down_proj.weight": "model-00002-of-00101.safetensors", + "model.layers.3.mlp.experts.58.gate_proj.weight": "model-00002-of-00101.safetensors", + "model.layers.3.mlp.experts.58.up_proj.weight": "model-00002-of-00101.safetensors", + "model.layers.3.mlp.experts.58.down_proj.weight": "model-00002-of-00101.safetensors", + "model.layers.3.mlp.experts.59.gate_proj.weight": "model-00002-of-00101.safetensors", + "model.layers.3.mlp.experts.59.up_proj.weight": "model-00002-of-00101.safetensors", + "model.layers.3.mlp.experts.59.down_proj.weight": "model-00002-of-00101.safetensors", + "model.layers.3.mlp.experts.60.gate_proj.weight": "model-00002-of-00101.safetensors", + "model.layers.3.mlp.experts.60.up_proj.weight": "model-00002-of-00101.safetensors", + "model.layers.3.mlp.experts.60.down_proj.weight": "model-00002-of-00101.safetensors", + "model.layers.3.mlp.experts.61.gate_proj.weight": "model-00002-of-00101.safetensors", + "model.layers.3.mlp.experts.61.up_proj.weight": "model-00002-of-00101.safetensors", + "model.layers.3.mlp.experts.61.down_proj.weight": "model-00002-of-00101.safetensors", + "model.layers.3.mlp.experts.62.gate_proj.weight": "model-00002-of-00101.safetensors", + "model.layers.3.mlp.experts.62.up_proj.weight": "model-00002-of-00101.safetensors", + "model.layers.3.mlp.experts.62.down_proj.weight": "model-00002-of-00101.safetensors", + "model.layers.3.mlp.experts.63.gate_proj.weight": "model-00002-of-00101.safetensors", + "model.layers.3.mlp.experts.63.up_proj.weight": "model-00002-of-00101.safetensors", + "model.layers.3.mlp.experts.63.down_proj.weight": "model-00002-of-00101.safetensors", + "model.layers.3.mlp.experts.64.gate_proj.weight": "model-00002-of-00101.safetensors", + "model.layers.3.mlp.experts.64.up_proj.weight": "model-00002-of-00101.safetensors", + "model.layers.3.mlp.experts.64.down_proj.weight": "model-00002-of-00101.safetensors", + "model.layers.3.mlp.experts.65.gate_proj.weight": "model-00002-of-00101.safetensors", + "model.layers.3.mlp.experts.65.up_proj.weight": "model-00002-of-00101.safetensors", + "model.layers.3.mlp.experts.65.down_proj.weight": "model-00002-of-00101.safetensors", + "model.layers.3.mlp.experts.66.gate_proj.weight": "model-00002-of-00101.safetensors", + "model.layers.3.mlp.experts.66.up_proj.weight": "model-00002-of-00101.safetensors", + "model.layers.3.mlp.experts.66.down_proj.weight": "model-00002-of-00101.safetensors", + "model.layers.3.mlp.experts.67.gate_proj.weight": "model-00002-of-00101.safetensors", + "model.layers.3.mlp.experts.67.up_proj.weight": "model-00002-of-00101.safetensors", + "model.layers.3.mlp.experts.67.down_proj.weight": "model-00002-of-00101.safetensors", + "model.layers.3.mlp.experts.68.gate_proj.weight": "model-00002-of-00101.safetensors", + "model.layers.3.mlp.experts.68.up_proj.weight": "model-00002-of-00101.safetensors", + "model.layers.3.mlp.experts.68.down_proj.weight": "model-00002-of-00101.safetensors", + "model.layers.3.mlp.experts.69.gate_proj.weight": "model-00002-of-00101.safetensors", + "model.layers.3.mlp.experts.69.up_proj.weight": "model-00002-of-00101.safetensors", + "model.layers.3.mlp.experts.69.down_proj.weight": "model-00002-of-00101.safetensors", + "model.layers.3.mlp.experts.70.gate_proj.weight": "model-00002-of-00101.safetensors", + "model.layers.3.mlp.experts.70.up_proj.weight": "model-00002-of-00101.safetensors", + "model.layers.3.mlp.experts.70.down_proj.weight": "model-00002-of-00101.safetensors", + "model.layers.3.mlp.experts.71.gate_proj.weight": "model-00002-of-00101.safetensors", + "model.layers.3.mlp.experts.71.up_proj.weight": "model-00002-of-00101.safetensors", + "model.layers.3.mlp.experts.71.down_proj.weight": "model-00002-of-00101.safetensors", + "model.layers.3.mlp.experts.72.gate_proj.weight": "model-00002-of-00101.safetensors", + "model.layers.3.mlp.experts.72.up_proj.weight": "model-00002-of-00101.safetensors", + "model.layers.3.mlp.experts.72.down_proj.weight": "model-00002-of-00101.safetensors", + "model.layers.3.mlp.experts.73.gate_proj.weight": "model-00002-of-00101.safetensors", + "model.layers.3.mlp.experts.73.up_proj.weight": "model-00002-of-00101.safetensors", + "model.layers.3.mlp.experts.73.down_proj.weight": "model-00002-of-00101.safetensors", + "model.layers.3.mlp.experts.74.gate_proj.weight": "model-00002-of-00101.safetensors", + "model.layers.3.mlp.experts.74.up_proj.weight": "model-00002-of-00101.safetensors", + "model.layers.3.mlp.experts.74.down_proj.weight": "model-00002-of-00101.safetensors", + "model.layers.3.mlp.experts.75.gate_proj.weight": "model-00002-of-00101.safetensors", + "model.layers.3.mlp.experts.75.up_proj.weight": "model-00002-of-00101.safetensors", + "model.layers.3.mlp.experts.75.down_proj.weight": "model-00002-of-00101.safetensors", + "model.layers.3.mlp.experts.76.gate_proj.weight": "model-00002-of-00101.safetensors", + "model.layers.3.mlp.experts.76.up_proj.weight": "model-00002-of-00101.safetensors", + "model.layers.3.mlp.experts.76.down_proj.weight": "model-00002-of-00101.safetensors", + "model.layers.3.mlp.experts.77.gate_proj.weight": "model-00002-of-00101.safetensors", + "model.layers.3.mlp.experts.77.up_proj.weight": "model-00002-of-00101.safetensors", + "model.layers.3.mlp.experts.77.down_proj.weight": "model-00002-of-00101.safetensors", + "model.layers.3.mlp.experts.78.gate_proj.weight": "model-00002-of-00101.safetensors", + "model.layers.3.mlp.experts.78.up_proj.weight": "model-00002-of-00101.safetensors", + "model.layers.3.mlp.experts.78.down_proj.weight": "model-00002-of-00101.safetensors", + "model.layers.3.mlp.experts.79.gate_proj.weight": "model-00002-of-00101.safetensors", + "model.layers.3.mlp.experts.79.up_proj.weight": "model-00002-of-00101.safetensors", + "model.layers.3.mlp.experts.79.down_proj.weight": "model-00002-of-00101.safetensors", + "model.layers.3.mlp.experts.80.gate_proj.weight": "model-00002-of-00101.safetensors", + "model.layers.3.mlp.experts.80.up_proj.weight": "model-00002-of-00101.safetensors", + "model.layers.3.mlp.experts.80.down_proj.weight": "model-00002-of-00101.safetensors", + "model.layers.3.mlp.experts.81.gate_proj.weight": "model-00002-of-00101.safetensors", + "model.layers.3.mlp.experts.81.up_proj.weight": "model-00002-of-00101.safetensors", + "model.layers.3.mlp.experts.81.down_proj.weight": "model-00002-of-00101.safetensors", + "model.layers.3.mlp.experts.82.gate_proj.weight": "model-00002-of-00101.safetensors", + "model.layers.3.mlp.experts.82.up_proj.weight": "model-00002-of-00101.safetensors", + "model.layers.3.mlp.experts.82.down_proj.weight": "model-00002-of-00101.safetensors", + "model.layers.3.mlp.experts.83.gate_proj.weight": "model-00002-of-00101.safetensors", + "model.layers.3.mlp.experts.83.up_proj.weight": "model-00002-of-00101.safetensors", + "model.layers.3.mlp.experts.83.down_proj.weight": "model-00002-of-00101.safetensors", + "model.layers.3.mlp.experts.84.gate_proj.weight": "model-00002-of-00101.safetensors", + "model.layers.3.mlp.experts.84.up_proj.weight": "model-00002-of-00101.safetensors", + "model.layers.3.mlp.experts.84.down_proj.weight": "model-00002-of-00101.safetensors", + "model.layers.3.mlp.experts.85.gate_proj.weight": "model-00002-of-00101.safetensors", + "model.layers.3.mlp.experts.85.up_proj.weight": "model-00002-of-00101.safetensors", + "model.layers.3.mlp.experts.85.down_proj.weight": "model-00002-of-00101.safetensors", + "model.layers.3.mlp.experts.86.gate_proj.weight": "model-00002-of-00101.safetensors", + "model.layers.3.mlp.experts.86.up_proj.weight": "model-00002-of-00101.safetensors", + "model.layers.3.mlp.experts.86.down_proj.weight": "model-00002-of-00101.safetensors", + "model.layers.3.mlp.experts.87.gate_proj.weight": "model-00002-of-00101.safetensors", + "model.layers.3.mlp.experts.87.up_proj.weight": "model-00002-of-00101.safetensors", + "model.layers.3.mlp.experts.87.down_proj.weight": "model-00002-of-00101.safetensors", + "model.layers.3.mlp.experts.88.gate_proj.weight": "model-00002-of-00101.safetensors", + "model.layers.3.mlp.experts.88.up_proj.weight": "model-00002-of-00101.safetensors", + "model.layers.3.mlp.experts.88.down_proj.weight": "model-00002-of-00101.safetensors", + "model.layers.3.mlp.experts.89.gate_proj.weight": "model-00002-of-00101.safetensors", + "model.layers.3.mlp.experts.89.up_proj.weight": "model-00002-of-00101.safetensors", + "model.layers.3.mlp.experts.89.down_proj.weight": "model-00002-of-00101.safetensors", + "model.layers.3.mlp.experts.90.gate_proj.weight": "model-00002-of-00101.safetensors", + "model.layers.3.mlp.experts.90.up_proj.weight": "model-00002-of-00101.safetensors", + "model.layers.3.mlp.experts.90.down_proj.weight": "model-00002-of-00101.safetensors", + "model.layers.3.mlp.experts.91.gate_proj.weight": "model-00002-of-00101.safetensors", + "model.layers.3.mlp.experts.91.up_proj.weight": "model-00002-of-00101.safetensors", + "model.layers.3.mlp.experts.91.down_proj.weight": "model-00002-of-00101.safetensors", + "model.layers.3.mlp.experts.92.gate_proj.weight": "model-00002-of-00101.safetensors", + "model.layers.3.mlp.experts.92.up_proj.weight": "model-00002-of-00101.safetensors", + "model.layers.3.mlp.experts.92.down_proj.weight": "model-00002-of-00101.safetensors", + "model.layers.3.mlp.experts.93.gate_proj.weight": "model-00002-of-00101.safetensors", + "model.layers.3.mlp.experts.93.up_proj.weight": "model-00002-of-00101.safetensors", + "model.layers.3.mlp.experts.93.down_proj.weight": "model-00002-of-00101.safetensors", + "model.layers.3.mlp.experts.94.gate_proj.weight": "model-00002-of-00101.safetensors", + "model.layers.3.mlp.experts.94.up_proj.weight": "model-00002-of-00101.safetensors", + "model.layers.3.mlp.experts.94.down_proj.weight": "model-00002-of-00101.safetensors", + "model.layers.3.mlp.experts.95.gate_proj.weight": "model-00002-of-00101.safetensors", + "model.layers.3.mlp.experts.95.up_proj.weight": "model-00002-of-00101.safetensors", + "model.layers.3.mlp.experts.95.down_proj.weight": "model-00002-of-00101.safetensors", + "model.layers.3.mlp.experts.96.gate_proj.weight": "model-00002-of-00101.safetensors", + "model.layers.3.mlp.experts.96.up_proj.weight": "model-00002-of-00101.safetensors", + "model.layers.3.mlp.experts.96.down_proj.weight": "model-00002-of-00101.safetensors", + "model.layers.3.mlp.experts.97.gate_proj.weight": "model-00002-of-00101.safetensors", + "model.layers.3.mlp.experts.97.up_proj.weight": "model-00002-of-00101.safetensors", + "model.layers.3.mlp.experts.97.down_proj.weight": "model-00002-of-00101.safetensors", + "model.layers.3.mlp.experts.98.gate_proj.weight": "model-00002-of-00101.safetensors", + "model.layers.3.mlp.experts.98.up_proj.weight": "model-00002-of-00101.safetensors", + "model.layers.3.mlp.experts.98.down_proj.weight": "model-00002-of-00101.safetensors", + "model.layers.3.mlp.experts.99.gate_proj.weight": "model-00002-of-00101.safetensors", + "model.layers.3.mlp.experts.99.up_proj.weight": "model-00002-of-00101.safetensors", + "model.layers.3.mlp.experts.99.down_proj.weight": "model-00002-of-00101.safetensors", + "model.layers.3.mlp.experts.100.gate_proj.weight": "model-00002-of-00101.safetensors", + "model.layers.3.mlp.experts.100.up_proj.weight": "model-00002-of-00101.safetensors", + "model.layers.3.mlp.experts.100.down_proj.weight": "model-00002-of-00101.safetensors", + "model.layers.3.mlp.experts.101.gate_proj.weight": "model-00002-of-00101.safetensors", + "model.layers.3.mlp.experts.101.up_proj.weight": "model-00002-of-00101.safetensors", + "model.layers.3.mlp.experts.101.down_proj.weight": "model-00002-of-00101.safetensors", + "model.layers.3.mlp.experts.102.gate_proj.weight": "model-00002-of-00101.safetensors", + "model.layers.3.mlp.experts.102.up_proj.weight": "model-00002-of-00101.safetensors", + "model.layers.3.mlp.experts.102.down_proj.weight": "model-00002-of-00101.safetensors", + "model.layers.3.mlp.experts.103.gate_proj.weight": "model-00002-of-00101.safetensors", + "model.layers.3.mlp.experts.103.up_proj.weight": "model-00002-of-00101.safetensors", + "model.layers.3.mlp.experts.103.down_proj.weight": "model-00002-of-00101.safetensors", + "model.layers.3.mlp.experts.104.gate_proj.weight": "model-00002-of-00101.safetensors", + "model.layers.3.mlp.experts.104.up_proj.weight": "model-00002-of-00101.safetensors", + "model.layers.3.mlp.experts.104.down_proj.weight": "model-00002-of-00101.safetensors", + "model.layers.3.mlp.experts.105.gate_proj.weight": "model-00002-of-00101.safetensors", + "model.layers.3.mlp.experts.105.up_proj.weight": "model-00002-of-00101.safetensors", + "model.layers.3.mlp.experts.105.down_proj.weight": "model-00002-of-00101.safetensors", + "model.layers.3.mlp.experts.106.gate_proj.weight": "model-00002-of-00101.safetensors", + "model.layers.3.mlp.experts.106.up_proj.weight": "model-00002-of-00101.safetensors", + "model.layers.3.mlp.experts.106.down_proj.weight": "model-00002-of-00101.safetensors", + "model.layers.3.mlp.experts.107.gate_proj.weight": "model-00002-of-00101.safetensors", + "model.layers.3.mlp.experts.107.up_proj.weight": "model-00002-of-00101.safetensors", + "model.layers.3.mlp.experts.107.down_proj.weight": "model-00002-of-00101.safetensors", + "model.layers.3.mlp.experts.108.gate_proj.weight": "model-00002-of-00101.safetensors", + "model.layers.3.mlp.experts.108.up_proj.weight": "model-00002-of-00101.safetensors", + "model.layers.3.mlp.experts.108.down_proj.weight": "model-00002-of-00101.safetensors", + "model.layers.3.mlp.experts.109.gate_proj.weight": "model-00002-of-00101.safetensors", + "model.layers.3.mlp.experts.109.up_proj.weight": "model-00002-of-00101.safetensors", + "model.layers.3.mlp.experts.109.down_proj.weight": "model-00002-of-00101.safetensors", + "model.layers.3.mlp.experts.110.gate_proj.weight": "model-00002-of-00101.safetensors", + "model.layers.3.mlp.experts.110.up_proj.weight": "model-00002-of-00101.safetensors", + "model.layers.3.mlp.experts.110.down_proj.weight": "model-00002-of-00101.safetensors", + "model.layers.3.mlp.experts.111.gate_proj.weight": "model-00002-of-00101.safetensors", + "model.layers.3.mlp.experts.111.up_proj.weight": "model-00002-of-00101.safetensors", + "model.layers.3.mlp.experts.111.down_proj.weight": "model-00002-of-00101.safetensors", + "model.layers.3.mlp.experts.112.gate_proj.weight": "model-00002-of-00101.safetensors", + "model.layers.3.mlp.experts.112.up_proj.weight": "model-00002-of-00101.safetensors", + "model.layers.3.mlp.experts.112.down_proj.weight": "model-00002-of-00101.safetensors", + "model.layers.3.mlp.experts.113.gate_proj.weight": "model-00002-of-00101.safetensors", + "model.layers.3.mlp.experts.113.up_proj.weight": "model-00002-of-00101.safetensors", + "model.layers.3.mlp.experts.113.down_proj.weight": "model-00002-of-00101.safetensors", + "model.layers.3.mlp.experts.114.gate_proj.weight": "model-00002-of-00101.safetensors", + "model.layers.3.mlp.experts.114.up_proj.weight": "model-00002-of-00101.safetensors", + "model.layers.3.mlp.experts.114.down_proj.weight": "model-00002-of-00101.safetensors", + "model.layers.3.mlp.experts.115.gate_proj.weight": "model-00002-of-00101.safetensors", + "model.layers.3.mlp.experts.115.up_proj.weight": "model-00002-of-00101.safetensors", + "model.layers.3.mlp.experts.115.down_proj.weight": "model-00002-of-00101.safetensors", + "model.layers.3.mlp.experts.116.gate_proj.weight": "model-00002-of-00101.safetensors", + "model.layers.3.mlp.experts.116.up_proj.weight": "model-00002-of-00101.safetensors", + "model.layers.3.mlp.experts.116.down_proj.weight": "model-00002-of-00101.safetensors", + "model.layers.3.mlp.experts.117.gate_proj.weight": "model-00002-of-00101.safetensors", + "model.layers.3.mlp.experts.117.up_proj.weight": "model-00002-of-00101.safetensors", + "model.layers.3.mlp.experts.117.down_proj.weight": "model-00002-of-00101.safetensors", + "model.layers.3.mlp.experts.118.gate_proj.weight": "model-00002-of-00101.safetensors", + "model.layers.3.mlp.experts.118.up_proj.weight": "model-00002-of-00101.safetensors", + "model.layers.3.mlp.experts.118.down_proj.weight": "model-00002-of-00101.safetensors", + "model.layers.3.mlp.experts.119.gate_proj.weight": "model-00002-of-00101.safetensors", + "model.layers.3.mlp.experts.119.up_proj.weight": "model-00002-of-00101.safetensors", + "model.layers.3.mlp.experts.119.down_proj.weight": "model-00002-of-00101.safetensors", + "model.layers.3.mlp.gate.weight": "model-00002-of-00101.safetensors", + "model.layers.3.mlp.gate.e_score_correction_bias": "model-00002-of-00101.safetensors", + "model.layers.3.mlp.shared_experts.gate_proj.weight": "model-00002-of-00101.safetensors", + "model.layers.3.mlp.shared_experts.up_proj.weight": "model-00002-of-00101.safetensors", + "model.layers.3.mlp.shared_experts.down_proj.weight": "model-00002-of-00101.safetensors", + "model.layers.3.input_layernorm.weight": "model-00002-of-00101.safetensors", + "model.layers.3.post_attention_layernorm.weight": "model-00002-of-00101.safetensors", + "model.layers.4.self_attn.q_proj.weight": "model-00002-of-00101.safetensors", + "model.layers.4.self_attn.q_proj.bias": "model-00002-of-00101.safetensors", + "model.layers.4.self_attn.k_proj.weight": "model-00002-of-00101.safetensors", + "model.layers.4.self_attn.k_proj.bias": "model-00002-of-00101.safetensors", + "model.layers.4.self_attn.v_proj.weight": "model-00002-of-00101.safetensors", + "model.layers.4.self_attn.v_proj.bias": "model-00002-of-00101.safetensors", + "model.layers.4.self_attn.o_proj.weight": "model-00002-of-00101.safetensors", + "model.layers.4.self_attn.q_norm.weight": "model-00002-of-00101.safetensors", + "model.layers.4.self_attn.k_norm.weight": "model-00002-of-00101.safetensors", + "model.layers.4.mlp.experts.0.gate_proj.weight": "model-00002-of-00101.safetensors", + "model.layers.4.mlp.experts.0.up_proj.weight": "model-00002-of-00101.safetensors", + "model.layers.4.mlp.experts.0.down_proj.weight": "model-00002-of-00101.safetensors", + "model.layers.4.mlp.experts.1.gate_proj.weight": "model-00002-of-00101.safetensors", + "model.layers.4.mlp.experts.1.up_proj.weight": "model-00002-of-00101.safetensors", + "model.layers.4.mlp.experts.1.down_proj.weight": "model-00002-of-00101.safetensors", + "model.layers.4.mlp.experts.2.gate_proj.weight": "model-00002-of-00101.safetensors", + "model.layers.4.mlp.experts.2.up_proj.weight": "model-00002-of-00101.safetensors", + "model.layers.4.mlp.experts.2.down_proj.weight": "model-00002-of-00101.safetensors", + "model.layers.4.mlp.experts.3.gate_proj.weight": "model-00002-of-00101.safetensors", + "model.layers.4.mlp.experts.3.up_proj.weight": "model-00002-of-00101.safetensors", + "model.layers.4.mlp.experts.3.down_proj.weight": "model-00002-of-00101.safetensors", + "model.layers.4.mlp.experts.4.gate_proj.weight": "model-00002-of-00101.safetensors", + "model.layers.4.mlp.experts.4.up_proj.weight": "model-00002-of-00101.safetensors", + "model.layers.4.mlp.experts.4.down_proj.weight": "model-00002-of-00101.safetensors", + "model.layers.4.mlp.experts.5.gate_proj.weight": "model-00002-of-00101.safetensors", + "model.layers.4.mlp.experts.5.up_proj.weight": "model-00002-of-00101.safetensors", + "model.layers.4.mlp.experts.5.down_proj.weight": "model-00002-of-00101.safetensors", + "model.layers.4.mlp.experts.6.gate_proj.weight": "model-00002-of-00101.safetensors", + "model.layers.4.mlp.experts.6.up_proj.weight": "model-00002-of-00101.safetensors", + "model.layers.4.mlp.experts.6.down_proj.weight": "model-00002-of-00101.safetensors", + "model.layers.4.mlp.experts.7.gate_proj.weight": "model-00002-of-00101.safetensors", + "model.layers.4.mlp.experts.7.up_proj.weight": "model-00002-of-00101.safetensors", + "model.layers.4.mlp.experts.7.down_proj.weight": "model-00002-of-00101.safetensors", + "model.layers.4.mlp.experts.8.gate_proj.weight": "model-00002-of-00101.safetensors", + "model.layers.4.mlp.experts.8.up_proj.weight": "model-00002-of-00101.safetensors", + "model.layers.4.mlp.experts.8.down_proj.weight": "model-00002-of-00101.safetensors", + "model.layers.4.mlp.experts.9.gate_proj.weight": "model-00002-of-00101.safetensors", + "model.layers.4.mlp.experts.9.up_proj.weight": "model-00002-of-00101.safetensors", + "model.layers.4.mlp.experts.9.down_proj.weight": "model-00002-of-00101.safetensors", + "model.layers.4.mlp.experts.10.gate_proj.weight": "model-00002-of-00101.safetensors", + "model.layers.4.mlp.experts.10.up_proj.weight": "model-00002-of-00101.safetensors", + "model.layers.4.mlp.experts.10.down_proj.weight": "model-00002-of-00101.safetensors", + "model.layers.4.mlp.experts.11.gate_proj.weight": "model-00002-of-00101.safetensors", + "model.layers.4.mlp.experts.11.up_proj.weight": "model-00002-of-00101.safetensors", + "model.layers.4.mlp.experts.11.down_proj.weight": "model-00002-of-00101.safetensors", + "model.layers.4.mlp.experts.12.gate_proj.weight": "model-00002-of-00101.safetensors", + "model.layers.4.mlp.experts.12.up_proj.weight": "model-00002-of-00101.safetensors", + "model.layers.4.mlp.experts.12.down_proj.weight": "model-00002-of-00101.safetensors", + "model.layers.4.mlp.experts.13.gate_proj.weight": "model-00002-of-00101.safetensors", + "model.layers.4.mlp.experts.13.up_proj.weight": "model-00002-of-00101.safetensors", + "model.layers.4.mlp.experts.13.down_proj.weight": "model-00002-of-00101.safetensors", + "model.layers.4.mlp.experts.14.gate_proj.weight": "model-00002-of-00101.safetensors", + "model.layers.4.mlp.experts.14.up_proj.weight": "model-00002-of-00101.safetensors", + "model.layers.4.mlp.experts.14.down_proj.weight": "model-00002-of-00101.safetensors", + "model.layers.4.mlp.experts.15.gate_proj.weight": "model-00002-of-00101.safetensors", + "model.layers.4.mlp.experts.15.up_proj.weight": "model-00002-of-00101.safetensors", + "model.layers.4.mlp.experts.15.down_proj.weight": "model-00002-of-00101.safetensors", + "model.layers.4.mlp.experts.16.gate_proj.weight": "model-00002-of-00101.safetensors", + "model.layers.4.mlp.experts.16.up_proj.weight": "model-00002-of-00101.safetensors", + "model.layers.4.mlp.experts.16.down_proj.weight": "model-00002-of-00101.safetensors", + "model.layers.4.mlp.experts.17.gate_proj.weight": "model-00002-of-00101.safetensors", + "model.layers.4.mlp.experts.17.up_proj.weight": "model-00002-of-00101.safetensors", + "model.layers.4.mlp.experts.17.down_proj.weight": "model-00002-of-00101.safetensors", + "model.layers.4.mlp.experts.18.gate_proj.weight": "model-00002-of-00101.safetensors", + "model.layers.4.mlp.experts.18.up_proj.weight": "model-00002-of-00101.safetensors", + "model.layers.4.mlp.experts.18.down_proj.weight": "model-00002-of-00101.safetensors", + "model.layers.4.mlp.experts.19.gate_proj.weight": "model-00002-of-00101.safetensors", + "model.layers.4.mlp.experts.19.up_proj.weight": "model-00002-of-00101.safetensors", + "model.layers.4.mlp.experts.19.down_proj.weight": "model-00002-of-00101.safetensors", + "model.layers.4.mlp.experts.20.gate_proj.weight": "model-00002-of-00101.safetensors", + "model.layers.4.mlp.experts.20.up_proj.weight": "model-00003-of-00101.safetensors", + "model.layers.4.mlp.experts.20.down_proj.weight": "model-00003-of-00101.safetensors", + "model.layers.4.mlp.experts.21.gate_proj.weight": "model-00003-of-00101.safetensors", + "model.layers.4.mlp.experts.21.up_proj.weight": "model-00003-of-00101.safetensors", + "model.layers.4.mlp.experts.21.down_proj.weight": "model-00003-of-00101.safetensors", + "model.layers.4.mlp.experts.22.gate_proj.weight": "model-00003-of-00101.safetensors", + "model.layers.4.mlp.experts.22.up_proj.weight": "model-00003-of-00101.safetensors", + "model.layers.4.mlp.experts.22.down_proj.weight": "model-00003-of-00101.safetensors", + "model.layers.4.mlp.experts.23.gate_proj.weight": "model-00003-of-00101.safetensors", + "model.layers.4.mlp.experts.23.up_proj.weight": "model-00003-of-00101.safetensors", + "model.layers.4.mlp.experts.23.down_proj.weight": "model-00003-of-00101.safetensors", + "model.layers.4.mlp.experts.24.gate_proj.weight": "model-00003-of-00101.safetensors", + "model.layers.4.mlp.experts.24.up_proj.weight": "model-00003-of-00101.safetensors", + "model.layers.4.mlp.experts.24.down_proj.weight": "model-00003-of-00101.safetensors", + "model.layers.4.mlp.experts.25.gate_proj.weight": "model-00003-of-00101.safetensors", + "model.layers.4.mlp.experts.25.up_proj.weight": "model-00003-of-00101.safetensors", + "model.layers.4.mlp.experts.25.down_proj.weight": "model-00003-of-00101.safetensors", + "model.layers.4.mlp.experts.26.gate_proj.weight": "model-00003-of-00101.safetensors", + "model.layers.4.mlp.experts.26.up_proj.weight": "model-00003-of-00101.safetensors", + "model.layers.4.mlp.experts.26.down_proj.weight": "model-00003-of-00101.safetensors", + "model.layers.4.mlp.experts.27.gate_proj.weight": "model-00003-of-00101.safetensors", + "model.layers.4.mlp.experts.27.up_proj.weight": "model-00003-of-00101.safetensors", + "model.layers.4.mlp.experts.27.down_proj.weight": "model-00003-of-00101.safetensors", + "model.layers.4.mlp.experts.28.gate_proj.weight": "model-00003-of-00101.safetensors", + "model.layers.4.mlp.experts.28.up_proj.weight": "model-00003-of-00101.safetensors", + "model.layers.4.mlp.experts.28.down_proj.weight": "model-00003-of-00101.safetensors", + "model.layers.4.mlp.experts.29.gate_proj.weight": "model-00003-of-00101.safetensors", + "model.layers.4.mlp.experts.29.up_proj.weight": "model-00003-of-00101.safetensors", + "model.layers.4.mlp.experts.29.down_proj.weight": "model-00003-of-00101.safetensors", + "model.layers.4.mlp.experts.30.gate_proj.weight": "model-00003-of-00101.safetensors", + "model.layers.4.mlp.experts.30.up_proj.weight": "model-00003-of-00101.safetensors", + "model.layers.4.mlp.experts.30.down_proj.weight": "model-00003-of-00101.safetensors", + "model.layers.4.mlp.experts.31.gate_proj.weight": "model-00003-of-00101.safetensors", + "model.layers.4.mlp.experts.31.up_proj.weight": "model-00003-of-00101.safetensors", + "model.layers.4.mlp.experts.31.down_proj.weight": "model-00003-of-00101.safetensors", + "model.layers.4.mlp.experts.32.gate_proj.weight": "model-00003-of-00101.safetensors", + "model.layers.4.mlp.experts.32.up_proj.weight": "model-00003-of-00101.safetensors", + "model.layers.4.mlp.experts.32.down_proj.weight": "model-00003-of-00101.safetensors", + "model.layers.4.mlp.experts.33.gate_proj.weight": "model-00003-of-00101.safetensors", + "model.layers.4.mlp.experts.33.up_proj.weight": "model-00003-of-00101.safetensors", + "model.layers.4.mlp.experts.33.down_proj.weight": "model-00003-of-00101.safetensors", + "model.layers.4.mlp.experts.34.gate_proj.weight": "model-00003-of-00101.safetensors", + "model.layers.4.mlp.experts.34.up_proj.weight": "model-00003-of-00101.safetensors", + "model.layers.4.mlp.experts.34.down_proj.weight": "model-00003-of-00101.safetensors", + "model.layers.4.mlp.experts.35.gate_proj.weight": "model-00003-of-00101.safetensors", + "model.layers.4.mlp.experts.35.up_proj.weight": "model-00003-of-00101.safetensors", + "model.layers.4.mlp.experts.35.down_proj.weight": "model-00003-of-00101.safetensors", + "model.layers.4.mlp.experts.36.gate_proj.weight": "model-00003-of-00101.safetensors", + "model.layers.4.mlp.experts.36.up_proj.weight": "model-00003-of-00101.safetensors", + "model.layers.4.mlp.experts.36.down_proj.weight": "model-00003-of-00101.safetensors", + "model.layers.4.mlp.experts.37.gate_proj.weight": "model-00003-of-00101.safetensors", + "model.layers.4.mlp.experts.37.up_proj.weight": "model-00003-of-00101.safetensors", + "model.layers.4.mlp.experts.37.down_proj.weight": "model-00003-of-00101.safetensors", + "model.layers.4.mlp.experts.38.gate_proj.weight": "model-00003-of-00101.safetensors", + "model.layers.4.mlp.experts.38.up_proj.weight": "model-00003-of-00101.safetensors", + "model.layers.4.mlp.experts.38.down_proj.weight": "model-00003-of-00101.safetensors", + "model.layers.4.mlp.experts.39.gate_proj.weight": "model-00003-of-00101.safetensors", + "model.layers.4.mlp.experts.39.up_proj.weight": "model-00003-of-00101.safetensors", + "model.layers.4.mlp.experts.39.down_proj.weight": "model-00003-of-00101.safetensors", + "model.layers.4.mlp.experts.40.gate_proj.weight": "model-00003-of-00101.safetensors", + "model.layers.4.mlp.experts.40.up_proj.weight": "model-00003-of-00101.safetensors", + "model.layers.4.mlp.experts.40.down_proj.weight": "model-00003-of-00101.safetensors", + "model.layers.4.mlp.experts.41.gate_proj.weight": "model-00003-of-00101.safetensors", + "model.layers.4.mlp.experts.41.up_proj.weight": "model-00003-of-00101.safetensors", + "model.layers.4.mlp.experts.41.down_proj.weight": "model-00003-of-00101.safetensors", + "model.layers.4.mlp.experts.42.gate_proj.weight": "model-00003-of-00101.safetensors", + "model.layers.4.mlp.experts.42.up_proj.weight": "model-00003-of-00101.safetensors", + "model.layers.4.mlp.experts.42.down_proj.weight": "model-00003-of-00101.safetensors", + "model.layers.4.mlp.experts.43.gate_proj.weight": "model-00003-of-00101.safetensors", + "model.layers.4.mlp.experts.43.up_proj.weight": "model-00003-of-00101.safetensors", + "model.layers.4.mlp.experts.43.down_proj.weight": "model-00003-of-00101.safetensors", + "model.layers.4.mlp.experts.44.gate_proj.weight": "model-00003-of-00101.safetensors", + "model.layers.4.mlp.experts.44.up_proj.weight": "model-00003-of-00101.safetensors", + "model.layers.4.mlp.experts.44.down_proj.weight": "model-00003-of-00101.safetensors", + "model.layers.4.mlp.experts.45.gate_proj.weight": "model-00003-of-00101.safetensors", + "model.layers.4.mlp.experts.45.up_proj.weight": "model-00003-of-00101.safetensors", + "model.layers.4.mlp.experts.45.down_proj.weight": "model-00003-of-00101.safetensors", + "model.layers.4.mlp.experts.46.gate_proj.weight": "model-00003-of-00101.safetensors", + "model.layers.4.mlp.experts.46.up_proj.weight": "model-00003-of-00101.safetensors", + "model.layers.4.mlp.experts.46.down_proj.weight": "model-00003-of-00101.safetensors", + "model.layers.4.mlp.experts.47.gate_proj.weight": "model-00003-of-00101.safetensors", + "model.layers.4.mlp.experts.47.up_proj.weight": "model-00003-of-00101.safetensors", + "model.layers.4.mlp.experts.47.down_proj.weight": "model-00003-of-00101.safetensors", + "model.layers.4.mlp.experts.48.gate_proj.weight": "model-00003-of-00101.safetensors", + "model.layers.4.mlp.experts.48.up_proj.weight": "model-00003-of-00101.safetensors", + "model.layers.4.mlp.experts.48.down_proj.weight": "model-00003-of-00101.safetensors", + "model.layers.4.mlp.experts.49.gate_proj.weight": "model-00003-of-00101.safetensors", + "model.layers.4.mlp.experts.49.up_proj.weight": "model-00003-of-00101.safetensors", + "model.layers.4.mlp.experts.49.down_proj.weight": "model-00003-of-00101.safetensors", + "model.layers.4.mlp.experts.50.gate_proj.weight": "model-00003-of-00101.safetensors", + "model.layers.4.mlp.experts.50.up_proj.weight": "model-00003-of-00101.safetensors", + "model.layers.4.mlp.experts.50.down_proj.weight": "model-00003-of-00101.safetensors", + "model.layers.4.mlp.experts.51.gate_proj.weight": "model-00003-of-00101.safetensors", + "model.layers.4.mlp.experts.51.up_proj.weight": "model-00003-of-00101.safetensors", + "model.layers.4.mlp.experts.51.down_proj.weight": "model-00003-of-00101.safetensors", + "model.layers.4.mlp.experts.52.gate_proj.weight": "model-00003-of-00101.safetensors", + "model.layers.4.mlp.experts.52.up_proj.weight": "model-00003-of-00101.safetensors", + "model.layers.4.mlp.experts.52.down_proj.weight": "model-00003-of-00101.safetensors", + "model.layers.4.mlp.experts.53.gate_proj.weight": "model-00003-of-00101.safetensors", + "model.layers.4.mlp.experts.53.up_proj.weight": "model-00003-of-00101.safetensors", + "model.layers.4.mlp.experts.53.down_proj.weight": "model-00003-of-00101.safetensors", + "model.layers.4.mlp.experts.54.gate_proj.weight": "model-00003-of-00101.safetensors", + "model.layers.4.mlp.experts.54.up_proj.weight": "model-00003-of-00101.safetensors", + "model.layers.4.mlp.experts.54.down_proj.weight": "model-00003-of-00101.safetensors", + "model.layers.4.mlp.experts.55.gate_proj.weight": "model-00003-of-00101.safetensors", + "model.layers.4.mlp.experts.55.up_proj.weight": "model-00003-of-00101.safetensors", + "model.layers.4.mlp.experts.55.down_proj.weight": "model-00003-of-00101.safetensors", + "model.layers.4.mlp.experts.56.gate_proj.weight": "model-00003-of-00101.safetensors", + "model.layers.4.mlp.experts.56.up_proj.weight": "model-00003-of-00101.safetensors", + "model.layers.4.mlp.experts.56.down_proj.weight": "model-00003-of-00101.safetensors", + "model.layers.4.mlp.experts.57.gate_proj.weight": "model-00003-of-00101.safetensors", + "model.layers.4.mlp.experts.57.up_proj.weight": "model-00003-of-00101.safetensors", + "model.layers.4.mlp.experts.57.down_proj.weight": "model-00003-of-00101.safetensors", + "model.layers.4.mlp.experts.58.gate_proj.weight": "model-00003-of-00101.safetensors", + "model.layers.4.mlp.experts.58.up_proj.weight": "model-00003-of-00101.safetensors", + "model.layers.4.mlp.experts.58.down_proj.weight": "model-00003-of-00101.safetensors", + "model.layers.4.mlp.experts.59.gate_proj.weight": "model-00003-of-00101.safetensors", + "model.layers.4.mlp.experts.59.up_proj.weight": "model-00003-of-00101.safetensors", + "model.layers.4.mlp.experts.59.down_proj.weight": "model-00003-of-00101.safetensors", + "model.layers.4.mlp.experts.60.gate_proj.weight": "model-00003-of-00101.safetensors", + "model.layers.4.mlp.experts.60.up_proj.weight": "model-00003-of-00101.safetensors", + "model.layers.4.mlp.experts.60.down_proj.weight": "model-00003-of-00101.safetensors", + "model.layers.4.mlp.experts.61.gate_proj.weight": "model-00003-of-00101.safetensors", + "model.layers.4.mlp.experts.61.up_proj.weight": "model-00003-of-00101.safetensors", + "model.layers.4.mlp.experts.61.down_proj.weight": "model-00003-of-00101.safetensors", + "model.layers.4.mlp.experts.62.gate_proj.weight": "model-00003-of-00101.safetensors", + "model.layers.4.mlp.experts.62.up_proj.weight": "model-00003-of-00101.safetensors", + "model.layers.4.mlp.experts.62.down_proj.weight": "model-00003-of-00101.safetensors", + "model.layers.4.mlp.experts.63.gate_proj.weight": "model-00003-of-00101.safetensors", + "model.layers.4.mlp.experts.63.up_proj.weight": "model-00003-of-00101.safetensors", + "model.layers.4.mlp.experts.63.down_proj.weight": "model-00003-of-00101.safetensors", + "model.layers.4.mlp.experts.64.gate_proj.weight": "model-00003-of-00101.safetensors", + "model.layers.4.mlp.experts.64.up_proj.weight": "model-00003-of-00101.safetensors", + "model.layers.4.mlp.experts.64.down_proj.weight": "model-00003-of-00101.safetensors", + "model.layers.4.mlp.experts.65.gate_proj.weight": "model-00003-of-00101.safetensors", + "model.layers.4.mlp.experts.65.up_proj.weight": "model-00003-of-00101.safetensors", + "model.layers.4.mlp.experts.65.down_proj.weight": "model-00003-of-00101.safetensors", + "model.layers.4.mlp.experts.66.gate_proj.weight": "model-00003-of-00101.safetensors", + "model.layers.4.mlp.experts.66.up_proj.weight": "model-00003-of-00101.safetensors", + "model.layers.4.mlp.experts.66.down_proj.weight": "model-00003-of-00101.safetensors", + "model.layers.4.mlp.experts.67.gate_proj.weight": "model-00003-of-00101.safetensors", + "model.layers.4.mlp.experts.67.up_proj.weight": "model-00003-of-00101.safetensors", + "model.layers.4.mlp.experts.67.down_proj.weight": "model-00003-of-00101.safetensors", + "model.layers.4.mlp.experts.68.gate_proj.weight": "model-00003-of-00101.safetensors", + "model.layers.4.mlp.experts.68.up_proj.weight": "model-00003-of-00101.safetensors", + "model.layers.4.mlp.experts.68.down_proj.weight": "model-00003-of-00101.safetensors", + "model.layers.4.mlp.experts.69.gate_proj.weight": "model-00003-of-00101.safetensors", + "model.layers.4.mlp.experts.69.up_proj.weight": "model-00003-of-00101.safetensors", + "model.layers.4.mlp.experts.69.down_proj.weight": "model-00003-of-00101.safetensors", + "model.layers.4.mlp.experts.70.gate_proj.weight": "model-00003-of-00101.safetensors", + "model.layers.4.mlp.experts.70.up_proj.weight": "model-00003-of-00101.safetensors", + "model.layers.4.mlp.experts.70.down_proj.weight": "model-00003-of-00101.safetensors", + "model.layers.4.mlp.experts.71.gate_proj.weight": "model-00003-of-00101.safetensors", + "model.layers.4.mlp.experts.71.up_proj.weight": "model-00003-of-00101.safetensors", + "model.layers.4.mlp.experts.71.down_proj.weight": "model-00003-of-00101.safetensors", + "model.layers.4.mlp.experts.72.gate_proj.weight": "model-00003-of-00101.safetensors", + "model.layers.4.mlp.experts.72.up_proj.weight": "model-00003-of-00101.safetensors", + "model.layers.4.mlp.experts.72.down_proj.weight": "model-00003-of-00101.safetensors", + "model.layers.4.mlp.experts.73.gate_proj.weight": "model-00003-of-00101.safetensors", + "model.layers.4.mlp.experts.73.up_proj.weight": "model-00003-of-00101.safetensors", + "model.layers.4.mlp.experts.73.down_proj.weight": "model-00003-of-00101.safetensors", + "model.layers.4.mlp.experts.74.gate_proj.weight": "model-00003-of-00101.safetensors", + "model.layers.4.mlp.experts.74.up_proj.weight": "model-00003-of-00101.safetensors", + "model.layers.4.mlp.experts.74.down_proj.weight": "model-00003-of-00101.safetensors", + "model.layers.4.mlp.experts.75.gate_proj.weight": "model-00003-of-00101.safetensors", + "model.layers.4.mlp.experts.75.up_proj.weight": "model-00003-of-00101.safetensors", + "model.layers.4.mlp.experts.75.down_proj.weight": "model-00003-of-00101.safetensors", + "model.layers.4.mlp.experts.76.gate_proj.weight": "model-00003-of-00101.safetensors", + "model.layers.4.mlp.experts.76.up_proj.weight": "model-00003-of-00101.safetensors", + "model.layers.4.mlp.experts.76.down_proj.weight": "model-00003-of-00101.safetensors", + "model.layers.4.mlp.experts.77.gate_proj.weight": "model-00003-of-00101.safetensors", + "model.layers.4.mlp.experts.77.up_proj.weight": "model-00003-of-00101.safetensors", + "model.layers.4.mlp.experts.77.down_proj.weight": "model-00003-of-00101.safetensors", + "model.layers.4.mlp.experts.78.gate_proj.weight": "model-00003-of-00101.safetensors", + "model.layers.4.mlp.experts.78.up_proj.weight": "model-00003-of-00101.safetensors", + "model.layers.4.mlp.experts.78.down_proj.weight": "model-00003-of-00101.safetensors", + "model.layers.4.mlp.experts.79.gate_proj.weight": "model-00003-of-00101.safetensors", + "model.layers.4.mlp.experts.79.up_proj.weight": "model-00003-of-00101.safetensors", + "model.layers.4.mlp.experts.79.down_proj.weight": "model-00003-of-00101.safetensors", + "model.layers.4.mlp.experts.80.gate_proj.weight": "model-00003-of-00101.safetensors", + "model.layers.4.mlp.experts.80.up_proj.weight": "model-00003-of-00101.safetensors", + "model.layers.4.mlp.experts.80.down_proj.weight": "model-00003-of-00101.safetensors", + "model.layers.4.mlp.experts.81.gate_proj.weight": "model-00003-of-00101.safetensors", + "model.layers.4.mlp.experts.81.up_proj.weight": "model-00003-of-00101.safetensors", + "model.layers.4.mlp.experts.81.down_proj.weight": "model-00003-of-00101.safetensors", + "model.layers.4.mlp.experts.82.gate_proj.weight": "model-00003-of-00101.safetensors", + "model.layers.4.mlp.experts.82.up_proj.weight": "model-00003-of-00101.safetensors", + "model.layers.4.mlp.experts.82.down_proj.weight": "model-00003-of-00101.safetensors", + "model.layers.4.mlp.experts.83.gate_proj.weight": "model-00003-of-00101.safetensors", + "model.layers.4.mlp.experts.83.up_proj.weight": "model-00003-of-00101.safetensors", + "model.layers.4.mlp.experts.83.down_proj.weight": "model-00003-of-00101.safetensors", + "model.layers.4.mlp.experts.84.gate_proj.weight": "model-00003-of-00101.safetensors", + "model.layers.4.mlp.experts.84.up_proj.weight": "model-00003-of-00101.safetensors", + "model.layers.4.mlp.experts.84.down_proj.weight": "model-00003-of-00101.safetensors", + "model.layers.4.mlp.experts.85.gate_proj.weight": "model-00003-of-00101.safetensors", + "model.layers.4.mlp.experts.85.up_proj.weight": "model-00003-of-00101.safetensors", + "model.layers.4.mlp.experts.85.down_proj.weight": "model-00003-of-00101.safetensors", + "model.layers.4.mlp.experts.86.gate_proj.weight": "model-00003-of-00101.safetensors", + "model.layers.4.mlp.experts.86.up_proj.weight": "model-00003-of-00101.safetensors", + "model.layers.4.mlp.experts.86.down_proj.weight": "model-00003-of-00101.safetensors", + "model.layers.4.mlp.experts.87.gate_proj.weight": "model-00003-of-00101.safetensors", + "model.layers.4.mlp.experts.87.up_proj.weight": "model-00003-of-00101.safetensors", + "model.layers.4.mlp.experts.87.down_proj.weight": "model-00003-of-00101.safetensors", + "model.layers.4.mlp.experts.88.gate_proj.weight": "model-00003-of-00101.safetensors", + "model.layers.4.mlp.experts.88.up_proj.weight": "model-00003-of-00101.safetensors", + "model.layers.4.mlp.experts.88.down_proj.weight": "model-00003-of-00101.safetensors", + "model.layers.4.mlp.experts.89.gate_proj.weight": "model-00003-of-00101.safetensors", + "model.layers.4.mlp.experts.89.up_proj.weight": "model-00003-of-00101.safetensors", + "model.layers.4.mlp.experts.89.down_proj.weight": "model-00003-of-00101.safetensors", + "model.layers.4.mlp.experts.90.gate_proj.weight": "model-00003-of-00101.safetensors", + "model.layers.4.mlp.experts.90.up_proj.weight": "model-00003-of-00101.safetensors", + "model.layers.4.mlp.experts.90.down_proj.weight": "model-00003-of-00101.safetensors", + "model.layers.4.mlp.experts.91.gate_proj.weight": "model-00003-of-00101.safetensors", + "model.layers.4.mlp.experts.91.up_proj.weight": "model-00003-of-00101.safetensors", + "model.layers.4.mlp.experts.91.down_proj.weight": "model-00003-of-00101.safetensors", + "model.layers.4.mlp.experts.92.gate_proj.weight": "model-00003-of-00101.safetensors", + "model.layers.4.mlp.experts.92.up_proj.weight": "model-00003-of-00101.safetensors", + "model.layers.4.mlp.experts.92.down_proj.weight": "model-00003-of-00101.safetensors", + "model.layers.4.mlp.experts.93.gate_proj.weight": "model-00003-of-00101.safetensors", + "model.layers.4.mlp.experts.93.up_proj.weight": "model-00003-of-00101.safetensors", + "model.layers.4.mlp.experts.93.down_proj.weight": "model-00003-of-00101.safetensors", + "model.layers.4.mlp.experts.94.gate_proj.weight": "model-00003-of-00101.safetensors", + "model.layers.4.mlp.experts.94.up_proj.weight": "model-00003-of-00101.safetensors", + "model.layers.4.mlp.experts.94.down_proj.weight": "model-00003-of-00101.safetensors", + "model.layers.4.mlp.experts.95.gate_proj.weight": "model-00003-of-00101.safetensors", + "model.layers.4.mlp.experts.95.up_proj.weight": "model-00003-of-00101.safetensors", + "model.layers.4.mlp.experts.95.down_proj.weight": "model-00003-of-00101.safetensors", + "model.layers.4.mlp.experts.96.gate_proj.weight": "model-00003-of-00101.safetensors", + "model.layers.4.mlp.experts.96.up_proj.weight": "model-00003-of-00101.safetensors", + "model.layers.4.mlp.experts.96.down_proj.weight": "model-00003-of-00101.safetensors", + "model.layers.4.mlp.experts.97.gate_proj.weight": "model-00003-of-00101.safetensors", + "model.layers.4.mlp.experts.97.up_proj.weight": "model-00003-of-00101.safetensors", + "model.layers.4.mlp.experts.97.down_proj.weight": "model-00003-of-00101.safetensors", + "model.layers.4.mlp.experts.98.gate_proj.weight": "model-00003-of-00101.safetensors", + "model.layers.4.mlp.experts.98.up_proj.weight": "model-00003-of-00101.safetensors", + "model.layers.4.mlp.experts.98.down_proj.weight": "model-00003-of-00101.safetensors", + "model.layers.4.mlp.experts.99.gate_proj.weight": "model-00003-of-00101.safetensors", + "model.layers.4.mlp.experts.99.up_proj.weight": "model-00003-of-00101.safetensors", + "model.layers.4.mlp.experts.99.down_proj.weight": "model-00003-of-00101.safetensors", + "model.layers.4.mlp.experts.100.gate_proj.weight": "model-00003-of-00101.safetensors", + "model.layers.4.mlp.experts.100.up_proj.weight": "model-00003-of-00101.safetensors", + "model.layers.4.mlp.experts.100.down_proj.weight": "model-00003-of-00101.safetensors", + "model.layers.4.mlp.experts.101.gate_proj.weight": "model-00003-of-00101.safetensors", + "model.layers.4.mlp.experts.101.up_proj.weight": "model-00003-of-00101.safetensors", + "model.layers.4.mlp.experts.101.down_proj.weight": "model-00003-of-00101.safetensors", + "model.layers.4.mlp.experts.102.gate_proj.weight": "model-00003-of-00101.safetensors", + "model.layers.4.mlp.experts.102.up_proj.weight": "model-00003-of-00101.safetensors", + "model.layers.4.mlp.experts.102.down_proj.weight": "model-00003-of-00101.safetensors", + "model.layers.4.mlp.experts.103.gate_proj.weight": "model-00003-of-00101.safetensors", + "model.layers.4.mlp.experts.103.up_proj.weight": "model-00003-of-00101.safetensors", + "model.layers.4.mlp.experts.103.down_proj.weight": "model-00003-of-00101.safetensors", + "model.layers.4.mlp.experts.104.gate_proj.weight": "model-00003-of-00101.safetensors", + "model.layers.4.mlp.experts.104.up_proj.weight": "model-00003-of-00101.safetensors", + "model.layers.4.mlp.experts.104.down_proj.weight": "model-00003-of-00101.safetensors", + "model.layers.4.mlp.experts.105.gate_proj.weight": "model-00003-of-00101.safetensors", + "model.layers.4.mlp.experts.105.up_proj.weight": "model-00003-of-00101.safetensors", + "model.layers.4.mlp.experts.105.down_proj.weight": "model-00003-of-00101.safetensors", + "model.layers.4.mlp.experts.106.gate_proj.weight": "model-00003-of-00101.safetensors", + "model.layers.4.mlp.experts.106.up_proj.weight": "model-00003-of-00101.safetensors", + "model.layers.4.mlp.experts.106.down_proj.weight": "model-00003-of-00101.safetensors", + "model.layers.4.mlp.experts.107.gate_proj.weight": "model-00003-of-00101.safetensors", + "model.layers.4.mlp.experts.107.up_proj.weight": "model-00003-of-00101.safetensors", + "model.layers.4.mlp.experts.107.down_proj.weight": "model-00003-of-00101.safetensors", + "model.layers.4.mlp.experts.108.gate_proj.weight": "model-00003-of-00101.safetensors", + "model.layers.4.mlp.experts.108.up_proj.weight": "model-00003-of-00101.safetensors", + "model.layers.4.mlp.experts.108.down_proj.weight": "model-00003-of-00101.safetensors", + "model.layers.4.mlp.experts.109.gate_proj.weight": "model-00003-of-00101.safetensors", + "model.layers.4.mlp.experts.109.up_proj.weight": "model-00003-of-00101.safetensors", + "model.layers.4.mlp.experts.109.down_proj.weight": "model-00003-of-00101.safetensors", + "model.layers.4.mlp.experts.110.gate_proj.weight": "model-00003-of-00101.safetensors", + "model.layers.4.mlp.experts.110.up_proj.weight": "model-00003-of-00101.safetensors", + "model.layers.4.mlp.experts.110.down_proj.weight": "model-00003-of-00101.safetensors", + "model.layers.4.mlp.experts.111.gate_proj.weight": "model-00003-of-00101.safetensors", + "model.layers.4.mlp.experts.111.up_proj.weight": "model-00003-of-00101.safetensors", + "model.layers.4.mlp.experts.111.down_proj.weight": "model-00003-of-00101.safetensors", + "model.layers.4.mlp.experts.112.gate_proj.weight": "model-00003-of-00101.safetensors", + "model.layers.4.mlp.experts.112.up_proj.weight": "model-00003-of-00101.safetensors", + "model.layers.4.mlp.experts.112.down_proj.weight": "model-00003-of-00101.safetensors", + "model.layers.4.mlp.experts.113.gate_proj.weight": "model-00003-of-00101.safetensors", + "model.layers.4.mlp.experts.113.up_proj.weight": "model-00003-of-00101.safetensors", + "model.layers.4.mlp.experts.113.down_proj.weight": "model-00003-of-00101.safetensors", + "model.layers.4.mlp.experts.114.gate_proj.weight": "model-00003-of-00101.safetensors", + "model.layers.4.mlp.experts.114.up_proj.weight": "model-00003-of-00101.safetensors", + "model.layers.4.mlp.experts.114.down_proj.weight": "model-00003-of-00101.safetensors", + "model.layers.4.mlp.experts.115.gate_proj.weight": "model-00003-of-00101.safetensors", + "model.layers.4.mlp.experts.115.up_proj.weight": "model-00003-of-00101.safetensors", + "model.layers.4.mlp.experts.115.down_proj.weight": "model-00003-of-00101.safetensors", + "model.layers.4.mlp.experts.116.gate_proj.weight": "model-00003-of-00101.safetensors", + "model.layers.4.mlp.experts.116.up_proj.weight": "model-00003-of-00101.safetensors", + "model.layers.4.mlp.experts.116.down_proj.weight": "model-00003-of-00101.safetensors", + "model.layers.4.mlp.experts.117.gate_proj.weight": "model-00003-of-00101.safetensors", + "model.layers.4.mlp.experts.117.up_proj.weight": "model-00003-of-00101.safetensors", + "model.layers.4.mlp.experts.117.down_proj.weight": "model-00003-of-00101.safetensors", + "model.layers.4.mlp.experts.118.gate_proj.weight": "model-00003-of-00101.safetensors", + "model.layers.4.mlp.experts.118.up_proj.weight": "model-00003-of-00101.safetensors", + "model.layers.4.mlp.experts.118.down_proj.weight": "model-00003-of-00101.safetensors", + "model.layers.4.mlp.experts.119.gate_proj.weight": "model-00003-of-00101.safetensors", + "model.layers.4.mlp.experts.119.up_proj.weight": "model-00003-of-00101.safetensors", + "model.layers.4.mlp.experts.119.down_proj.weight": "model-00003-of-00101.safetensors", + "model.layers.4.mlp.gate.weight": "model-00003-of-00101.safetensors", + "model.layers.4.mlp.gate.e_score_correction_bias": "model-00003-of-00101.safetensors", + "model.layers.4.mlp.shared_experts.gate_proj.weight": "model-00003-of-00101.safetensors", + "model.layers.4.mlp.shared_experts.up_proj.weight": "model-00003-of-00101.safetensors", + "model.layers.4.mlp.shared_experts.down_proj.weight": "model-00003-of-00101.safetensors", + "model.layers.4.input_layernorm.weight": "model-00003-of-00101.safetensors", + "model.layers.4.post_attention_layernorm.weight": "model-00003-of-00101.safetensors", + "model.layers.5.self_attn.q_proj.weight": "model-00003-of-00101.safetensors", + "model.layers.5.self_attn.q_proj.bias": "model-00003-of-00101.safetensors", + "model.layers.5.self_attn.k_proj.weight": "model-00003-of-00101.safetensors", + "model.layers.5.self_attn.k_proj.bias": "model-00003-of-00101.safetensors", + "model.layers.5.self_attn.v_proj.weight": "model-00003-of-00101.safetensors", + "model.layers.5.self_attn.v_proj.bias": "model-00003-of-00101.safetensors", + "model.layers.5.self_attn.o_proj.weight": "model-00003-of-00101.safetensors", + "model.layers.5.self_attn.q_norm.weight": "model-00003-of-00101.safetensors", + "model.layers.5.self_attn.k_norm.weight": "model-00003-of-00101.safetensors", + "model.layers.5.mlp.experts.0.gate_proj.weight": "model-00003-of-00101.safetensors", + "model.layers.5.mlp.experts.0.up_proj.weight": "model-00003-of-00101.safetensors", + "model.layers.5.mlp.experts.0.down_proj.weight": "model-00003-of-00101.safetensors", + "model.layers.5.mlp.experts.1.gate_proj.weight": "model-00003-of-00101.safetensors", + "model.layers.5.mlp.experts.1.up_proj.weight": "model-00003-of-00101.safetensors", + "model.layers.5.mlp.experts.1.down_proj.weight": "model-00003-of-00101.safetensors", + "model.layers.5.mlp.experts.2.gate_proj.weight": "model-00003-of-00101.safetensors", + "model.layers.5.mlp.experts.2.up_proj.weight": "model-00003-of-00101.safetensors", + "model.layers.5.mlp.experts.2.down_proj.weight": "model-00003-of-00101.safetensors", + "model.layers.5.mlp.experts.3.gate_proj.weight": "model-00003-of-00101.safetensors", + "model.layers.5.mlp.experts.3.up_proj.weight": "model-00003-of-00101.safetensors", + "model.layers.5.mlp.experts.3.down_proj.weight": "model-00003-of-00101.safetensors", + "model.layers.5.mlp.experts.4.gate_proj.weight": "model-00003-of-00101.safetensors", + "model.layers.5.mlp.experts.4.up_proj.weight": "model-00003-of-00101.safetensors", + "model.layers.5.mlp.experts.4.down_proj.weight": "model-00003-of-00101.safetensors", + "model.layers.5.mlp.experts.5.gate_proj.weight": "model-00003-of-00101.safetensors", + "model.layers.5.mlp.experts.5.up_proj.weight": "model-00003-of-00101.safetensors", + "model.layers.5.mlp.experts.5.down_proj.weight": "model-00003-of-00101.safetensors", + "model.layers.5.mlp.experts.6.gate_proj.weight": "model-00003-of-00101.safetensors", + "model.layers.5.mlp.experts.6.up_proj.weight": "model-00003-of-00101.safetensors", + "model.layers.5.mlp.experts.6.down_proj.weight": "model-00003-of-00101.safetensors", + "model.layers.5.mlp.experts.7.gate_proj.weight": "model-00004-of-00101.safetensors", + "model.layers.5.mlp.experts.7.up_proj.weight": "model-00004-of-00101.safetensors", + "model.layers.5.mlp.experts.7.down_proj.weight": "model-00004-of-00101.safetensors", + "model.layers.5.mlp.experts.8.gate_proj.weight": "model-00004-of-00101.safetensors", + "model.layers.5.mlp.experts.8.up_proj.weight": "model-00004-of-00101.safetensors", + "model.layers.5.mlp.experts.8.down_proj.weight": "model-00004-of-00101.safetensors", + "model.layers.5.mlp.experts.9.gate_proj.weight": "model-00004-of-00101.safetensors", + "model.layers.5.mlp.experts.9.up_proj.weight": "model-00004-of-00101.safetensors", + "model.layers.5.mlp.experts.9.down_proj.weight": "model-00004-of-00101.safetensors", + "model.layers.5.mlp.experts.10.gate_proj.weight": "model-00004-of-00101.safetensors", + "model.layers.5.mlp.experts.10.up_proj.weight": "model-00004-of-00101.safetensors", + "model.layers.5.mlp.experts.10.down_proj.weight": "model-00004-of-00101.safetensors", + "model.layers.5.mlp.experts.11.gate_proj.weight": "model-00004-of-00101.safetensors", + "model.layers.5.mlp.experts.11.up_proj.weight": "model-00004-of-00101.safetensors", + "model.layers.5.mlp.experts.11.down_proj.weight": "model-00004-of-00101.safetensors", + "model.layers.5.mlp.experts.12.gate_proj.weight": "model-00004-of-00101.safetensors", + "model.layers.5.mlp.experts.12.up_proj.weight": "model-00004-of-00101.safetensors", + "model.layers.5.mlp.experts.12.down_proj.weight": "model-00004-of-00101.safetensors", + "model.layers.5.mlp.experts.13.gate_proj.weight": "model-00004-of-00101.safetensors", + "model.layers.5.mlp.experts.13.up_proj.weight": "model-00004-of-00101.safetensors", + "model.layers.5.mlp.experts.13.down_proj.weight": "model-00004-of-00101.safetensors", + "model.layers.5.mlp.experts.14.gate_proj.weight": "model-00004-of-00101.safetensors", + "model.layers.5.mlp.experts.14.up_proj.weight": "model-00004-of-00101.safetensors", + "model.layers.5.mlp.experts.14.down_proj.weight": "model-00004-of-00101.safetensors", + "model.layers.5.mlp.experts.15.gate_proj.weight": "model-00004-of-00101.safetensors", + "model.layers.5.mlp.experts.15.up_proj.weight": "model-00004-of-00101.safetensors", + "model.layers.5.mlp.experts.15.down_proj.weight": "model-00004-of-00101.safetensors", + "model.layers.5.mlp.experts.16.gate_proj.weight": "model-00004-of-00101.safetensors", + "model.layers.5.mlp.experts.16.up_proj.weight": "model-00004-of-00101.safetensors", + "model.layers.5.mlp.experts.16.down_proj.weight": "model-00004-of-00101.safetensors", + "model.layers.5.mlp.experts.17.gate_proj.weight": "model-00004-of-00101.safetensors", + "model.layers.5.mlp.experts.17.up_proj.weight": "model-00004-of-00101.safetensors", + "model.layers.5.mlp.experts.17.down_proj.weight": "model-00004-of-00101.safetensors", + "model.layers.5.mlp.experts.18.gate_proj.weight": "model-00004-of-00101.safetensors", + "model.layers.5.mlp.experts.18.up_proj.weight": "model-00004-of-00101.safetensors", + "model.layers.5.mlp.experts.18.down_proj.weight": "model-00004-of-00101.safetensors", + "model.layers.5.mlp.experts.19.gate_proj.weight": "model-00004-of-00101.safetensors", + "model.layers.5.mlp.experts.19.up_proj.weight": "model-00004-of-00101.safetensors", + "model.layers.5.mlp.experts.19.down_proj.weight": "model-00004-of-00101.safetensors", + "model.layers.5.mlp.experts.20.gate_proj.weight": "model-00004-of-00101.safetensors", + "model.layers.5.mlp.experts.20.up_proj.weight": "model-00004-of-00101.safetensors", + "model.layers.5.mlp.experts.20.down_proj.weight": "model-00004-of-00101.safetensors", + "model.layers.5.mlp.experts.21.gate_proj.weight": "model-00004-of-00101.safetensors", + "model.layers.5.mlp.experts.21.up_proj.weight": "model-00004-of-00101.safetensors", + "model.layers.5.mlp.experts.21.down_proj.weight": "model-00004-of-00101.safetensors", + "model.layers.5.mlp.experts.22.gate_proj.weight": "model-00004-of-00101.safetensors", + "model.layers.5.mlp.experts.22.up_proj.weight": "model-00004-of-00101.safetensors", + "model.layers.5.mlp.experts.22.down_proj.weight": "model-00004-of-00101.safetensors", + "model.layers.5.mlp.experts.23.gate_proj.weight": "model-00004-of-00101.safetensors", + "model.layers.5.mlp.experts.23.up_proj.weight": "model-00004-of-00101.safetensors", + "model.layers.5.mlp.experts.23.down_proj.weight": "model-00004-of-00101.safetensors", + "model.layers.5.mlp.experts.24.gate_proj.weight": "model-00004-of-00101.safetensors", + "model.layers.5.mlp.experts.24.up_proj.weight": "model-00004-of-00101.safetensors", + "model.layers.5.mlp.experts.24.down_proj.weight": "model-00004-of-00101.safetensors", + "model.layers.5.mlp.experts.25.gate_proj.weight": "model-00004-of-00101.safetensors", + "model.layers.5.mlp.experts.25.up_proj.weight": "model-00004-of-00101.safetensors", + "model.layers.5.mlp.experts.25.down_proj.weight": "model-00004-of-00101.safetensors", + "model.layers.5.mlp.experts.26.gate_proj.weight": "model-00004-of-00101.safetensors", + "model.layers.5.mlp.experts.26.up_proj.weight": "model-00004-of-00101.safetensors", + "model.layers.5.mlp.experts.26.down_proj.weight": "model-00004-of-00101.safetensors", + "model.layers.5.mlp.experts.27.gate_proj.weight": "model-00004-of-00101.safetensors", + "model.layers.5.mlp.experts.27.up_proj.weight": "model-00004-of-00101.safetensors", + "model.layers.5.mlp.experts.27.down_proj.weight": "model-00004-of-00101.safetensors", + "model.layers.5.mlp.experts.28.gate_proj.weight": "model-00004-of-00101.safetensors", + "model.layers.5.mlp.experts.28.up_proj.weight": "model-00004-of-00101.safetensors", + "model.layers.5.mlp.experts.28.down_proj.weight": "model-00004-of-00101.safetensors", + "model.layers.5.mlp.experts.29.gate_proj.weight": "model-00004-of-00101.safetensors", + "model.layers.5.mlp.experts.29.up_proj.weight": "model-00004-of-00101.safetensors", + "model.layers.5.mlp.experts.29.down_proj.weight": "model-00004-of-00101.safetensors", + "model.layers.5.mlp.experts.30.gate_proj.weight": "model-00004-of-00101.safetensors", + "model.layers.5.mlp.experts.30.up_proj.weight": "model-00004-of-00101.safetensors", + "model.layers.5.mlp.experts.30.down_proj.weight": "model-00004-of-00101.safetensors", + "model.layers.5.mlp.experts.31.gate_proj.weight": "model-00004-of-00101.safetensors", + "model.layers.5.mlp.experts.31.up_proj.weight": "model-00004-of-00101.safetensors", + "model.layers.5.mlp.experts.31.down_proj.weight": "model-00004-of-00101.safetensors", + "model.layers.5.mlp.experts.32.gate_proj.weight": "model-00004-of-00101.safetensors", + "model.layers.5.mlp.experts.32.up_proj.weight": "model-00004-of-00101.safetensors", + "model.layers.5.mlp.experts.32.down_proj.weight": "model-00004-of-00101.safetensors", + "model.layers.5.mlp.experts.33.gate_proj.weight": "model-00004-of-00101.safetensors", + "model.layers.5.mlp.experts.33.up_proj.weight": "model-00004-of-00101.safetensors", + "model.layers.5.mlp.experts.33.down_proj.weight": "model-00004-of-00101.safetensors", + "model.layers.5.mlp.experts.34.gate_proj.weight": "model-00004-of-00101.safetensors", + "model.layers.5.mlp.experts.34.up_proj.weight": "model-00004-of-00101.safetensors", + "model.layers.5.mlp.experts.34.down_proj.weight": "model-00004-of-00101.safetensors", + "model.layers.5.mlp.experts.35.gate_proj.weight": "model-00004-of-00101.safetensors", + "model.layers.5.mlp.experts.35.up_proj.weight": "model-00004-of-00101.safetensors", + "model.layers.5.mlp.experts.35.down_proj.weight": "model-00004-of-00101.safetensors", + "model.layers.5.mlp.experts.36.gate_proj.weight": "model-00004-of-00101.safetensors", + "model.layers.5.mlp.experts.36.up_proj.weight": "model-00004-of-00101.safetensors", + "model.layers.5.mlp.experts.36.down_proj.weight": "model-00004-of-00101.safetensors", + "model.layers.5.mlp.experts.37.gate_proj.weight": "model-00004-of-00101.safetensors", + "model.layers.5.mlp.experts.37.up_proj.weight": "model-00004-of-00101.safetensors", + "model.layers.5.mlp.experts.37.down_proj.weight": "model-00004-of-00101.safetensors", + "model.layers.5.mlp.experts.38.gate_proj.weight": "model-00004-of-00101.safetensors", + "model.layers.5.mlp.experts.38.up_proj.weight": "model-00004-of-00101.safetensors", + "model.layers.5.mlp.experts.38.down_proj.weight": "model-00004-of-00101.safetensors", + "model.layers.5.mlp.experts.39.gate_proj.weight": "model-00004-of-00101.safetensors", + "model.layers.5.mlp.experts.39.up_proj.weight": "model-00004-of-00101.safetensors", + "model.layers.5.mlp.experts.39.down_proj.weight": "model-00004-of-00101.safetensors", + "model.layers.5.mlp.experts.40.gate_proj.weight": "model-00004-of-00101.safetensors", + "model.layers.5.mlp.experts.40.up_proj.weight": "model-00004-of-00101.safetensors", + "model.layers.5.mlp.experts.40.down_proj.weight": "model-00004-of-00101.safetensors", + "model.layers.5.mlp.experts.41.gate_proj.weight": "model-00004-of-00101.safetensors", + "model.layers.5.mlp.experts.41.up_proj.weight": "model-00004-of-00101.safetensors", + "model.layers.5.mlp.experts.41.down_proj.weight": "model-00004-of-00101.safetensors", + "model.layers.5.mlp.experts.42.gate_proj.weight": "model-00004-of-00101.safetensors", + "model.layers.5.mlp.experts.42.up_proj.weight": "model-00004-of-00101.safetensors", + "model.layers.5.mlp.experts.42.down_proj.weight": "model-00004-of-00101.safetensors", + "model.layers.5.mlp.experts.43.gate_proj.weight": "model-00004-of-00101.safetensors", + "model.layers.5.mlp.experts.43.up_proj.weight": "model-00004-of-00101.safetensors", + "model.layers.5.mlp.experts.43.down_proj.weight": "model-00004-of-00101.safetensors", + "model.layers.5.mlp.experts.44.gate_proj.weight": "model-00004-of-00101.safetensors", + "model.layers.5.mlp.experts.44.up_proj.weight": "model-00004-of-00101.safetensors", + "model.layers.5.mlp.experts.44.down_proj.weight": "model-00004-of-00101.safetensors", + "model.layers.5.mlp.experts.45.gate_proj.weight": "model-00004-of-00101.safetensors", + "model.layers.5.mlp.experts.45.up_proj.weight": "model-00004-of-00101.safetensors", + "model.layers.5.mlp.experts.45.down_proj.weight": "model-00004-of-00101.safetensors", + "model.layers.5.mlp.experts.46.gate_proj.weight": "model-00004-of-00101.safetensors", + "model.layers.5.mlp.experts.46.up_proj.weight": "model-00004-of-00101.safetensors", + "model.layers.5.mlp.experts.46.down_proj.weight": "model-00004-of-00101.safetensors", + "model.layers.5.mlp.experts.47.gate_proj.weight": "model-00004-of-00101.safetensors", + "model.layers.5.mlp.experts.47.up_proj.weight": "model-00004-of-00101.safetensors", + "model.layers.5.mlp.experts.47.down_proj.weight": "model-00004-of-00101.safetensors", + "model.layers.5.mlp.experts.48.gate_proj.weight": "model-00004-of-00101.safetensors", + "model.layers.5.mlp.experts.48.up_proj.weight": "model-00004-of-00101.safetensors", + "model.layers.5.mlp.experts.48.down_proj.weight": "model-00004-of-00101.safetensors", + "model.layers.5.mlp.experts.49.gate_proj.weight": "model-00004-of-00101.safetensors", + "model.layers.5.mlp.experts.49.up_proj.weight": "model-00004-of-00101.safetensors", + "model.layers.5.mlp.experts.49.down_proj.weight": "model-00004-of-00101.safetensors", + "model.layers.5.mlp.experts.50.gate_proj.weight": "model-00004-of-00101.safetensors", + "model.layers.5.mlp.experts.50.up_proj.weight": "model-00004-of-00101.safetensors", + "model.layers.5.mlp.experts.50.down_proj.weight": "model-00004-of-00101.safetensors", + "model.layers.5.mlp.experts.51.gate_proj.weight": "model-00004-of-00101.safetensors", + "model.layers.5.mlp.experts.51.up_proj.weight": "model-00004-of-00101.safetensors", + "model.layers.5.mlp.experts.51.down_proj.weight": "model-00004-of-00101.safetensors", + "model.layers.5.mlp.experts.52.gate_proj.weight": "model-00004-of-00101.safetensors", + "model.layers.5.mlp.experts.52.up_proj.weight": "model-00004-of-00101.safetensors", + "model.layers.5.mlp.experts.52.down_proj.weight": "model-00004-of-00101.safetensors", + "model.layers.5.mlp.experts.53.gate_proj.weight": "model-00004-of-00101.safetensors", + "model.layers.5.mlp.experts.53.up_proj.weight": "model-00004-of-00101.safetensors", + "model.layers.5.mlp.experts.53.down_proj.weight": "model-00004-of-00101.safetensors", + "model.layers.5.mlp.experts.54.gate_proj.weight": "model-00004-of-00101.safetensors", + "model.layers.5.mlp.experts.54.up_proj.weight": "model-00004-of-00101.safetensors", + "model.layers.5.mlp.experts.54.down_proj.weight": "model-00004-of-00101.safetensors", + "model.layers.5.mlp.experts.55.gate_proj.weight": "model-00004-of-00101.safetensors", + "model.layers.5.mlp.experts.55.up_proj.weight": "model-00004-of-00101.safetensors", + "model.layers.5.mlp.experts.55.down_proj.weight": "model-00004-of-00101.safetensors", + "model.layers.5.mlp.experts.56.gate_proj.weight": "model-00004-of-00101.safetensors", + "model.layers.5.mlp.experts.56.up_proj.weight": "model-00004-of-00101.safetensors", + "model.layers.5.mlp.experts.56.down_proj.weight": "model-00004-of-00101.safetensors", + "model.layers.5.mlp.experts.57.gate_proj.weight": "model-00004-of-00101.safetensors", + "model.layers.5.mlp.experts.57.up_proj.weight": "model-00004-of-00101.safetensors", + "model.layers.5.mlp.experts.57.down_proj.weight": "model-00004-of-00101.safetensors", + "model.layers.5.mlp.experts.58.gate_proj.weight": "model-00004-of-00101.safetensors", + "model.layers.5.mlp.experts.58.up_proj.weight": "model-00004-of-00101.safetensors", + "model.layers.5.mlp.experts.58.down_proj.weight": "model-00004-of-00101.safetensors", + "model.layers.5.mlp.experts.59.gate_proj.weight": "model-00004-of-00101.safetensors", + "model.layers.5.mlp.experts.59.up_proj.weight": "model-00004-of-00101.safetensors", + "model.layers.5.mlp.experts.59.down_proj.weight": "model-00004-of-00101.safetensors", + "model.layers.5.mlp.experts.60.gate_proj.weight": "model-00004-of-00101.safetensors", + "model.layers.5.mlp.experts.60.up_proj.weight": "model-00004-of-00101.safetensors", + "model.layers.5.mlp.experts.60.down_proj.weight": "model-00004-of-00101.safetensors", + "model.layers.5.mlp.experts.61.gate_proj.weight": "model-00004-of-00101.safetensors", + "model.layers.5.mlp.experts.61.up_proj.weight": "model-00004-of-00101.safetensors", + "model.layers.5.mlp.experts.61.down_proj.weight": "model-00004-of-00101.safetensors", + "model.layers.5.mlp.experts.62.gate_proj.weight": "model-00004-of-00101.safetensors", + "model.layers.5.mlp.experts.62.up_proj.weight": "model-00004-of-00101.safetensors", + "model.layers.5.mlp.experts.62.down_proj.weight": "model-00004-of-00101.safetensors", + "model.layers.5.mlp.experts.63.gate_proj.weight": "model-00004-of-00101.safetensors", + "model.layers.5.mlp.experts.63.up_proj.weight": "model-00004-of-00101.safetensors", + "model.layers.5.mlp.experts.63.down_proj.weight": "model-00004-of-00101.safetensors", + "model.layers.5.mlp.experts.64.gate_proj.weight": "model-00004-of-00101.safetensors", + "model.layers.5.mlp.experts.64.up_proj.weight": "model-00004-of-00101.safetensors", + "model.layers.5.mlp.experts.64.down_proj.weight": "model-00004-of-00101.safetensors", + "model.layers.5.mlp.experts.65.gate_proj.weight": "model-00004-of-00101.safetensors", + "model.layers.5.mlp.experts.65.up_proj.weight": "model-00004-of-00101.safetensors", + "model.layers.5.mlp.experts.65.down_proj.weight": "model-00004-of-00101.safetensors", + "model.layers.5.mlp.experts.66.gate_proj.weight": "model-00004-of-00101.safetensors", + "model.layers.5.mlp.experts.66.up_proj.weight": "model-00004-of-00101.safetensors", + "model.layers.5.mlp.experts.66.down_proj.weight": "model-00004-of-00101.safetensors", + "model.layers.5.mlp.experts.67.gate_proj.weight": "model-00004-of-00101.safetensors", + "model.layers.5.mlp.experts.67.up_proj.weight": "model-00004-of-00101.safetensors", + "model.layers.5.mlp.experts.67.down_proj.weight": "model-00004-of-00101.safetensors", + "model.layers.5.mlp.experts.68.gate_proj.weight": "model-00004-of-00101.safetensors", + "model.layers.5.mlp.experts.68.up_proj.weight": "model-00004-of-00101.safetensors", + "model.layers.5.mlp.experts.68.down_proj.weight": "model-00004-of-00101.safetensors", + "model.layers.5.mlp.experts.69.gate_proj.weight": "model-00004-of-00101.safetensors", + "model.layers.5.mlp.experts.69.up_proj.weight": "model-00004-of-00101.safetensors", + "model.layers.5.mlp.experts.69.down_proj.weight": "model-00004-of-00101.safetensors", + "model.layers.5.mlp.experts.70.gate_proj.weight": "model-00004-of-00101.safetensors", + "model.layers.5.mlp.experts.70.up_proj.weight": "model-00004-of-00101.safetensors", + "model.layers.5.mlp.experts.70.down_proj.weight": "model-00004-of-00101.safetensors", + "model.layers.5.mlp.experts.71.gate_proj.weight": "model-00004-of-00101.safetensors", + "model.layers.5.mlp.experts.71.up_proj.weight": "model-00004-of-00101.safetensors", + "model.layers.5.mlp.experts.71.down_proj.weight": "model-00004-of-00101.safetensors", + "model.layers.5.mlp.experts.72.gate_proj.weight": "model-00004-of-00101.safetensors", + "model.layers.5.mlp.experts.72.up_proj.weight": "model-00004-of-00101.safetensors", + "model.layers.5.mlp.experts.72.down_proj.weight": "model-00004-of-00101.safetensors", + "model.layers.5.mlp.experts.73.gate_proj.weight": "model-00004-of-00101.safetensors", + "model.layers.5.mlp.experts.73.up_proj.weight": "model-00004-of-00101.safetensors", + "model.layers.5.mlp.experts.73.down_proj.weight": "model-00004-of-00101.safetensors", + "model.layers.5.mlp.experts.74.gate_proj.weight": "model-00004-of-00101.safetensors", + "model.layers.5.mlp.experts.74.up_proj.weight": "model-00004-of-00101.safetensors", + "model.layers.5.mlp.experts.74.down_proj.weight": "model-00004-of-00101.safetensors", + "model.layers.5.mlp.experts.75.gate_proj.weight": "model-00004-of-00101.safetensors", + "model.layers.5.mlp.experts.75.up_proj.weight": "model-00004-of-00101.safetensors", + "model.layers.5.mlp.experts.75.down_proj.weight": "model-00004-of-00101.safetensors", + "model.layers.5.mlp.experts.76.gate_proj.weight": "model-00004-of-00101.safetensors", + "model.layers.5.mlp.experts.76.up_proj.weight": "model-00004-of-00101.safetensors", + "model.layers.5.mlp.experts.76.down_proj.weight": "model-00004-of-00101.safetensors", + "model.layers.5.mlp.experts.77.gate_proj.weight": "model-00004-of-00101.safetensors", + "model.layers.5.mlp.experts.77.up_proj.weight": "model-00004-of-00101.safetensors", + "model.layers.5.mlp.experts.77.down_proj.weight": "model-00004-of-00101.safetensors", + "model.layers.5.mlp.experts.78.gate_proj.weight": "model-00004-of-00101.safetensors", + "model.layers.5.mlp.experts.78.up_proj.weight": "model-00004-of-00101.safetensors", + "model.layers.5.mlp.experts.78.down_proj.weight": "model-00004-of-00101.safetensors", + "model.layers.5.mlp.experts.79.gate_proj.weight": "model-00004-of-00101.safetensors", + "model.layers.5.mlp.experts.79.up_proj.weight": "model-00004-of-00101.safetensors", + "model.layers.5.mlp.experts.79.down_proj.weight": "model-00004-of-00101.safetensors", + "model.layers.5.mlp.experts.80.gate_proj.weight": "model-00004-of-00101.safetensors", + "model.layers.5.mlp.experts.80.up_proj.weight": "model-00004-of-00101.safetensors", + "model.layers.5.mlp.experts.80.down_proj.weight": "model-00004-of-00101.safetensors", + "model.layers.5.mlp.experts.81.gate_proj.weight": "model-00004-of-00101.safetensors", + "model.layers.5.mlp.experts.81.up_proj.weight": "model-00004-of-00101.safetensors", + "model.layers.5.mlp.experts.81.down_proj.weight": "model-00004-of-00101.safetensors", + "model.layers.5.mlp.experts.82.gate_proj.weight": "model-00004-of-00101.safetensors", + "model.layers.5.mlp.experts.82.up_proj.weight": "model-00004-of-00101.safetensors", + "model.layers.5.mlp.experts.82.down_proj.weight": "model-00004-of-00101.safetensors", + "model.layers.5.mlp.experts.83.gate_proj.weight": "model-00004-of-00101.safetensors", + "model.layers.5.mlp.experts.83.up_proj.weight": "model-00004-of-00101.safetensors", + "model.layers.5.mlp.experts.83.down_proj.weight": "model-00004-of-00101.safetensors", + "model.layers.5.mlp.experts.84.gate_proj.weight": "model-00004-of-00101.safetensors", + "model.layers.5.mlp.experts.84.up_proj.weight": "model-00004-of-00101.safetensors", + "model.layers.5.mlp.experts.84.down_proj.weight": "model-00004-of-00101.safetensors", + "model.layers.5.mlp.experts.85.gate_proj.weight": "model-00004-of-00101.safetensors", + "model.layers.5.mlp.experts.85.up_proj.weight": "model-00004-of-00101.safetensors", + "model.layers.5.mlp.experts.85.down_proj.weight": "model-00004-of-00101.safetensors", + "model.layers.5.mlp.experts.86.gate_proj.weight": "model-00004-of-00101.safetensors", + "model.layers.5.mlp.experts.86.up_proj.weight": "model-00004-of-00101.safetensors", + "model.layers.5.mlp.experts.86.down_proj.weight": "model-00004-of-00101.safetensors", + "model.layers.5.mlp.experts.87.gate_proj.weight": "model-00004-of-00101.safetensors", + "model.layers.5.mlp.experts.87.up_proj.weight": "model-00004-of-00101.safetensors", + "model.layers.5.mlp.experts.87.down_proj.weight": "model-00004-of-00101.safetensors", + "model.layers.5.mlp.experts.88.gate_proj.weight": "model-00004-of-00101.safetensors", + "model.layers.5.mlp.experts.88.up_proj.weight": "model-00004-of-00101.safetensors", + "model.layers.5.mlp.experts.88.down_proj.weight": "model-00004-of-00101.safetensors", + "model.layers.5.mlp.experts.89.gate_proj.weight": "model-00004-of-00101.safetensors", + "model.layers.5.mlp.experts.89.up_proj.weight": "model-00004-of-00101.safetensors", + "model.layers.5.mlp.experts.89.down_proj.weight": "model-00004-of-00101.safetensors", + "model.layers.5.mlp.experts.90.gate_proj.weight": "model-00004-of-00101.safetensors", + "model.layers.5.mlp.experts.90.up_proj.weight": "model-00004-of-00101.safetensors", + "model.layers.5.mlp.experts.90.down_proj.weight": "model-00004-of-00101.safetensors", + "model.layers.5.mlp.experts.91.gate_proj.weight": "model-00004-of-00101.safetensors", + "model.layers.5.mlp.experts.91.up_proj.weight": "model-00004-of-00101.safetensors", + "model.layers.5.mlp.experts.91.down_proj.weight": "model-00004-of-00101.safetensors", + "model.layers.5.mlp.experts.92.gate_proj.weight": "model-00004-of-00101.safetensors", + "model.layers.5.mlp.experts.92.up_proj.weight": "model-00004-of-00101.safetensors", + "model.layers.5.mlp.experts.92.down_proj.weight": "model-00004-of-00101.safetensors", + "model.layers.5.mlp.experts.93.gate_proj.weight": "model-00004-of-00101.safetensors", + "model.layers.5.mlp.experts.93.up_proj.weight": "model-00004-of-00101.safetensors", + "model.layers.5.mlp.experts.93.down_proj.weight": "model-00004-of-00101.safetensors", + "model.layers.5.mlp.experts.94.gate_proj.weight": "model-00004-of-00101.safetensors", + "model.layers.5.mlp.experts.94.up_proj.weight": "model-00004-of-00101.safetensors", + "model.layers.5.mlp.experts.94.down_proj.weight": "model-00004-of-00101.safetensors", + "model.layers.5.mlp.experts.95.gate_proj.weight": "model-00004-of-00101.safetensors", + "model.layers.5.mlp.experts.95.up_proj.weight": "model-00004-of-00101.safetensors", + "model.layers.5.mlp.experts.95.down_proj.weight": "model-00004-of-00101.safetensors", + "model.layers.5.mlp.experts.96.gate_proj.weight": "model-00004-of-00101.safetensors", + "model.layers.5.mlp.experts.96.up_proj.weight": "model-00004-of-00101.safetensors", + "model.layers.5.mlp.experts.96.down_proj.weight": "model-00004-of-00101.safetensors", + "model.layers.5.mlp.experts.97.gate_proj.weight": "model-00004-of-00101.safetensors", + "model.layers.5.mlp.experts.97.up_proj.weight": "model-00004-of-00101.safetensors", + "model.layers.5.mlp.experts.97.down_proj.weight": "model-00004-of-00101.safetensors", + "model.layers.5.mlp.experts.98.gate_proj.weight": "model-00004-of-00101.safetensors", + "model.layers.5.mlp.experts.98.up_proj.weight": "model-00004-of-00101.safetensors", + "model.layers.5.mlp.experts.98.down_proj.weight": "model-00004-of-00101.safetensors", + "model.layers.5.mlp.experts.99.gate_proj.weight": "model-00004-of-00101.safetensors", + "model.layers.5.mlp.experts.99.up_proj.weight": "model-00004-of-00101.safetensors", + "model.layers.5.mlp.experts.99.down_proj.weight": "model-00004-of-00101.safetensors", + "model.layers.5.mlp.experts.100.gate_proj.weight": "model-00004-of-00101.safetensors", + "model.layers.5.mlp.experts.100.up_proj.weight": "model-00004-of-00101.safetensors", + "model.layers.5.mlp.experts.100.down_proj.weight": "model-00004-of-00101.safetensors", + "model.layers.5.mlp.experts.101.gate_proj.weight": "model-00004-of-00101.safetensors", + "model.layers.5.mlp.experts.101.up_proj.weight": "model-00004-of-00101.safetensors", + "model.layers.5.mlp.experts.101.down_proj.weight": "model-00004-of-00101.safetensors", + "model.layers.5.mlp.experts.102.gate_proj.weight": "model-00004-of-00101.safetensors", + "model.layers.5.mlp.experts.102.up_proj.weight": "model-00004-of-00101.safetensors", + "model.layers.5.mlp.experts.102.down_proj.weight": "model-00004-of-00101.safetensors", + "model.layers.5.mlp.experts.103.gate_proj.weight": "model-00004-of-00101.safetensors", + "model.layers.5.mlp.experts.103.up_proj.weight": "model-00004-of-00101.safetensors", + "model.layers.5.mlp.experts.103.down_proj.weight": "model-00004-of-00101.safetensors", + "model.layers.5.mlp.experts.104.gate_proj.weight": "model-00004-of-00101.safetensors", + "model.layers.5.mlp.experts.104.up_proj.weight": "model-00004-of-00101.safetensors", + "model.layers.5.mlp.experts.104.down_proj.weight": "model-00004-of-00101.safetensors", + "model.layers.5.mlp.experts.105.gate_proj.weight": "model-00004-of-00101.safetensors", + "model.layers.5.mlp.experts.105.up_proj.weight": "model-00004-of-00101.safetensors", + "model.layers.5.mlp.experts.105.down_proj.weight": "model-00004-of-00101.safetensors", + "model.layers.5.mlp.experts.106.gate_proj.weight": "model-00004-of-00101.safetensors", + "model.layers.5.mlp.experts.106.up_proj.weight": "model-00004-of-00101.safetensors", + "model.layers.5.mlp.experts.106.down_proj.weight": "model-00004-of-00101.safetensors", + "model.layers.5.mlp.experts.107.gate_proj.weight": "model-00004-of-00101.safetensors", + "model.layers.5.mlp.experts.107.up_proj.weight": "model-00004-of-00101.safetensors", + "model.layers.5.mlp.experts.107.down_proj.weight": "model-00004-of-00101.safetensors", + "model.layers.5.mlp.experts.108.gate_proj.weight": "model-00004-of-00101.safetensors", + "model.layers.5.mlp.experts.108.up_proj.weight": "model-00004-of-00101.safetensors", + "model.layers.5.mlp.experts.108.down_proj.weight": "model-00004-of-00101.safetensors", + "model.layers.5.mlp.experts.109.gate_proj.weight": "model-00004-of-00101.safetensors", + "model.layers.5.mlp.experts.109.up_proj.weight": "model-00004-of-00101.safetensors", + "model.layers.5.mlp.experts.109.down_proj.weight": "model-00004-of-00101.safetensors", + "model.layers.5.mlp.experts.110.gate_proj.weight": "model-00004-of-00101.safetensors", + "model.layers.5.mlp.experts.110.up_proj.weight": "model-00004-of-00101.safetensors", + "model.layers.5.mlp.experts.110.down_proj.weight": "model-00004-of-00101.safetensors", + "model.layers.5.mlp.experts.111.gate_proj.weight": "model-00004-of-00101.safetensors", + "model.layers.5.mlp.experts.111.up_proj.weight": "model-00004-of-00101.safetensors", + "model.layers.5.mlp.experts.111.down_proj.weight": "model-00004-of-00101.safetensors", + "model.layers.5.mlp.experts.112.gate_proj.weight": "model-00004-of-00101.safetensors", + "model.layers.5.mlp.experts.112.up_proj.weight": "model-00004-of-00101.safetensors", + "model.layers.5.mlp.experts.112.down_proj.weight": "model-00004-of-00101.safetensors", + "model.layers.5.mlp.experts.113.gate_proj.weight": "model-00004-of-00101.safetensors", + "model.layers.5.mlp.experts.113.up_proj.weight": "model-00004-of-00101.safetensors", + "model.layers.5.mlp.experts.113.down_proj.weight": "model-00004-of-00101.safetensors", + "model.layers.5.mlp.experts.114.gate_proj.weight": "model-00004-of-00101.safetensors", + "model.layers.5.mlp.experts.114.up_proj.weight": "model-00004-of-00101.safetensors", + "model.layers.5.mlp.experts.114.down_proj.weight": "model-00004-of-00101.safetensors", + "model.layers.5.mlp.experts.115.gate_proj.weight": "model-00004-of-00101.safetensors", + "model.layers.5.mlp.experts.115.up_proj.weight": "model-00004-of-00101.safetensors", + "model.layers.5.mlp.experts.115.down_proj.weight": "model-00004-of-00101.safetensors", + "model.layers.5.mlp.experts.116.gate_proj.weight": "model-00004-of-00101.safetensors", + "model.layers.5.mlp.experts.116.up_proj.weight": "model-00004-of-00101.safetensors", + "model.layers.5.mlp.experts.116.down_proj.weight": "model-00004-of-00101.safetensors", + "model.layers.5.mlp.experts.117.gate_proj.weight": "model-00004-of-00101.safetensors", + "model.layers.5.mlp.experts.117.up_proj.weight": "model-00004-of-00101.safetensors", + "model.layers.5.mlp.experts.117.down_proj.weight": "model-00004-of-00101.safetensors", + "model.layers.5.mlp.experts.118.gate_proj.weight": "model-00004-of-00101.safetensors", + "model.layers.5.mlp.experts.118.up_proj.weight": "model-00004-of-00101.safetensors", + "model.layers.5.mlp.experts.118.down_proj.weight": "model-00004-of-00101.safetensors", + "model.layers.5.mlp.experts.119.gate_proj.weight": "model-00004-of-00101.safetensors", + "model.layers.5.mlp.experts.119.up_proj.weight": "model-00004-of-00101.safetensors", + "model.layers.5.mlp.experts.119.down_proj.weight": "model-00004-of-00101.safetensors", + "model.layers.5.mlp.gate.weight": "model-00004-of-00101.safetensors", + "model.layers.5.mlp.gate.e_score_correction_bias": "model-00004-of-00101.safetensors", + "model.layers.5.mlp.shared_experts.gate_proj.weight": "model-00004-of-00101.safetensors", + "model.layers.5.mlp.shared_experts.up_proj.weight": "model-00004-of-00101.safetensors", + "model.layers.5.mlp.shared_experts.down_proj.weight": "model-00005-of-00101.safetensors", + "model.layers.5.input_layernorm.weight": "model-00005-of-00101.safetensors", + "model.layers.5.post_attention_layernorm.weight": "model-00005-of-00101.safetensors", + "model.layers.6.self_attn.q_proj.weight": "model-00005-of-00101.safetensors", + "model.layers.6.self_attn.q_proj.bias": "model-00005-of-00101.safetensors", + "model.layers.6.self_attn.k_proj.weight": "model-00005-of-00101.safetensors", + "model.layers.6.self_attn.k_proj.bias": "model-00005-of-00101.safetensors", + "model.layers.6.self_attn.v_proj.weight": "model-00005-of-00101.safetensors", + "model.layers.6.self_attn.v_proj.bias": "model-00005-of-00101.safetensors", + "model.layers.6.self_attn.o_proj.weight": "model-00005-of-00101.safetensors", + "model.layers.6.self_attn.q_norm.weight": "model-00005-of-00101.safetensors", + "model.layers.6.self_attn.k_norm.weight": "model-00005-of-00101.safetensors", + "model.layers.6.mlp.experts.0.gate_proj.weight": "model-00005-of-00101.safetensors", + "model.layers.6.mlp.experts.0.up_proj.weight": "model-00005-of-00101.safetensors", + "model.layers.6.mlp.experts.0.down_proj.weight": "model-00005-of-00101.safetensors", + "model.layers.6.mlp.experts.1.gate_proj.weight": "model-00005-of-00101.safetensors", + "model.layers.6.mlp.experts.1.up_proj.weight": "model-00005-of-00101.safetensors", + "model.layers.6.mlp.experts.1.down_proj.weight": "model-00005-of-00101.safetensors", + "model.layers.6.mlp.experts.2.gate_proj.weight": "model-00005-of-00101.safetensors", + "model.layers.6.mlp.experts.2.up_proj.weight": "model-00005-of-00101.safetensors", + "model.layers.6.mlp.experts.2.down_proj.weight": "model-00005-of-00101.safetensors", + "model.layers.6.mlp.experts.3.gate_proj.weight": "model-00005-of-00101.safetensors", + "model.layers.6.mlp.experts.3.up_proj.weight": "model-00005-of-00101.safetensors", + "model.layers.6.mlp.experts.3.down_proj.weight": "model-00005-of-00101.safetensors", + "model.layers.6.mlp.experts.4.gate_proj.weight": "model-00005-of-00101.safetensors", + "model.layers.6.mlp.experts.4.up_proj.weight": "model-00005-of-00101.safetensors", + "model.layers.6.mlp.experts.4.down_proj.weight": "model-00005-of-00101.safetensors", + "model.layers.6.mlp.experts.5.gate_proj.weight": "model-00005-of-00101.safetensors", + "model.layers.6.mlp.experts.5.up_proj.weight": "model-00005-of-00101.safetensors", + "model.layers.6.mlp.experts.5.down_proj.weight": "model-00005-of-00101.safetensors", + "model.layers.6.mlp.experts.6.gate_proj.weight": "model-00005-of-00101.safetensors", + "model.layers.6.mlp.experts.6.up_proj.weight": "model-00005-of-00101.safetensors", + "model.layers.6.mlp.experts.6.down_proj.weight": "model-00005-of-00101.safetensors", + "model.layers.6.mlp.experts.7.gate_proj.weight": "model-00005-of-00101.safetensors", + "model.layers.6.mlp.experts.7.up_proj.weight": "model-00005-of-00101.safetensors", + "model.layers.6.mlp.experts.7.down_proj.weight": "model-00005-of-00101.safetensors", + "model.layers.6.mlp.experts.8.gate_proj.weight": "model-00005-of-00101.safetensors", + "model.layers.6.mlp.experts.8.up_proj.weight": "model-00005-of-00101.safetensors", + "model.layers.6.mlp.experts.8.down_proj.weight": "model-00005-of-00101.safetensors", + "model.layers.6.mlp.experts.9.gate_proj.weight": "model-00005-of-00101.safetensors", + "model.layers.6.mlp.experts.9.up_proj.weight": "model-00005-of-00101.safetensors", + "model.layers.6.mlp.experts.9.down_proj.weight": "model-00005-of-00101.safetensors", + "model.layers.6.mlp.experts.10.gate_proj.weight": "model-00005-of-00101.safetensors", + "model.layers.6.mlp.experts.10.up_proj.weight": "model-00005-of-00101.safetensors", + "model.layers.6.mlp.experts.10.down_proj.weight": "model-00005-of-00101.safetensors", + "model.layers.6.mlp.experts.11.gate_proj.weight": "model-00005-of-00101.safetensors", + "model.layers.6.mlp.experts.11.up_proj.weight": "model-00005-of-00101.safetensors", + "model.layers.6.mlp.experts.11.down_proj.weight": "model-00005-of-00101.safetensors", + "model.layers.6.mlp.experts.12.gate_proj.weight": "model-00005-of-00101.safetensors", + "model.layers.6.mlp.experts.12.up_proj.weight": "model-00005-of-00101.safetensors", + "model.layers.6.mlp.experts.12.down_proj.weight": "model-00005-of-00101.safetensors", + "model.layers.6.mlp.experts.13.gate_proj.weight": "model-00005-of-00101.safetensors", + "model.layers.6.mlp.experts.13.up_proj.weight": "model-00005-of-00101.safetensors", + "model.layers.6.mlp.experts.13.down_proj.weight": "model-00005-of-00101.safetensors", + "model.layers.6.mlp.experts.14.gate_proj.weight": "model-00005-of-00101.safetensors", + "model.layers.6.mlp.experts.14.up_proj.weight": "model-00005-of-00101.safetensors", + "model.layers.6.mlp.experts.14.down_proj.weight": "model-00005-of-00101.safetensors", + "model.layers.6.mlp.experts.15.gate_proj.weight": "model-00005-of-00101.safetensors", + "model.layers.6.mlp.experts.15.up_proj.weight": "model-00005-of-00101.safetensors", + "model.layers.6.mlp.experts.15.down_proj.weight": "model-00005-of-00101.safetensors", + "model.layers.6.mlp.experts.16.gate_proj.weight": "model-00005-of-00101.safetensors", + "model.layers.6.mlp.experts.16.up_proj.weight": "model-00005-of-00101.safetensors", + "model.layers.6.mlp.experts.16.down_proj.weight": "model-00005-of-00101.safetensors", + "model.layers.6.mlp.experts.17.gate_proj.weight": "model-00005-of-00101.safetensors", + "model.layers.6.mlp.experts.17.up_proj.weight": "model-00005-of-00101.safetensors", + "model.layers.6.mlp.experts.17.down_proj.weight": "model-00005-of-00101.safetensors", + "model.layers.6.mlp.experts.18.gate_proj.weight": "model-00005-of-00101.safetensors", + "model.layers.6.mlp.experts.18.up_proj.weight": "model-00005-of-00101.safetensors", + "model.layers.6.mlp.experts.18.down_proj.weight": "model-00005-of-00101.safetensors", + "model.layers.6.mlp.experts.19.gate_proj.weight": "model-00005-of-00101.safetensors", + "model.layers.6.mlp.experts.19.up_proj.weight": "model-00005-of-00101.safetensors", + "model.layers.6.mlp.experts.19.down_proj.weight": "model-00005-of-00101.safetensors", + "model.layers.6.mlp.experts.20.gate_proj.weight": "model-00005-of-00101.safetensors", + "model.layers.6.mlp.experts.20.up_proj.weight": "model-00005-of-00101.safetensors", + "model.layers.6.mlp.experts.20.down_proj.weight": "model-00005-of-00101.safetensors", + "model.layers.6.mlp.experts.21.gate_proj.weight": "model-00005-of-00101.safetensors", + "model.layers.6.mlp.experts.21.up_proj.weight": "model-00005-of-00101.safetensors", + "model.layers.6.mlp.experts.21.down_proj.weight": "model-00005-of-00101.safetensors", + "model.layers.6.mlp.experts.22.gate_proj.weight": "model-00005-of-00101.safetensors", + "model.layers.6.mlp.experts.22.up_proj.weight": "model-00005-of-00101.safetensors", + "model.layers.6.mlp.experts.22.down_proj.weight": "model-00005-of-00101.safetensors", + "model.layers.6.mlp.experts.23.gate_proj.weight": "model-00005-of-00101.safetensors", + "model.layers.6.mlp.experts.23.up_proj.weight": "model-00005-of-00101.safetensors", + "model.layers.6.mlp.experts.23.down_proj.weight": "model-00005-of-00101.safetensors", + "model.layers.6.mlp.experts.24.gate_proj.weight": "model-00005-of-00101.safetensors", + "model.layers.6.mlp.experts.24.up_proj.weight": "model-00005-of-00101.safetensors", + "model.layers.6.mlp.experts.24.down_proj.weight": "model-00005-of-00101.safetensors", + "model.layers.6.mlp.experts.25.gate_proj.weight": "model-00005-of-00101.safetensors", + "model.layers.6.mlp.experts.25.up_proj.weight": "model-00005-of-00101.safetensors", + "model.layers.6.mlp.experts.25.down_proj.weight": "model-00005-of-00101.safetensors", + "model.layers.6.mlp.experts.26.gate_proj.weight": "model-00005-of-00101.safetensors", + "model.layers.6.mlp.experts.26.up_proj.weight": "model-00005-of-00101.safetensors", + "model.layers.6.mlp.experts.26.down_proj.weight": "model-00005-of-00101.safetensors", + "model.layers.6.mlp.experts.27.gate_proj.weight": "model-00005-of-00101.safetensors", + "model.layers.6.mlp.experts.27.up_proj.weight": "model-00005-of-00101.safetensors", + "model.layers.6.mlp.experts.27.down_proj.weight": "model-00005-of-00101.safetensors", + "model.layers.6.mlp.experts.28.gate_proj.weight": "model-00005-of-00101.safetensors", + "model.layers.6.mlp.experts.28.up_proj.weight": "model-00005-of-00101.safetensors", + "model.layers.6.mlp.experts.28.down_proj.weight": "model-00005-of-00101.safetensors", + "model.layers.6.mlp.experts.29.gate_proj.weight": "model-00005-of-00101.safetensors", + "model.layers.6.mlp.experts.29.up_proj.weight": "model-00005-of-00101.safetensors", + "model.layers.6.mlp.experts.29.down_proj.weight": "model-00005-of-00101.safetensors", + "model.layers.6.mlp.experts.30.gate_proj.weight": "model-00005-of-00101.safetensors", + "model.layers.6.mlp.experts.30.up_proj.weight": "model-00005-of-00101.safetensors", + "model.layers.6.mlp.experts.30.down_proj.weight": "model-00005-of-00101.safetensors", + "model.layers.6.mlp.experts.31.gate_proj.weight": "model-00005-of-00101.safetensors", + "model.layers.6.mlp.experts.31.up_proj.weight": "model-00005-of-00101.safetensors", + "model.layers.6.mlp.experts.31.down_proj.weight": "model-00005-of-00101.safetensors", + "model.layers.6.mlp.experts.32.gate_proj.weight": "model-00005-of-00101.safetensors", + "model.layers.6.mlp.experts.32.up_proj.weight": "model-00005-of-00101.safetensors", + "model.layers.6.mlp.experts.32.down_proj.weight": "model-00005-of-00101.safetensors", + "model.layers.6.mlp.experts.33.gate_proj.weight": "model-00005-of-00101.safetensors", + "model.layers.6.mlp.experts.33.up_proj.weight": "model-00005-of-00101.safetensors", + "model.layers.6.mlp.experts.33.down_proj.weight": "model-00005-of-00101.safetensors", + "model.layers.6.mlp.experts.34.gate_proj.weight": "model-00005-of-00101.safetensors", + "model.layers.6.mlp.experts.34.up_proj.weight": "model-00005-of-00101.safetensors", + "model.layers.6.mlp.experts.34.down_proj.weight": "model-00005-of-00101.safetensors", + "model.layers.6.mlp.experts.35.gate_proj.weight": "model-00005-of-00101.safetensors", + "model.layers.6.mlp.experts.35.up_proj.weight": "model-00005-of-00101.safetensors", + "model.layers.6.mlp.experts.35.down_proj.weight": "model-00005-of-00101.safetensors", + "model.layers.6.mlp.experts.36.gate_proj.weight": "model-00005-of-00101.safetensors", + "model.layers.6.mlp.experts.36.up_proj.weight": "model-00005-of-00101.safetensors", + "model.layers.6.mlp.experts.36.down_proj.weight": "model-00005-of-00101.safetensors", + "model.layers.6.mlp.experts.37.gate_proj.weight": "model-00005-of-00101.safetensors", + "model.layers.6.mlp.experts.37.up_proj.weight": "model-00005-of-00101.safetensors", + "model.layers.6.mlp.experts.37.down_proj.weight": "model-00005-of-00101.safetensors", + "model.layers.6.mlp.experts.38.gate_proj.weight": "model-00005-of-00101.safetensors", + "model.layers.6.mlp.experts.38.up_proj.weight": "model-00005-of-00101.safetensors", + "model.layers.6.mlp.experts.38.down_proj.weight": "model-00005-of-00101.safetensors", + "model.layers.6.mlp.experts.39.gate_proj.weight": "model-00005-of-00101.safetensors", + "model.layers.6.mlp.experts.39.up_proj.weight": "model-00005-of-00101.safetensors", + "model.layers.6.mlp.experts.39.down_proj.weight": "model-00005-of-00101.safetensors", + "model.layers.6.mlp.experts.40.gate_proj.weight": "model-00005-of-00101.safetensors", + "model.layers.6.mlp.experts.40.up_proj.weight": "model-00005-of-00101.safetensors", + "model.layers.6.mlp.experts.40.down_proj.weight": "model-00005-of-00101.safetensors", + "model.layers.6.mlp.experts.41.gate_proj.weight": "model-00005-of-00101.safetensors", + "model.layers.6.mlp.experts.41.up_proj.weight": "model-00005-of-00101.safetensors", + "model.layers.6.mlp.experts.41.down_proj.weight": "model-00005-of-00101.safetensors", + "model.layers.6.mlp.experts.42.gate_proj.weight": "model-00005-of-00101.safetensors", + "model.layers.6.mlp.experts.42.up_proj.weight": "model-00005-of-00101.safetensors", + "model.layers.6.mlp.experts.42.down_proj.weight": "model-00005-of-00101.safetensors", + "model.layers.6.mlp.experts.43.gate_proj.weight": "model-00005-of-00101.safetensors", + "model.layers.6.mlp.experts.43.up_proj.weight": "model-00005-of-00101.safetensors", + "model.layers.6.mlp.experts.43.down_proj.weight": "model-00005-of-00101.safetensors", + "model.layers.6.mlp.experts.44.gate_proj.weight": "model-00005-of-00101.safetensors", + "model.layers.6.mlp.experts.44.up_proj.weight": "model-00005-of-00101.safetensors", + "model.layers.6.mlp.experts.44.down_proj.weight": "model-00005-of-00101.safetensors", + "model.layers.6.mlp.experts.45.gate_proj.weight": "model-00005-of-00101.safetensors", + "model.layers.6.mlp.experts.45.up_proj.weight": "model-00005-of-00101.safetensors", + "model.layers.6.mlp.experts.45.down_proj.weight": "model-00005-of-00101.safetensors", + "model.layers.6.mlp.experts.46.gate_proj.weight": "model-00005-of-00101.safetensors", + "model.layers.6.mlp.experts.46.up_proj.weight": "model-00005-of-00101.safetensors", + "model.layers.6.mlp.experts.46.down_proj.weight": "model-00005-of-00101.safetensors", + "model.layers.6.mlp.experts.47.gate_proj.weight": "model-00005-of-00101.safetensors", + "model.layers.6.mlp.experts.47.up_proj.weight": "model-00005-of-00101.safetensors", + "model.layers.6.mlp.experts.47.down_proj.weight": "model-00005-of-00101.safetensors", + "model.layers.6.mlp.experts.48.gate_proj.weight": "model-00005-of-00101.safetensors", + "model.layers.6.mlp.experts.48.up_proj.weight": "model-00005-of-00101.safetensors", + "model.layers.6.mlp.experts.48.down_proj.weight": "model-00005-of-00101.safetensors", + "model.layers.6.mlp.experts.49.gate_proj.weight": "model-00005-of-00101.safetensors", + "model.layers.6.mlp.experts.49.up_proj.weight": "model-00005-of-00101.safetensors", + "model.layers.6.mlp.experts.49.down_proj.weight": "model-00005-of-00101.safetensors", + "model.layers.6.mlp.experts.50.gate_proj.weight": "model-00005-of-00101.safetensors", + "model.layers.6.mlp.experts.50.up_proj.weight": "model-00005-of-00101.safetensors", + "model.layers.6.mlp.experts.50.down_proj.weight": "model-00005-of-00101.safetensors", + "model.layers.6.mlp.experts.51.gate_proj.weight": "model-00005-of-00101.safetensors", + "model.layers.6.mlp.experts.51.up_proj.weight": "model-00005-of-00101.safetensors", + "model.layers.6.mlp.experts.51.down_proj.weight": "model-00005-of-00101.safetensors", + "model.layers.6.mlp.experts.52.gate_proj.weight": "model-00005-of-00101.safetensors", + "model.layers.6.mlp.experts.52.up_proj.weight": "model-00005-of-00101.safetensors", + "model.layers.6.mlp.experts.52.down_proj.weight": "model-00005-of-00101.safetensors", + "model.layers.6.mlp.experts.53.gate_proj.weight": "model-00005-of-00101.safetensors", + "model.layers.6.mlp.experts.53.up_proj.weight": "model-00005-of-00101.safetensors", + "model.layers.6.mlp.experts.53.down_proj.weight": "model-00005-of-00101.safetensors", + "model.layers.6.mlp.experts.54.gate_proj.weight": "model-00005-of-00101.safetensors", + "model.layers.6.mlp.experts.54.up_proj.weight": "model-00005-of-00101.safetensors", + "model.layers.6.mlp.experts.54.down_proj.weight": "model-00005-of-00101.safetensors", + "model.layers.6.mlp.experts.55.gate_proj.weight": "model-00005-of-00101.safetensors", + "model.layers.6.mlp.experts.55.up_proj.weight": "model-00005-of-00101.safetensors", + "model.layers.6.mlp.experts.55.down_proj.weight": "model-00005-of-00101.safetensors", + "model.layers.6.mlp.experts.56.gate_proj.weight": "model-00005-of-00101.safetensors", + "model.layers.6.mlp.experts.56.up_proj.weight": "model-00005-of-00101.safetensors", + "model.layers.6.mlp.experts.56.down_proj.weight": "model-00005-of-00101.safetensors", + "model.layers.6.mlp.experts.57.gate_proj.weight": "model-00005-of-00101.safetensors", + "model.layers.6.mlp.experts.57.up_proj.weight": "model-00005-of-00101.safetensors", + "model.layers.6.mlp.experts.57.down_proj.weight": "model-00005-of-00101.safetensors", + "model.layers.6.mlp.experts.58.gate_proj.weight": "model-00005-of-00101.safetensors", + "model.layers.6.mlp.experts.58.up_proj.weight": "model-00005-of-00101.safetensors", + "model.layers.6.mlp.experts.58.down_proj.weight": "model-00005-of-00101.safetensors", + "model.layers.6.mlp.experts.59.gate_proj.weight": "model-00005-of-00101.safetensors", + "model.layers.6.mlp.experts.59.up_proj.weight": "model-00005-of-00101.safetensors", + "model.layers.6.mlp.experts.59.down_proj.weight": "model-00005-of-00101.safetensors", + "model.layers.6.mlp.experts.60.gate_proj.weight": "model-00005-of-00101.safetensors", + "model.layers.6.mlp.experts.60.up_proj.weight": "model-00005-of-00101.safetensors", + "model.layers.6.mlp.experts.60.down_proj.weight": "model-00005-of-00101.safetensors", + "model.layers.6.mlp.experts.61.gate_proj.weight": "model-00005-of-00101.safetensors", + "model.layers.6.mlp.experts.61.up_proj.weight": "model-00005-of-00101.safetensors", + "model.layers.6.mlp.experts.61.down_proj.weight": "model-00005-of-00101.safetensors", + "model.layers.6.mlp.experts.62.gate_proj.weight": "model-00005-of-00101.safetensors", + "model.layers.6.mlp.experts.62.up_proj.weight": "model-00005-of-00101.safetensors", + "model.layers.6.mlp.experts.62.down_proj.weight": "model-00005-of-00101.safetensors", + "model.layers.6.mlp.experts.63.gate_proj.weight": "model-00005-of-00101.safetensors", + "model.layers.6.mlp.experts.63.up_proj.weight": "model-00005-of-00101.safetensors", + "model.layers.6.mlp.experts.63.down_proj.weight": "model-00005-of-00101.safetensors", + "model.layers.6.mlp.experts.64.gate_proj.weight": "model-00005-of-00101.safetensors", + "model.layers.6.mlp.experts.64.up_proj.weight": "model-00005-of-00101.safetensors", + "model.layers.6.mlp.experts.64.down_proj.weight": "model-00005-of-00101.safetensors", + "model.layers.6.mlp.experts.65.gate_proj.weight": "model-00005-of-00101.safetensors", + "model.layers.6.mlp.experts.65.up_proj.weight": "model-00005-of-00101.safetensors", + "model.layers.6.mlp.experts.65.down_proj.weight": "model-00005-of-00101.safetensors", + "model.layers.6.mlp.experts.66.gate_proj.weight": "model-00005-of-00101.safetensors", + "model.layers.6.mlp.experts.66.up_proj.weight": "model-00005-of-00101.safetensors", + "model.layers.6.mlp.experts.66.down_proj.weight": "model-00005-of-00101.safetensors", + "model.layers.6.mlp.experts.67.gate_proj.weight": "model-00005-of-00101.safetensors", + "model.layers.6.mlp.experts.67.up_proj.weight": "model-00005-of-00101.safetensors", + "model.layers.6.mlp.experts.67.down_proj.weight": "model-00005-of-00101.safetensors", + "model.layers.6.mlp.experts.68.gate_proj.weight": "model-00005-of-00101.safetensors", + "model.layers.6.mlp.experts.68.up_proj.weight": "model-00005-of-00101.safetensors", + "model.layers.6.mlp.experts.68.down_proj.weight": "model-00005-of-00101.safetensors", + "model.layers.6.mlp.experts.69.gate_proj.weight": "model-00005-of-00101.safetensors", + "model.layers.6.mlp.experts.69.up_proj.weight": "model-00005-of-00101.safetensors", + "model.layers.6.mlp.experts.69.down_proj.weight": "model-00005-of-00101.safetensors", + "model.layers.6.mlp.experts.70.gate_proj.weight": "model-00005-of-00101.safetensors", + "model.layers.6.mlp.experts.70.up_proj.weight": "model-00005-of-00101.safetensors", + "model.layers.6.mlp.experts.70.down_proj.weight": "model-00005-of-00101.safetensors", + "model.layers.6.mlp.experts.71.gate_proj.weight": "model-00005-of-00101.safetensors", + "model.layers.6.mlp.experts.71.up_proj.weight": "model-00005-of-00101.safetensors", + "model.layers.6.mlp.experts.71.down_proj.weight": "model-00005-of-00101.safetensors", + "model.layers.6.mlp.experts.72.gate_proj.weight": "model-00005-of-00101.safetensors", + "model.layers.6.mlp.experts.72.up_proj.weight": "model-00005-of-00101.safetensors", + "model.layers.6.mlp.experts.72.down_proj.weight": "model-00005-of-00101.safetensors", + "model.layers.6.mlp.experts.73.gate_proj.weight": "model-00005-of-00101.safetensors", + "model.layers.6.mlp.experts.73.up_proj.weight": "model-00005-of-00101.safetensors", + "model.layers.6.mlp.experts.73.down_proj.weight": "model-00005-of-00101.safetensors", + "model.layers.6.mlp.experts.74.gate_proj.weight": "model-00005-of-00101.safetensors", + "model.layers.6.mlp.experts.74.up_proj.weight": "model-00005-of-00101.safetensors", + "model.layers.6.mlp.experts.74.down_proj.weight": "model-00005-of-00101.safetensors", + "model.layers.6.mlp.experts.75.gate_proj.weight": "model-00005-of-00101.safetensors", + "model.layers.6.mlp.experts.75.up_proj.weight": "model-00005-of-00101.safetensors", + "model.layers.6.mlp.experts.75.down_proj.weight": "model-00005-of-00101.safetensors", + "model.layers.6.mlp.experts.76.gate_proj.weight": "model-00005-of-00101.safetensors", + "model.layers.6.mlp.experts.76.up_proj.weight": "model-00005-of-00101.safetensors", + "model.layers.6.mlp.experts.76.down_proj.weight": "model-00005-of-00101.safetensors", + "model.layers.6.mlp.experts.77.gate_proj.weight": "model-00005-of-00101.safetensors", + "model.layers.6.mlp.experts.77.up_proj.weight": "model-00005-of-00101.safetensors", + "model.layers.6.mlp.experts.77.down_proj.weight": "model-00005-of-00101.safetensors", + "model.layers.6.mlp.experts.78.gate_proj.weight": "model-00005-of-00101.safetensors", + "model.layers.6.mlp.experts.78.up_proj.weight": "model-00005-of-00101.safetensors", + "model.layers.6.mlp.experts.78.down_proj.weight": "model-00005-of-00101.safetensors", + "model.layers.6.mlp.experts.79.gate_proj.weight": "model-00005-of-00101.safetensors", + "model.layers.6.mlp.experts.79.up_proj.weight": "model-00005-of-00101.safetensors", + "model.layers.6.mlp.experts.79.down_proj.weight": "model-00005-of-00101.safetensors", + "model.layers.6.mlp.experts.80.gate_proj.weight": "model-00005-of-00101.safetensors", + "model.layers.6.mlp.experts.80.up_proj.weight": "model-00005-of-00101.safetensors", + "model.layers.6.mlp.experts.80.down_proj.weight": "model-00005-of-00101.safetensors", + "model.layers.6.mlp.experts.81.gate_proj.weight": "model-00005-of-00101.safetensors", + "model.layers.6.mlp.experts.81.up_proj.weight": "model-00005-of-00101.safetensors", + "model.layers.6.mlp.experts.81.down_proj.weight": "model-00005-of-00101.safetensors", + "model.layers.6.mlp.experts.82.gate_proj.weight": "model-00005-of-00101.safetensors", + "model.layers.6.mlp.experts.82.up_proj.weight": "model-00005-of-00101.safetensors", + "model.layers.6.mlp.experts.82.down_proj.weight": "model-00005-of-00101.safetensors", + "model.layers.6.mlp.experts.83.gate_proj.weight": "model-00005-of-00101.safetensors", + "model.layers.6.mlp.experts.83.up_proj.weight": "model-00005-of-00101.safetensors", + "model.layers.6.mlp.experts.83.down_proj.weight": "model-00005-of-00101.safetensors", + "model.layers.6.mlp.experts.84.gate_proj.weight": "model-00005-of-00101.safetensors", + "model.layers.6.mlp.experts.84.up_proj.weight": "model-00005-of-00101.safetensors", + "model.layers.6.mlp.experts.84.down_proj.weight": "model-00005-of-00101.safetensors", + "model.layers.6.mlp.experts.85.gate_proj.weight": "model-00005-of-00101.safetensors", + "model.layers.6.mlp.experts.85.up_proj.weight": "model-00005-of-00101.safetensors", + "model.layers.6.mlp.experts.85.down_proj.weight": "model-00005-of-00101.safetensors", + "model.layers.6.mlp.experts.86.gate_proj.weight": "model-00005-of-00101.safetensors", + "model.layers.6.mlp.experts.86.up_proj.weight": "model-00005-of-00101.safetensors", + "model.layers.6.mlp.experts.86.down_proj.weight": "model-00005-of-00101.safetensors", + "model.layers.6.mlp.experts.87.gate_proj.weight": "model-00005-of-00101.safetensors", + "model.layers.6.mlp.experts.87.up_proj.weight": "model-00005-of-00101.safetensors", + "model.layers.6.mlp.experts.87.down_proj.weight": "model-00005-of-00101.safetensors", + "model.layers.6.mlp.experts.88.gate_proj.weight": "model-00005-of-00101.safetensors", + "model.layers.6.mlp.experts.88.up_proj.weight": "model-00005-of-00101.safetensors", + "model.layers.6.mlp.experts.88.down_proj.weight": "model-00005-of-00101.safetensors", + "model.layers.6.mlp.experts.89.gate_proj.weight": "model-00005-of-00101.safetensors", + "model.layers.6.mlp.experts.89.up_proj.weight": "model-00005-of-00101.safetensors", + "model.layers.6.mlp.experts.89.down_proj.weight": "model-00005-of-00101.safetensors", + "model.layers.6.mlp.experts.90.gate_proj.weight": "model-00005-of-00101.safetensors", + "model.layers.6.mlp.experts.90.up_proj.weight": "model-00005-of-00101.safetensors", + "model.layers.6.mlp.experts.90.down_proj.weight": "model-00005-of-00101.safetensors", + "model.layers.6.mlp.experts.91.gate_proj.weight": "model-00005-of-00101.safetensors", + "model.layers.6.mlp.experts.91.up_proj.weight": "model-00005-of-00101.safetensors", + "model.layers.6.mlp.experts.91.down_proj.weight": "model-00005-of-00101.safetensors", + "model.layers.6.mlp.experts.92.gate_proj.weight": "model-00005-of-00101.safetensors", + "model.layers.6.mlp.experts.92.up_proj.weight": "model-00005-of-00101.safetensors", + "model.layers.6.mlp.experts.92.down_proj.weight": "model-00005-of-00101.safetensors", + "model.layers.6.mlp.experts.93.gate_proj.weight": "model-00005-of-00101.safetensors", + "model.layers.6.mlp.experts.93.up_proj.weight": "model-00005-of-00101.safetensors", + "model.layers.6.mlp.experts.93.down_proj.weight": "model-00005-of-00101.safetensors", + "model.layers.6.mlp.experts.94.gate_proj.weight": "model-00005-of-00101.safetensors", + "model.layers.6.mlp.experts.94.up_proj.weight": "model-00005-of-00101.safetensors", + "model.layers.6.mlp.experts.94.down_proj.weight": "model-00005-of-00101.safetensors", + "model.layers.6.mlp.experts.95.gate_proj.weight": "model-00005-of-00101.safetensors", + "model.layers.6.mlp.experts.95.up_proj.weight": "model-00005-of-00101.safetensors", + "model.layers.6.mlp.experts.95.down_proj.weight": "model-00005-of-00101.safetensors", + "model.layers.6.mlp.experts.96.gate_proj.weight": "model-00005-of-00101.safetensors", + "model.layers.6.mlp.experts.96.up_proj.weight": "model-00005-of-00101.safetensors", + "model.layers.6.mlp.experts.96.down_proj.weight": "model-00005-of-00101.safetensors", + "model.layers.6.mlp.experts.97.gate_proj.weight": "model-00005-of-00101.safetensors", + "model.layers.6.mlp.experts.97.up_proj.weight": "model-00005-of-00101.safetensors", + "model.layers.6.mlp.experts.97.down_proj.weight": "model-00005-of-00101.safetensors", + "model.layers.6.mlp.experts.98.gate_proj.weight": "model-00005-of-00101.safetensors", + "model.layers.6.mlp.experts.98.up_proj.weight": "model-00005-of-00101.safetensors", + "model.layers.6.mlp.experts.98.down_proj.weight": "model-00005-of-00101.safetensors", + "model.layers.6.mlp.experts.99.gate_proj.weight": "model-00005-of-00101.safetensors", + "model.layers.6.mlp.experts.99.up_proj.weight": "model-00005-of-00101.safetensors", + "model.layers.6.mlp.experts.99.down_proj.weight": "model-00005-of-00101.safetensors", + "model.layers.6.mlp.experts.100.gate_proj.weight": "model-00005-of-00101.safetensors", + "model.layers.6.mlp.experts.100.up_proj.weight": "model-00005-of-00101.safetensors", + "model.layers.6.mlp.experts.100.down_proj.weight": "model-00005-of-00101.safetensors", + "model.layers.6.mlp.experts.101.gate_proj.weight": "model-00005-of-00101.safetensors", + "model.layers.6.mlp.experts.101.up_proj.weight": "model-00005-of-00101.safetensors", + "model.layers.6.mlp.experts.101.down_proj.weight": "model-00005-of-00101.safetensors", + "model.layers.6.mlp.experts.102.gate_proj.weight": "model-00005-of-00101.safetensors", + "model.layers.6.mlp.experts.102.up_proj.weight": "model-00005-of-00101.safetensors", + "model.layers.6.mlp.experts.102.down_proj.weight": "model-00005-of-00101.safetensors", + "model.layers.6.mlp.experts.103.gate_proj.weight": "model-00005-of-00101.safetensors", + "model.layers.6.mlp.experts.103.up_proj.weight": "model-00005-of-00101.safetensors", + "model.layers.6.mlp.experts.103.down_proj.weight": "model-00005-of-00101.safetensors", + "model.layers.6.mlp.experts.104.gate_proj.weight": "model-00005-of-00101.safetensors", + "model.layers.6.mlp.experts.104.up_proj.weight": "model-00005-of-00101.safetensors", + "model.layers.6.mlp.experts.104.down_proj.weight": "model-00005-of-00101.safetensors", + "model.layers.6.mlp.experts.105.gate_proj.weight": "model-00005-of-00101.safetensors", + "model.layers.6.mlp.experts.105.up_proj.weight": "model-00005-of-00101.safetensors", + "model.layers.6.mlp.experts.105.down_proj.weight": "model-00005-of-00101.safetensors", + "model.layers.6.mlp.experts.106.gate_proj.weight": "model-00005-of-00101.safetensors", + "model.layers.6.mlp.experts.106.up_proj.weight": "model-00005-of-00101.safetensors", + "model.layers.6.mlp.experts.106.down_proj.weight": "model-00005-of-00101.safetensors", + "model.layers.6.mlp.experts.107.gate_proj.weight": "model-00005-of-00101.safetensors", + "model.layers.6.mlp.experts.107.up_proj.weight": "model-00006-of-00101.safetensors", + "model.layers.6.mlp.experts.107.down_proj.weight": "model-00006-of-00101.safetensors", + "model.layers.6.mlp.experts.108.gate_proj.weight": "model-00006-of-00101.safetensors", + "model.layers.6.mlp.experts.108.up_proj.weight": "model-00006-of-00101.safetensors", + "model.layers.6.mlp.experts.108.down_proj.weight": "model-00006-of-00101.safetensors", + "model.layers.6.mlp.experts.109.gate_proj.weight": "model-00006-of-00101.safetensors", + "model.layers.6.mlp.experts.109.up_proj.weight": "model-00006-of-00101.safetensors", + "model.layers.6.mlp.experts.109.down_proj.weight": "model-00006-of-00101.safetensors", + "model.layers.6.mlp.experts.110.gate_proj.weight": "model-00006-of-00101.safetensors", + "model.layers.6.mlp.experts.110.up_proj.weight": "model-00006-of-00101.safetensors", + "model.layers.6.mlp.experts.110.down_proj.weight": "model-00006-of-00101.safetensors", + "model.layers.6.mlp.experts.111.gate_proj.weight": "model-00006-of-00101.safetensors", + "model.layers.6.mlp.experts.111.up_proj.weight": "model-00006-of-00101.safetensors", + "model.layers.6.mlp.experts.111.down_proj.weight": "model-00006-of-00101.safetensors", + "model.layers.6.mlp.experts.112.gate_proj.weight": "model-00006-of-00101.safetensors", + "model.layers.6.mlp.experts.112.up_proj.weight": "model-00006-of-00101.safetensors", + "model.layers.6.mlp.experts.112.down_proj.weight": "model-00006-of-00101.safetensors", + "model.layers.6.mlp.experts.113.gate_proj.weight": "model-00006-of-00101.safetensors", + "model.layers.6.mlp.experts.113.up_proj.weight": "model-00006-of-00101.safetensors", + "model.layers.6.mlp.experts.113.down_proj.weight": "model-00006-of-00101.safetensors", + "model.layers.6.mlp.experts.114.gate_proj.weight": "model-00006-of-00101.safetensors", + "model.layers.6.mlp.experts.114.up_proj.weight": "model-00006-of-00101.safetensors", + "model.layers.6.mlp.experts.114.down_proj.weight": "model-00006-of-00101.safetensors", + "model.layers.6.mlp.experts.115.gate_proj.weight": "model-00006-of-00101.safetensors", + "model.layers.6.mlp.experts.115.up_proj.weight": "model-00006-of-00101.safetensors", + "model.layers.6.mlp.experts.115.down_proj.weight": "model-00006-of-00101.safetensors", + "model.layers.6.mlp.experts.116.gate_proj.weight": "model-00006-of-00101.safetensors", + "model.layers.6.mlp.experts.116.up_proj.weight": "model-00006-of-00101.safetensors", + "model.layers.6.mlp.experts.116.down_proj.weight": "model-00006-of-00101.safetensors", + "model.layers.6.mlp.experts.117.gate_proj.weight": "model-00006-of-00101.safetensors", + "model.layers.6.mlp.experts.117.up_proj.weight": "model-00006-of-00101.safetensors", + "model.layers.6.mlp.experts.117.down_proj.weight": "model-00006-of-00101.safetensors", + "model.layers.6.mlp.experts.118.gate_proj.weight": "model-00006-of-00101.safetensors", + "model.layers.6.mlp.experts.118.up_proj.weight": "model-00006-of-00101.safetensors", + "model.layers.6.mlp.experts.118.down_proj.weight": "model-00006-of-00101.safetensors", + "model.layers.6.mlp.experts.119.gate_proj.weight": "model-00006-of-00101.safetensors", + "model.layers.6.mlp.experts.119.up_proj.weight": "model-00006-of-00101.safetensors", + "model.layers.6.mlp.experts.119.down_proj.weight": "model-00006-of-00101.safetensors", + "model.layers.6.mlp.gate.weight": "model-00006-of-00101.safetensors", + "model.layers.6.mlp.gate.e_score_correction_bias": "model-00006-of-00101.safetensors", + "model.layers.6.mlp.shared_experts.gate_proj.weight": "model-00006-of-00101.safetensors", + "model.layers.6.mlp.shared_experts.up_proj.weight": "model-00006-of-00101.safetensors", + "model.layers.6.mlp.shared_experts.down_proj.weight": "model-00006-of-00101.safetensors", + "model.layers.6.input_layernorm.weight": "model-00006-of-00101.safetensors", + "model.layers.6.post_attention_layernorm.weight": "model-00006-of-00101.safetensors", + "model.layers.7.self_attn.q_proj.weight": "model-00006-of-00101.safetensors", + "model.layers.7.self_attn.q_proj.bias": "model-00006-of-00101.safetensors", + "model.layers.7.self_attn.k_proj.weight": "model-00006-of-00101.safetensors", + "model.layers.7.self_attn.k_proj.bias": "model-00006-of-00101.safetensors", + "model.layers.7.self_attn.v_proj.weight": "model-00006-of-00101.safetensors", + "model.layers.7.self_attn.v_proj.bias": "model-00006-of-00101.safetensors", + "model.layers.7.self_attn.o_proj.weight": "model-00006-of-00101.safetensors", + "model.layers.7.self_attn.q_norm.weight": "model-00006-of-00101.safetensors", + "model.layers.7.self_attn.k_norm.weight": "model-00006-of-00101.safetensors", + "model.layers.7.mlp.experts.0.gate_proj.weight": "model-00006-of-00101.safetensors", + "model.layers.7.mlp.experts.0.up_proj.weight": "model-00006-of-00101.safetensors", + "model.layers.7.mlp.experts.0.down_proj.weight": "model-00006-of-00101.safetensors", + "model.layers.7.mlp.experts.1.gate_proj.weight": "model-00006-of-00101.safetensors", + "model.layers.7.mlp.experts.1.up_proj.weight": "model-00006-of-00101.safetensors", + "model.layers.7.mlp.experts.1.down_proj.weight": "model-00006-of-00101.safetensors", + "model.layers.7.mlp.experts.2.gate_proj.weight": "model-00006-of-00101.safetensors", + "model.layers.7.mlp.experts.2.up_proj.weight": "model-00006-of-00101.safetensors", + "model.layers.7.mlp.experts.2.down_proj.weight": "model-00006-of-00101.safetensors", + "model.layers.7.mlp.experts.3.gate_proj.weight": "model-00006-of-00101.safetensors", + "model.layers.7.mlp.experts.3.up_proj.weight": "model-00006-of-00101.safetensors", + "model.layers.7.mlp.experts.3.down_proj.weight": "model-00006-of-00101.safetensors", + "model.layers.7.mlp.experts.4.gate_proj.weight": "model-00006-of-00101.safetensors", + "model.layers.7.mlp.experts.4.up_proj.weight": "model-00006-of-00101.safetensors", + "model.layers.7.mlp.experts.4.down_proj.weight": "model-00006-of-00101.safetensors", + "model.layers.7.mlp.experts.5.gate_proj.weight": "model-00006-of-00101.safetensors", + "model.layers.7.mlp.experts.5.up_proj.weight": "model-00006-of-00101.safetensors", + "model.layers.7.mlp.experts.5.down_proj.weight": "model-00006-of-00101.safetensors", + "model.layers.7.mlp.experts.6.gate_proj.weight": "model-00006-of-00101.safetensors", + "model.layers.7.mlp.experts.6.up_proj.weight": "model-00006-of-00101.safetensors", + "model.layers.7.mlp.experts.6.down_proj.weight": "model-00006-of-00101.safetensors", + "model.layers.7.mlp.experts.7.gate_proj.weight": "model-00006-of-00101.safetensors", + "model.layers.7.mlp.experts.7.up_proj.weight": "model-00006-of-00101.safetensors", + "model.layers.7.mlp.experts.7.down_proj.weight": "model-00006-of-00101.safetensors", + "model.layers.7.mlp.experts.8.gate_proj.weight": "model-00006-of-00101.safetensors", + "model.layers.7.mlp.experts.8.up_proj.weight": "model-00006-of-00101.safetensors", + "model.layers.7.mlp.experts.8.down_proj.weight": "model-00006-of-00101.safetensors", + "model.layers.7.mlp.experts.9.gate_proj.weight": "model-00006-of-00101.safetensors", + "model.layers.7.mlp.experts.9.up_proj.weight": "model-00006-of-00101.safetensors", + "model.layers.7.mlp.experts.9.down_proj.weight": "model-00006-of-00101.safetensors", + "model.layers.7.mlp.experts.10.gate_proj.weight": "model-00006-of-00101.safetensors", + "model.layers.7.mlp.experts.10.up_proj.weight": "model-00006-of-00101.safetensors", + "model.layers.7.mlp.experts.10.down_proj.weight": "model-00006-of-00101.safetensors", + "model.layers.7.mlp.experts.11.gate_proj.weight": "model-00006-of-00101.safetensors", + "model.layers.7.mlp.experts.11.up_proj.weight": "model-00006-of-00101.safetensors", + "model.layers.7.mlp.experts.11.down_proj.weight": "model-00006-of-00101.safetensors", + "model.layers.7.mlp.experts.12.gate_proj.weight": "model-00006-of-00101.safetensors", + "model.layers.7.mlp.experts.12.up_proj.weight": "model-00006-of-00101.safetensors", + "model.layers.7.mlp.experts.12.down_proj.weight": "model-00006-of-00101.safetensors", + "model.layers.7.mlp.experts.13.gate_proj.weight": "model-00006-of-00101.safetensors", + "model.layers.7.mlp.experts.13.up_proj.weight": "model-00006-of-00101.safetensors", + "model.layers.7.mlp.experts.13.down_proj.weight": "model-00006-of-00101.safetensors", + "model.layers.7.mlp.experts.14.gate_proj.weight": "model-00006-of-00101.safetensors", + "model.layers.7.mlp.experts.14.up_proj.weight": "model-00006-of-00101.safetensors", + "model.layers.7.mlp.experts.14.down_proj.weight": "model-00006-of-00101.safetensors", + "model.layers.7.mlp.experts.15.gate_proj.weight": "model-00006-of-00101.safetensors", + "model.layers.7.mlp.experts.15.up_proj.weight": "model-00006-of-00101.safetensors", + "model.layers.7.mlp.experts.15.down_proj.weight": "model-00006-of-00101.safetensors", + "model.layers.7.mlp.experts.16.gate_proj.weight": "model-00006-of-00101.safetensors", + "model.layers.7.mlp.experts.16.up_proj.weight": "model-00006-of-00101.safetensors", + "model.layers.7.mlp.experts.16.down_proj.weight": "model-00006-of-00101.safetensors", + "model.layers.7.mlp.experts.17.gate_proj.weight": "model-00006-of-00101.safetensors", + "model.layers.7.mlp.experts.17.up_proj.weight": "model-00006-of-00101.safetensors", + "model.layers.7.mlp.experts.17.down_proj.weight": "model-00006-of-00101.safetensors", + "model.layers.7.mlp.experts.18.gate_proj.weight": "model-00006-of-00101.safetensors", + "model.layers.7.mlp.experts.18.up_proj.weight": "model-00006-of-00101.safetensors", + "model.layers.7.mlp.experts.18.down_proj.weight": "model-00006-of-00101.safetensors", + "model.layers.7.mlp.experts.19.gate_proj.weight": "model-00006-of-00101.safetensors", + "model.layers.7.mlp.experts.19.up_proj.weight": "model-00006-of-00101.safetensors", + "model.layers.7.mlp.experts.19.down_proj.weight": "model-00006-of-00101.safetensors", + "model.layers.7.mlp.experts.20.gate_proj.weight": "model-00006-of-00101.safetensors", + "model.layers.7.mlp.experts.20.up_proj.weight": "model-00006-of-00101.safetensors", + "model.layers.7.mlp.experts.20.down_proj.weight": "model-00006-of-00101.safetensors", + "model.layers.7.mlp.experts.21.gate_proj.weight": "model-00006-of-00101.safetensors", + "model.layers.7.mlp.experts.21.up_proj.weight": "model-00006-of-00101.safetensors", + "model.layers.7.mlp.experts.21.down_proj.weight": "model-00006-of-00101.safetensors", + "model.layers.7.mlp.experts.22.gate_proj.weight": "model-00006-of-00101.safetensors", + "model.layers.7.mlp.experts.22.up_proj.weight": "model-00006-of-00101.safetensors", + "model.layers.7.mlp.experts.22.down_proj.weight": "model-00006-of-00101.safetensors", + "model.layers.7.mlp.experts.23.gate_proj.weight": "model-00006-of-00101.safetensors", + "model.layers.7.mlp.experts.23.up_proj.weight": "model-00006-of-00101.safetensors", + "model.layers.7.mlp.experts.23.down_proj.weight": "model-00006-of-00101.safetensors", + "model.layers.7.mlp.experts.24.gate_proj.weight": "model-00006-of-00101.safetensors", + "model.layers.7.mlp.experts.24.up_proj.weight": "model-00006-of-00101.safetensors", + "model.layers.7.mlp.experts.24.down_proj.weight": "model-00006-of-00101.safetensors", + "model.layers.7.mlp.experts.25.gate_proj.weight": "model-00006-of-00101.safetensors", + "model.layers.7.mlp.experts.25.up_proj.weight": "model-00006-of-00101.safetensors", + "model.layers.7.mlp.experts.25.down_proj.weight": "model-00006-of-00101.safetensors", + "model.layers.7.mlp.experts.26.gate_proj.weight": "model-00006-of-00101.safetensors", + "model.layers.7.mlp.experts.26.up_proj.weight": "model-00006-of-00101.safetensors", + "model.layers.7.mlp.experts.26.down_proj.weight": "model-00006-of-00101.safetensors", + "model.layers.7.mlp.experts.27.gate_proj.weight": "model-00006-of-00101.safetensors", + "model.layers.7.mlp.experts.27.up_proj.weight": "model-00006-of-00101.safetensors", + "model.layers.7.mlp.experts.27.down_proj.weight": "model-00006-of-00101.safetensors", + "model.layers.7.mlp.experts.28.gate_proj.weight": "model-00006-of-00101.safetensors", + "model.layers.7.mlp.experts.28.up_proj.weight": "model-00006-of-00101.safetensors", + "model.layers.7.mlp.experts.28.down_proj.weight": "model-00006-of-00101.safetensors", + "model.layers.7.mlp.experts.29.gate_proj.weight": "model-00006-of-00101.safetensors", + "model.layers.7.mlp.experts.29.up_proj.weight": "model-00006-of-00101.safetensors", + "model.layers.7.mlp.experts.29.down_proj.weight": "model-00006-of-00101.safetensors", + "model.layers.7.mlp.experts.30.gate_proj.weight": "model-00006-of-00101.safetensors", + "model.layers.7.mlp.experts.30.up_proj.weight": "model-00006-of-00101.safetensors", + "model.layers.7.mlp.experts.30.down_proj.weight": "model-00006-of-00101.safetensors", + "model.layers.7.mlp.experts.31.gate_proj.weight": "model-00006-of-00101.safetensors", + "model.layers.7.mlp.experts.31.up_proj.weight": "model-00006-of-00101.safetensors", + "model.layers.7.mlp.experts.31.down_proj.weight": "model-00006-of-00101.safetensors", + "model.layers.7.mlp.experts.32.gate_proj.weight": "model-00006-of-00101.safetensors", + "model.layers.7.mlp.experts.32.up_proj.weight": "model-00006-of-00101.safetensors", + "model.layers.7.mlp.experts.32.down_proj.weight": "model-00006-of-00101.safetensors", + "model.layers.7.mlp.experts.33.gate_proj.weight": "model-00006-of-00101.safetensors", + "model.layers.7.mlp.experts.33.up_proj.weight": "model-00006-of-00101.safetensors", + "model.layers.7.mlp.experts.33.down_proj.weight": "model-00006-of-00101.safetensors", + "model.layers.7.mlp.experts.34.gate_proj.weight": "model-00006-of-00101.safetensors", + "model.layers.7.mlp.experts.34.up_proj.weight": "model-00006-of-00101.safetensors", + "model.layers.7.mlp.experts.34.down_proj.weight": "model-00006-of-00101.safetensors", + "model.layers.7.mlp.experts.35.gate_proj.weight": "model-00006-of-00101.safetensors", + "model.layers.7.mlp.experts.35.up_proj.weight": "model-00006-of-00101.safetensors", + "model.layers.7.mlp.experts.35.down_proj.weight": "model-00006-of-00101.safetensors", + "model.layers.7.mlp.experts.36.gate_proj.weight": "model-00006-of-00101.safetensors", + "model.layers.7.mlp.experts.36.up_proj.weight": "model-00006-of-00101.safetensors", + "model.layers.7.mlp.experts.36.down_proj.weight": "model-00006-of-00101.safetensors", + "model.layers.7.mlp.experts.37.gate_proj.weight": "model-00006-of-00101.safetensors", + "model.layers.7.mlp.experts.37.up_proj.weight": "model-00006-of-00101.safetensors", + "model.layers.7.mlp.experts.37.down_proj.weight": "model-00006-of-00101.safetensors", + "model.layers.7.mlp.experts.38.gate_proj.weight": "model-00006-of-00101.safetensors", + "model.layers.7.mlp.experts.38.up_proj.weight": "model-00006-of-00101.safetensors", + "model.layers.7.mlp.experts.38.down_proj.weight": "model-00006-of-00101.safetensors", + "model.layers.7.mlp.experts.39.gate_proj.weight": "model-00006-of-00101.safetensors", + "model.layers.7.mlp.experts.39.up_proj.weight": "model-00006-of-00101.safetensors", + "model.layers.7.mlp.experts.39.down_proj.weight": "model-00006-of-00101.safetensors", + "model.layers.7.mlp.experts.40.gate_proj.weight": "model-00006-of-00101.safetensors", + "model.layers.7.mlp.experts.40.up_proj.weight": "model-00006-of-00101.safetensors", + "model.layers.7.mlp.experts.40.down_proj.weight": "model-00006-of-00101.safetensors", + "model.layers.7.mlp.experts.41.gate_proj.weight": "model-00006-of-00101.safetensors", + "model.layers.7.mlp.experts.41.up_proj.weight": "model-00006-of-00101.safetensors", + "model.layers.7.mlp.experts.41.down_proj.weight": "model-00006-of-00101.safetensors", + "model.layers.7.mlp.experts.42.gate_proj.weight": "model-00006-of-00101.safetensors", + "model.layers.7.mlp.experts.42.up_proj.weight": "model-00006-of-00101.safetensors", + "model.layers.7.mlp.experts.42.down_proj.weight": "model-00006-of-00101.safetensors", + "model.layers.7.mlp.experts.43.gate_proj.weight": "model-00006-of-00101.safetensors", + "model.layers.7.mlp.experts.43.up_proj.weight": "model-00006-of-00101.safetensors", + "model.layers.7.mlp.experts.43.down_proj.weight": "model-00006-of-00101.safetensors", + "model.layers.7.mlp.experts.44.gate_proj.weight": "model-00006-of-00101.safetensors", + "model.layers.7.mlp.experts.44.up_proj.weight": "model-00006-of-00101.safetensors", + "model.layers.7.mlp.experts.44.down_proj.weight": "model-00006-of-00101.safetensors", + "model.layers.7.mlp.experts.45.gate_proj.weight": "model-00006-of-00101.safetensors", + "model.layers.7.mlp.experts.45.up_proj.weight": "model-00006-of-00101.safetensors", + "model.layers.7.mlp.experts.45.down_proj.weight": "model-00006-of-00101.safetensors", + "model.layers.7.mlp.experts.46.gate_proj.weight": "model-00006-of-00101.safetensors", + "model.layers.7.mlp.experts.46.up_proj.weight": "model-00006-of-00101.safetensors", + "model.layers.7.mlp.experts.46.down_proj.weight": "model-00006-of-00101.safetensors", + "model.layers.7.mlp.experts.47.gate_proj.weight": "model-00006-of-00101.safetensors", + "model.layers.7.mlp.experts.47.up_proj.weight": "model-00006-of-00101.safetensors", + "model.layers.7.mlp.experts.47.down_proj.weight": "model-00006-of-00101.safetensors", + "model.layers.7.mlp.experts.48.gate_proj.weight": "model-00006-of-00101.safetensors", + "model.layers.7.mlp.experts.48.up_proj.weight": "model-00006-of-00101.safetensors", + "model.layers.7.mlp.experts.48.down_proj.weight": "model-00006-of-00101.safetensors", + "model.layers.7.mlp.experts.49.gate_proj.weight": "model-00006-of-00101.safetensors", + "model.layers.7.mlp.experts.49.up_proj.weight": "model-00006-of-00101.safetensors", + "model.layers.7.mlp.experts.49.down_proj.weight": "model-00006-of-00101.safetensors", + "model.layers.7.mlp.experts.50.gate_proj.weight": "model-00006-of-00101.safetensors", + "model.layers.7.mlp.experts.50.up_proj.weight": "model-00006-of-00101.safetensors", + "model.layers.7.mlp.experts.50.down_proj.weight": "model-00006-of-00101.safetensors", + "model.layers.7.mlp.experts.51.gate_proj.weight": "model-00006-of-00101.safetensors", + "model.layers.7.mlp.experts.51.up_proj.weight": "model-00006-of-00101.safetensors", + "model.layers.7.mlp.experts.51.down_proj.weight": "model-00006-of-00101.safetensors", + "model.layers.7.mlp.experts.52.gate_proj.weight": "model-00006-of-00101.safetensors", + "model.layers.7.mlp.experts.52.up_proj.weight": "model-00006-of-00101.safetensors", + "model.layers.7.mlp.experts.52.down_proj.weight": "model-00006-of-00101.safetensors", + "model.layers.7.mlp.experts.53.gate_proj.weight": "model-00006-of-00101.safetensors", + "model.layers.7.mlp.experts.53.up_proj.weight": "model-00006-of-00101.safetensors", + "model.layers.7.mlp.experts.53.down_proj.weight": "model-00006-of-00101.safetensors", + "model.layers.7.mlp.experts.54.gate_proj.weight": "model-00006-of-00101.safetensors", + "model.layers.7.mlp.experts.54.up_proj.weight": "model-00006-of-00101.safetensors", + "model.layers.7.mlp.experts.54.down_proj.weight": "model-00006-of-00101.safetensors", + "model.layers.7.mlp.experts.55.gate_proj.weight": "model-00006-of-00101.safetensors", + "model.layers.7.mlp.experts.55.up_proj.weight": "model-00006-of-00101.safetensors", + "model.layers.7.mlp.experts.55.down_proj.weight": "model-00006-of-00101.safetensors", + "model.layers.7.mlp.experts.56.gate_proj.weight": "model-00006-of-00101.safetensors", + "model.layers.7.mlp.experts.56.up_proj.weight": "model-00006-of-00101.safetensors", + "model.layers.7.mlp.experts.56.down_proj.weight": "model-00006-of-00101.safetensors", + "model.layers.7.mlp.experts.57.gate_proj.weight": "model-00006-of-00101.safetensors", + "model.layers.7.mlp.experts.57.up_proj.weight": "model-00006-of-00101.safetensors", + "model.layers.7.mlp.experts.57.down_proj.weight": "model-00006-of-00101.safetensors", + "model.layers.7.mlp.experts.58.gate_proj.weight": "model-00006-of-00101.safetensors", + "model.layers.7.mlp.experts.58.up_proj.weight": "model-00006-of-00101.safetensors", + "model.layers.7.mlp.experts.58.down_proj.weight": "model-00006-of-00101.safetensors", + "model.layers.7.mlp.experts.59.gate_proj.weight": "model-00006-of-00101.safetensors", + "model.layers.7.mlp.experts.59.up_proj.weight": "model-00006-of-00101.safetensors", + "model.layers.7.mlp.experts.59.down_proj.weight": "model-00006-of-00101.safetensors", + "model.layers.7.mlp.experts.60.gate_proj.weight": "model-00006-of-00101.safetensors", + "model.layers.7.mlp.experts.60.up_proj.weight": "model-00006-of-00101.safetensors", + "model.layers.7.mlp.experts.60.down_proj.weight": "model-00006-of-00101.safetensors", + "model.layers.7.mlp.experts.61.gate_proj.weight": "model-00006-of-00101.safetensors", + "model.layers.7.mlp.experts.61.up_proj.weight": "model-00006-of-00101.safetensors", + "model.layers.7.mlp.experts.61.down_proj.weight": "model-00006-of-00101.safetensors", + "model.layers.7.mlp.experts.62.gate_proj.weight": "model-00006-of-00101.safetensors", + "model.layers.7.mlp.experts.62.up_proj.weight": "model-00006-of-00101.safetensors", + "model.layers.7.mlp.experts.62.down_proj.weight": "model-00006-of-00101.safetensors", + "model.layers.7.mlp.experts.63.gate_proj.weight": "model-00006-of-00101.safetensors", + "model.layers.7.mlp.experts.63.up_proj.weight": "model-00006-of-00101.safetensors", + "model.layers.7.mlp.experts.63.down_proj.weight": "model-00006-of-00101.safetensors", + "model.layers.7.mlp.experts.64.gate_proj.weight": "model-00006-of-00101.safetensors", + "model.layers.7.mlp.experts.64.up_proj.weight": "model-00006-of-00101.safetensors", + "model.layers.7.mlp.experts.64.down_proj.weight": "model-00006-of-00101.safetensors", + "model.layers.7.mlp.experts.65.gate_proj.weight": "model-00006-of-00101.safetensors", + "model.layers.7.mlp.experts.65.up_proj.weight": "model-00006-of-00101.safetensors", + "model.layers.7.mlp.experts.65.down_proj.weight": "model-00006-of-00101.safetensors", + "model.layers.7.mlp.experts.66.gate_proj.weight": "model-00006-of-00101.safetensors", + "model.layers.7.mlp.experts.66.up_proj.weight": "model-00006-of-00101.safetensors", + "model.layers.7.mlp.experts.66.down_proj.weight": "model-00006-of-00101.safetensors", + "model.layers.7.mlp.experts.67.gate_proj.weight": "model-00006-of-00101.safetensors", + "model.layers.7.mlp.experts.67.up_proj.weight": "model-00006-of-00101.safetensors", + "model.layers.7.mlp.experts.67.down_proj.weight": "model-00006-of-00101.safetensors", + "model.layers.7.mlp.experts.68.gate_proj.weight": "model-00006-of-00101.safetensors", + "model.layers.7.mlp.experts.68.up_proj.weight": "model-00006-of-00101.safetensors", + "model.layers.7.mlp.experts.68.down_proj.weight": "model-00006-of-00101.safetensors", + "model.layers.7.mlp.experts.69.gate_proj.weight": "model-00006-of-00101.safetensors", + "model.layers.7.mlp.experts.69.up_proj.weight": "model-00006-of-00101.safetensors", + "model.layers.7.mlp.experts.69.down_proj.weight": "model-00006-of-00101.safetensors", + "model.layers.7.mlp.experts.70.gate_proj.weight": "model-00006-of-00101.safetensors", + "model.layers.7.mlp.experts.70.up_proj.weight": "model-00006-of-00101.safetensors", + "model.layers.7.mlp.experts.70.down_proj.weight": "model-00006-of-00101.safetensors", + "model.layers.7.mlp.experts.71.gate_proj.weight": "model-00006-of-00101.safetensors", + "model.layers.7.mlp.experts.71.up_proj.weight": "model-00006-of-00101.safetensors", + "model.layers.7.mlp.experts.71.down_proj.weight": "model-00006-of-00101.safetensors", + "model.layers.7.mlp.experts.72.gate_proj.weight": "model-00006-of-00101.safetensors", + "model.layers.7.mlp.experts.72.up_proj.weight": "model-00006-of-00101.safetensors", + "model.layers.7.mlp.experts.72.down_proj.weight": "model-00006-of-00101.safetensors", + "model.layers.7.mlp.experts.73.gate_proj.weight": "model-00006-of-00101.safetensors", + "model.layers.7.mlp.experts.73.up_proj.weight": "model-00006-of-00101.safetensors", + "model.layers.7.mlp.experts.73.down_proj.weight": "model-00006-of-00101.safetensors", + "model.layers.7.mlp.experts.74.gate_proj.weight": "model-00006-of-00101.safetensors", + "model.layers.7.mlp.experts.74.up_proj.weight": "model-00006-of-00101.safetensors", + "model.layers.7.mlp.experts.74.down_proj.weight": "model-00006-of-00101.safetensors", + "model.layers.7.mlp.experts.75.gate_proj.weight": "model-00006-of-00101.safetensors", + "model.layers.7.mlp.experts.75.up_proj.weight": "model-00006-of-00101.safetensors", + "model.layers.7.mlp.experts.75.down_proj.weight": "model-00006-of-00101.safetensors", + "model.layers.7.mlp.experts.76.gate_proj.weight": "model-00006-of-00101.safetensors", + "model.layers.7.mlp.experts.76.up_proj.weight": "model-00006-of-00101.safetensors", + "model.layers.7.mlp.experts.76.down_proj.weight": "model-00006-of-00101.safetensors", + "model.layers.7.mlp.experts.77.gate_proj.weight": "model-00006-of-00101.safetensors", + "model.layers.7.mlp.experts.77.up_proj.weight": "model-00006-of-00101.safetensors", + "model.layers.7.mlp.experts.77.down_proj.weight": "model-00006-of-00101.safetensors", + "model.layers.7.mlp.experts.78.gate_proj.weight": "model-00006-of-00101.safetensors", + "model.layers.7.mlp.experts.78.up_proj.weight": "model-00006-of-00101.safetensors", + "model.layers.7.mlp.experts.78.down_proj.weight": "model-00006-of-00101.safetensors", + "model.layers.7.mlp.experts.79.gate_proj.weight": "model-00006-of-00101.safetensors", + "model.layers.7.mlp.experts.79.up_proj.weight": "model-00006-of-00101.safetensors", + "model.layers.7.mlp.experts.79.down_proj.weight": "model-00006-of-00101.safetensors", + "model.layers.7.mlp.experts.80.gate_proj.weight": "model-00006-of-00101.safetensors", + "model.layers.7.mlp.experts.80.up_proj.weight": "model-00006-of-00101.safetensors", + "model.layers.7.mlp.experts.80.down_proj.weight": "model-00006-of-00101.safetensors", + "model.layers.7.mlp.experts.81.gate_proj.weight": "model-00006-of-00101.safetensors", + "model.layers.7.mlp.experts.81.up_proj.weight": "model-00006-of-00101.safetensors", + "model.layers.7.mlp.experts.81.down_proj.weight": "model-00006-of-00101.safetensors", + "model.layers.7.mlp.experts.82.gate_proj.weight": "model-00006-of-00101.safetensors", + "model.layers.7.mlp.experts.82.up_proj.weight": "model-00006-of-00101.safetensors", + "model.layers.7.mlp.experts.82.down_proj.weight": "model-00006-of-00101.safetensors", + "model.layers.7.mlp.experts.83.gate_proj.weight": "model-00006-of-00101.safetensors", + "model.layers.7.mlp.experts.83.up_proj.weight": "model-00006-of-00101.safetensors", + "model.layers.7.mlp.experts.83.down_proj.weight": "model-00006-of-00101.safetensors", + "model.layers.7.mlp.experts.84.gate_proj.weight": "model-00006-of-00101.safetensors", + "model.layers.7.mlp.experts.84.up_proj.weight": "model-00006-of-00101.safetensors", + "model.layers.7.mlp.experts.84.down_proj.weight": "model-00006-of-00101.safetensors", + "model.layers.7.mlp.experts.85.gate_proj.weight": "model-00006-of-00101.safetensors", + "model.layers.7.mlp.experts.85.up_proj.weight": "model-00006-of-00101.safetensors", + "model.layers.7.mlp.experts.85.down_proj.weight": "model-00006-of-00101.safetensors", + "model.layers.7.mlp.experts.86.gate_proj.weight": "model-00006-of-00101.safetensors", + "model.layers.7.mlp.experts.86.up_proj.weight": "model-00006-of-00101.safetensors", + "model.layers.7.mlp.experts.86.down_proj.weight": "model-00006-of-00101.safetensors", + "model.layers.7.mlp.experts.87.gate_proj.weight": "model-00006-of-00101.safetensors", + "model.layers.7.mlp.experts.87.up_proj.weight": "model-00006-of-00101.safetensors", + "model.layers.7.mlp.experts.87.down_proj.weight": "model-00006-of-00101.safetensors", + "model.layers.7.mlp.experts.88.gate_proj.weight": "model-00006-of-00101.safetensors", + "model.layers.7.mlp.experts.88.up_proj.weight": "model-00006-of-00101.safetensors", + "model.layers.7.mlp.experts.88.down_proj.weight": "model-00006-of-00101.safetensors", + "model.layers.7.mlp.experts.89.gate_proj.weight": "model-00006-of-00101.safetensors", + "model.layers.7.mlp.experts.89.up_proj.weight": "model-00006-of-00101.safetensors", + "model.layers.7.mlp.experts.89.down_proj.weight": "model-00006-of-00101.safetensors", + "model.layers.7.mlp.experts.90.gate_proj.weight": "model-00006-of-00101.safetensors", + "model.layers.7.mlp.experts.90.up_proj.weight": "model-00006-of-00101.safetensors", + "model.layers.7.mlp.experts.90.down_proj.weight": "model-00006-of-00101.safetensors", + "model.layers.7.mlp.experts.91.gate_proj.weight": "model-00006-of-00101.safetensors", + "model.layers.7.mlp.experts.91.up_proj.weight": "model-00006-of-00101.safetensors", + "model.layers.7.mlp.experts.91.down_proj.weight": "model-00006-of-00101.safetensors", + "model.layers.7.mlp.experts.92.gate_proj.weight": "model-00006-of-00101.safetensors", + "model.layers.7.mlp.experts.92.up_proj.weight": "model-00006-of-00101.safetensors", + "model.layers.7.mlp.experts.92.down_proj.weight": "model-00006-of-00101.safetensors", + "model.layers.7.mlp.experts.93.gate_proj.weight": "model-00006-of-00101.safetensors", + "model.layers.7.mlp.experts.93.up_proj.weight": "model-00006-of-00101.safetensors", + "model.layers.7.mlp.experts.93.down_proj.weight": "model-00006-of-00101.safetensors", + "model.layers.7.mlp.experts.94.gate_proj.weight": "model-00007-of-00101.safetensors", + "model.layers.7.mlp.experts.94.up_proj.weight": "model-00007-of-00101.safetensors", + "model.layers.7.mlp.experts.94.down_proj.weight": "model-00007-of-00101.safetensors", + "model.layers.7.mlp.experts.95.gate_proj.weight": "model-00007-of-00101.safetensors", + "model.layers.7.mlp.experts.95.up_proj.weight": "model-00007-of-00101.safetensors", + "model.layers.7.mlp.experts.95.down_proj.weight": "model-00007-of-00101.safetensors", + "model.layers.7.mlp.experts.96.gate_proj.weight": "model-00007-of-00101.safetensors", + "model.layers.7.mlp.experts.96.up_proj.weight": "model-00007-of-00101.safetensors", + "model.layers.7.mlp.experts.96.down_proj.weight": "model-00007-of-00101.safetensors", + "model.layers.7.mlp.experts.97.gate_proj.weight": "model-00007-of-00101.safetensors", + "model.layers.7.mlp.experts.97.up_proj.weight": "model-00007-of-00101.safetensors", + "model.layers.7.mlp.experts.97.down_proj.weight": "model-00007-of-00101.safetensors", + "model.layers.7.mlp.experts.98.gate_proj.weight": "model-00007-of-00101.safetensors", + "model.layers.7.mlp.experts.98.up_proj.weight": "model-00007-of-00101.safetensors", + "model.layers.7.mlp.experts.98.down_proj.weight": "model-00007-of-00101.safetensors", + "model.layers.7.mlp.experts.99.gate_proj.weight": "model-00007-of-00101.safetensors", + "model.layers.7.mlp.experts.99.up_proj.weight": "model-00007-of-00101.safetensors", + "model.layers.7.mlp.experts.99.down_proj.weight": "model-00007-of-00101.safetensors", + "model.layers.7.mlp.experts.100.gate_proj.weight": "model-00007-of-00101.safetensors", + "model.layers.7.mlp.experts.100.up_proj.weight": "model-00007-of-00101.safetensors", + "model.layers.7.mlp.experts.100.down_proj.weight": "model-00007-of-00101.safetensors", + "model.layers.7.mlp.experts.101.gate_proj.weight": "model-00007-of-00101.safetensors", + "model.layers.7.mlp.experts.101.up_proj.weight": "model-00007-of-00101.safetensors", + "model.layers.7.mlp.experts.101.down_proj.weight": "model-00007-of-00101.safetensors", + "model.layers.7.mlp.experts.102.gate_proj.weight": "model-00007-of-00101.safetensors", + "model.layers.7.mlp.experts.102.up_proj.weight": "model-00007-of-00101.safetensors", + "model.layers.7.mlp.experts.102.down_proj.weight": "model-00007-of-00101.safetensors", + "model.layers.7.mlp.experts.103.gate_proj.weight": "model-00007-of-00101.safetensors", + "model.layers.7.mlp.experts.103.up_proj.weight": "model-00007-of-00101.safetensors", + "model.layers.7.mlp.experts.103.down_proj.weight": "model-00007-of-00101.safetensors", + "model.layers.7.mlp.experts.104.gate_proj.weight": "model-00007-of-00101.safetensors", + "model.layers.7.mlp.experts.104.up_proj.weight": "model-00007-of-00101.safetensors", + "model.layers.7.mlp.experts.104.down_proj.weight": "model-00007-of-00101.safetensors", + "model.layers.7.mlp.experts.105.gate_proj.weight": "model-00007-of-00101.safetensors", + "model.layers.7.mlp.experts.105.up_proj.weight": "model-00007-of-00101.safetensors", + "model.layers.7.mlp.experts.105.down_proj.weight": "model-00007-of-00101.safetensors", + "model.layers.7.mlp.experts.106.gate_proj.weight": "model-00007-of-00101.safetensors", + "model.layers.7.mlp.experts.106.up_proj.weight": "model-00007-of-00101.safetensors", + "model.layers.7.mlp.experts.106.down_proj.weight": "model-00007-of-00101.safetensors", + "model.layers.7.mlp.experts.107.gate_proj.weight": "model-00007-of-00101.safetensors", + "model.layers.7.mlp.experts.107.up_proj.weight": "model-00007-of-00101.safetensors", + "model.layers.7.mlp.experts.107.down_proj.weight": "model-00007-of-00101.safetensors", + "model.layers.7.mlp.experts.108.gate_proj.weight": "model-00007-of-00101.safetensors", + "model.layers.7.mlp.experts.108.up_proj.weight": "model-00007-of-00101.safetensors", + "model.layers.7.mlp.experts.108.down_proj.weight": "model-00007-of-00101.safetensors", + "model.layers.7.mlp.experts.109.gate_proj.weight": "model-00007-of-00101.safetensors", + "model.layers.7.mlp.experts.109.up_proj.weight": "model-00007-of-00101.safetensors", + "model.layers.7.mlp.experts.109.down_proj.weight": "model-00007-of-00101.safetensors", + "model.layers.7.mlp.experts.110.gate_proj.weight": "model-00007-of-00101.safetensors", + "model.layers.7.mlp.experts.110.up_proj.weight": "model-00007-of-00101.safetensors", + "model.layers.7.mlp.experts.110.down_proj.weight": "model-00007-of-00101.safetensors", + "model.layers.7.mlp.experts.111.gate_proj.weight": "model-00007-of-00101.safetensors", + "model.layers.7.mlp.experts.111.up_proj.weight": "model-00007-of-00101.safetensors", + "model.layers.7.mlp.experts.111.down_proj.weight": "model-00007-of-00101.safetensors", + "model.layers.7.mlp.experts.112.gate_proj.weight": "model-00007-of-00101.safetensors", + "model.layers.7.mlp.experts.112.up_proj.weight": "model-00007-of-00101.safetensors", + "model.layers.7.mlp.experts.112.down_proj.weight": "model-00007-of-00101.safetensors", + "model.layers.7.mlp.experts.113.gate_proj.weight": "model-00007-of-00101.safetensors", + "model.layers.7.mlp.experts.113.up_proj.weight": "model-00007-of-00101.safetensors", + "model.layers.7.mlp.experts.113.down_proj.weight": "model-00007-of-00101.safetensors", + "model.layers.7.mlp.experts.114.gate_proj.weight": "model-00007-of-00101.safetensors", + "model.layers.7.mlp.experts.114.up_proj.weight": "model-00007-of-00101.safetensors", + "model.layers.7.mlp.experts.114.down_proj.weight": "model-00007-of-00101.safetensors", + "model.layers.7.mlp.experts.115.gate_proj.weight": "model-00007-of-00101.safetensors", + "model.layers.7.mlp.experts.115.up_proj.weight": "model-00007-of-00101.safetensors", + "model.layers.7.mlp.experts.115.down_proj.weight": "model-00007-of-00101.safetensors", + "model.layers.7.mlp.experts.116.gate_proj.weight": "model-00007-of-00101.safetensors", + "model.layers.7.mlp.experts.116.up_proj.weight": "model-00007-of-00101.safetensors", + "model.layers.7.mlp.experts.116.down_proj.weight": "model-00007-of-00101.safetensors", + "model.layers.7.mlp.experts.117.gate_proj.weight": "model-00007-of-00101.safetensors", + "model.layers.7.mlp.experts.117.up_proj.weight": "model-00007-of-00101.safetensors", + "model.layers.7.mlp.experts.117.down_proj.weight": "model-00007-of-00101.safetensors", + "model.layers.7.mlp.experts.118.gate_proj.weight": "model-00007-of-00101.safetensors", + "model.layers.7.mlp.experts.118.up_proj.weight": "model-00007-of-00101.safetensors", + "model.layers.7.mlp.experts.118.down_proj.weight": "model-00007-of-00101.safetensors", + "model.layers.7.mlp.experts.119.gate_proj.weight": "model-00007-of-00101.safetensors", + "model.layers.7.mlp.experts.119.up_proj.weight": "model-00007-of-00101.safetensors", + "model.layers.7.mlp.experts.119.down_proj.weight": "model-00007-of-00101.safetensors", + "model.layers.7.mlp.gate.weight": "model-00007-of-00101.safetensors", + "model.layers.7.mlp.gate.e_score_correction_bias": "model-00007-of-00101.safetensors", + "model.layers.7.mlp.shared_experts.gate_proj.weight": "model-00007-of-00101.safetensors", + "model.layers.7.mlp.shared_experts.up_proj.weight": "model-00007-of-00101.safetensors", + "model.layers.7.mlp.shared_experts.down_proj.weight": "model-00007-of-00101.safetensors", + "model.layers.7.input_layernorm.weight": "model-00007-of-00101.safetensors", + "model.layers.7.post_attention_layernorm.weight": "model-00007-of-00101.safetensors", + "model.layers.8.self_attn.q_proj.weight": "model-00007-of-00101.safetensors", + "model.layers.8.self_attn.q_proj.bias": "model-00007-of-00101.safetensors", + "model.layers.8.self_attn.k_proj.weight": "model-00007-of-00101.safetensors", + "model.layers.8.self_attn.k_proj.bias": "model-00007-of-00101.safetensors", + "model.layers.8.self_attn.v_proj.weight": "model-00007-of-00101.safetensors", + "model.layers.8.self_attn.v_proj.bias": "model-00007-of-00101.safetensors", + "model.layers.8.self_attn.o_proj.weight": "model-00007-of-00101.safetensors", + "model.layers.8.self_attn.q_norm.weight": "model-00007-of-00101.safetensors", + "model.layers.8.self_attn.k_norm.weight": "model-00007-of-00101.safetensors", + "model.layers.8.mlp.experts.0.gate_proj.weight": "model-00007-of-00101.safetensors", + "model.layers.8.mlp.experts.0.up_proj.weight": "model-00007-of-00101.safetensors", + "model.layers.8.mlp.experts.0.down_proj.weight": "model-00007-of-00101.safetensors", + "model.layers.8.mlp.experts.1.gate_proj.weight": "model-00007-of-00101.safetensors", + "model.layers.8.mlp.experts.1.up_proj.weight": "model-00007-of-00101.safetensors", + "model.layers.8.mlp.experts.1.down_proj.weight": "model-00007-of-00101.safetensors", + "model.layers.8.mlp.experts.2.gate_proj.weight": "model-00007-of-00101.safetensors", + "model.layers.8.mlp.experts.2.up_proj.weight": "model-00007-of-00101.safetensors", + "model.layers.8.mlp.experts.2.down_proj.weight": "model-00007-of-00101.safetensors", + "model.layers.8.mlp.experts.3.gate_proj.weight": "model-00007-of-00101.safetensors", + "model.layers.8.mlp.experts.3.up_proj.weight": "model-00007-of-00101.safetensors", + "model.layers.8.mlp.experts.3.down_proj.weight": "model-00007-of-00101.safetensors", + "model.layers.8.mlp.experts.4.gate_proj.weight": "model-00007-of-00101.safetensors", + "model.layers.8.mlp.experts.4.up_proj.weight": "model-00007-of-00101.safetensors", + "model.layers.8.mlp.experts.4.down_proj.weight": "model-00007-of-00101.safetensors", + "model.layers.8.mlp.experts.5.gate_proj.weight": "model-00007-of-00101.safetensors", + "model.layers.8.mlp.experts.5.up_proj.weight": "model-00007-of-00101.safetensors", + "model.layers.8.mlp.experts.5.down_proj.weight": "model-00007-of-00101.safetensors", + "model.layers.8.mlp.experts.6.gate_proj.weight": "model-00007-of-00101.safetensors", + "model.layers.8.mlp.experts.6.up_proj.weight": "model-00007-of-00101.safetensors", + "model.layers.8.mlp.experts.6.down_proj.weight": "model-00007-of-00101.safetensors", + "model.layers.8.mlp.experts.7.gate_proj.weight": "model-00007-of-00101.safetensors", + "model.layers.8.mlp.experts.7.up_proj.weight": "model-00007-of-00101.safetensors", + "model.layers.8.mlp.experts.7.down_proj.weight": "model-00007-of-00101.safetensors", + "model.layers.8.mlp.experts.8.gate_proj.weight": "model-00007-of-00101.safetensors", + "model.layers.8.mlp.experts.8.up_proj.weight": "model-00007-of-00101.safetensors", + "model.layers.8.mlp.experts.8.down_proj.weight": "model-00007-of-00101.safetensors", + "model.layers.8.mlp.experts.9.gate_proj.weight": "model-00007-of-00101.safetensors", + "model.layers.8.mlp.experts.9.up_proj.weight": "model-00007-of-00101.safetensors", + "model.layers.8.mlp.experts.9.down_proj.weight": "model-00007-of-00101.safetensors", + "model.layers.8.mlp.experts.10.gate_proj.weight": "model-00007-of-00101.safetensors", + "model.layers.8.mlp.experts.10.up_proj.weight": "model-00007-of-00101.safetensors", + "model.layers.8.mlp.experts.10.down_proj.weight": "model-00007-of-00101.safetensors", + "model.layers.8.mlp.experts.11.gate_proj.weight": "model-00007-of-00101.safetensors", + "model.layers.8.mlp.experts.11.up_proj.weight": "model-00007-of-00101.safetensors", + "model.layers.8.mlp.experts.11.down_proj.weight": "model-00007-of-00101.safetensors", + "model.layers.8.mlp.experts.12.gate_proj.weight": "model-00007-of-00101.safetensors", + "model.layers.8.mlp.experts.12.up_proj.weight": "model-00007-of-00101.safetensors", + "model.layers.8.mlp.experts.12.down_proj.weight": "model-00007-of-00101.safetensors", + "model.layers.8.mlp.experts.13.gate_proj.weight": "model-00007-of-00101.safetensors", + "model.layers.8.mlp.experts.13.up_proj.weight": "model-00007-of-00101.safetensors", + "model.layers.8.mlp.experts.13.down_proj.weight": "model-00007-of-00101.safetensors", + "model.layers.8.mlp.experts.14.gate_proj.weight": "model-00007-of-00101.safetensors", + "model.layers.8.mlp.experts.14.up_proj.weight": "model-00007-of-00101.safetensors", + "model.layers.8.mlp.experts.14.down_proj.weight": "model-00007-of-00101.safetensors", + "model.layers.8.mlp.experts.15.gate_proj.weight": "model-00007-of-00101.safetensors", + "model.layers.8.mlp.experts.15.up_proj.weight": "model-00007-of-00101.safetensors", + "model.layers.8.mlp.experts.15.down_proj.weight": "model-00007-of-00101.safetensors", + "model.layers.8.mlp.experts.16.gate_proj.weight": "model-00007-of-00101.safetensors", + "model.layers.8.mlp.experts.16.up_proj.weight": "model-00007-of-00101.safetensors", + "model.layers.8.mlp.experts.16.down_proj.weight": "model-00007-of-00101.safetensors", + "model.layers.8.mlp.experts.17.gate_proj.weight": "model-00007-of-00101.safetensors", + "model.layers.8.mlp.experts.17.up_proj.weight": "model-00007-of-00101.safetensors", + "model.layers.8.mlp.experts.17.down_proj.weight": "model-00007-of-00101.safetensors", + "model.layers.8.mlp.experts.18.gate_proj.weight": "model-00007-of-00101.safetensors", + "model.layers.8.mlp.experts.18.up_proj.weight": "model-00007-of-00101.safetensors", + "model.layers.8.mlp.experts.18.down_proj.weight": "model-00007-of-00101.safetensors", + "model.layers.8.mlp.experts.19.gate_proj.weight": "model-00007-of-00101.safetensors", + "model.layers.8.mlp.experts.19.up_proj.weight": "model-00007-of-00101.safetensors", + "model.layers.8.mlp.experts.19.down_proj.weight": "model-00007-of-00101.safetensors", + "model.layers.8.mlp.experts.20.gate_proj.weight": "model-00007-of-00101.safetensors", + "model.layers.8.mlp.experts.20.up_proj.weight": "model-00007-of-00101.safetensors", + "model.layers.8.mlp.experts.20.down_proj.weight": "model-00007-of-00101.safetensors", + "model.layers.8.mlp.experts.21.gate_proj.weight": "model-00007-of-00101.safetensors", + "model.layers.8.mlp.experts.21.up_proj.weight": "model-00007-of-00101.safetensors", + "model.layers.8.mlp.experts.21.down_proj.weight": "model-00007-of-00101.safetensors", + "model.layers.8.mlp.experts.22.gate_proj.weight": "model-00007-of-00101.safetensors", + "model.layers.8.mlp.experts.22.up_proj.weight": "model-00007-of-00101.safetensors", + "model.layers.8.mlp.experts.22.down_proj.weight": "model-00007-of-00101.safetensors", + "model.layers.8.mlp.experts.23.gate_proj.weight": "model-00007-of-00101.safetensors", + "model.layers.8.mlp.experts.23.up_proj.weight": "model-00007-of-00101.safetensors", + "model.layers.8.mlp.experts.23.down_proj.weight": "model-00007-of-00101.safetensors", + "model.layers.8.mlp.experts.24.gate_proj.weight": "model-00007-of-00101.safetensors", + "model.layers.8.mlp.experts.24.up_proj.weight": "model-00007-of-00101.safetensors", + "model.layers.8.mlp.experts.24.down_proj.weight": "model-00007-of-00101.safetensors", + "model.layers.8.mlp.experts.25.gate_proj.weight": "model-00007-of-00101.safetensors", + "model.layers.8.mlp.experts.25.up_proj.weight": "model-00007-of-00101.safetensors", + "model.layers.8.mlp.experts.25.down_proj.weight": "model-00007-of-00101.safetensors", + "model.layers.8.mlp.experts.26.gate_proj.weight": "model-00007-of-00101.safetensors", + "model.layers.8.mlp.experts.26.up_proj.weight": "model-00007-of-00101.safetensors", + "model.layers.8.mlp.experts.26.down_proj.weight": "model-00007-of-00101.safetensors", + "model.layers.8.mlp.experts.27.gate_proj.weight": "model-00007-of-00101.safetensors", + "model.layers.8.mlp.experts.27.up_proj.weight": "model-00007-of-00101.safetensors", + "model.layers.8.mlp.experts.27.down_proj.weight": "model-00007-of-00101.safetensors", + "model.layers.8.mlp.experts.28.gate_proj.weight": "model-00007-of-00101.safetensors", + "model.layers.8.mlp.experts.28.up_proj.weight": "model-00007-of-00101.safetensors", + "model.layers.8.mlp.experts.28.down_proj.weight": "model-00007-of-00101.safetensors", + "model.layers.8.mlp.experts.29.gate_proj.weight": "model-00007-of-00101.safetensors", + "model.layers.8.mlp.experts.29.up_proj.weight": "model-00007-of-00101.safetensors", + "model.layers.8.mlp.experts.29.down_proj.weight": "model-00007-of-00101.safetensors", + "model.layers.8.mlp.experts.30.gate_proj.weight": "model-00007-of-00101.safetensors", + "model.layers.8.mlp.experts.30.up_proj.weight": "model-00007-of-00101.safetensors", + "model.layers.8.mlp.experts.30.down_proj.weight": "model-00007-of-00101.safetensors", + "model.layers.8.mlp.experts.31.gate_proj.weight": "model-00007-of-00101.safetensors", + "model.layers.8.mlp.experts.31.up_proj.weight": "model-00007-of-00101.safetensors", + "model.layers.8.mlp.experts.31.down_proj.weight": "model-00007-of-00101.safetensors", + "model.layers.8.mlp.experts.32.gate_proj.weight": "model-00007-of-00101.safetensors", + "model.layers.8.mlp.experts.32.up_proj.weight": "model-00007-of-00101.safetensors", + "model.layers.8.mlp.experts.32.down_proj.weight": "model-00007-of-00101.safetensors", + "model.layers.8.mlp.experts.33.gate_proj.weight": "model-00007-of-00101.safetensors", + "model.layers.8.mlp.experts.33.up_proj.weight": "model-00007-of-00101.safetensors", + "model.layers.8.mlp.experts.33.down_proj.weight": "model-00007-of-00101.safetensors", + "model.layers.8.mlp.experts.34.gate_proj.weight": "model-00007-of-00101.safetensors", + "model.layers.8.mlp.experts.34.up_proj.weight": "model-00007-of-00101.safetensors", + "model.layers.8.mlp.experts.34.down_proj.weight": "model-00007-of-00101.safetensors", + "model.layers.8.mlp.experts.35.gate_proj.weight": "model-00007-of-00101.safetensors", + "model.layers.8.mlp.experts.35.up_proj.weight": "model-00007-of-00101.safetensors", + "model.layers.8.mlp.experts.35.down_proj.weight": "model-00007-of-00101.safetensors", + "model.layers.8.mlp.experts.36.gate_proj.weight": "model-00007-of-00101.safetensors", + "model.layers.8.mlp.experts.36.up_proj.weight": "model-00007-of-00101.safetensors", + "model.layers.8.mlp.experts.36.down_proj.weight": "model-00007-of-00101.safetensors", + "model.layers.8.mlp.experts.37.gate_proj.weight": "model-00007-of-00101.safetensors", + "model.layers.8.mlp.experts.37.up_proj.weight": "model-00007-of-00101.safetensors", + "model.layers.8.mlp.experts.37.down_proj.weight": "model-00007-of-00101.safetensors", + "model.layers.8.mlp.experts.38.gate_proj.weight": "model-00007-of-00101.safetensors", + "model.layers.8.mlp.experts.38.up_proj.weight": "model-00007-of-00101.safetensors", + "model.layers.8.mlp.experts.38.down_proj.weight": "model-00007-of-00101.safetensors", + "model.layers.8.mlp.experts.39.gate_proj.weight": "model-00007-of-00101.safetensors", + "model.layers.8.mlp.experts.39.up_proj.weight": "model-00007-of-00101.safetensors", + "model.layers.8.mlp.experts.39.down_proj.weight": "model-00007-of-00101.safetensors", + "model.layers.8.mlp.experts.40.gate_proj.weight": "model-00007-of-00101.safetensors", + "model.layers.8.mlp.experts.40.up_proj.weight": "model-00007-of-00101.safetensors", + "model.layers.8.mlp.experts.40.down_proj.weight": "model-00007-of-00101.safetensors", + "model.layers.8.mlp.experts.41.gate_proj.weight": "model-00007-of-00101.safetensors", + "model.layers.8.mlp.experts.41.up_proj.weight": "model-00007-of-00101.safetensors", + "model.layers.8.mlp.experts.41.down_proj.weight": "model-00007-of-00101.safetensors", + "model.layers.8.mlp.experts.42.gate_proj.weight": "model-00007-of-00101.safetensors", + "model.layers.8.mlp.experts.42.up_proj.weight": "model-00007-of-00101.safetensors", + "model.layers.8.mlp.experts.42.down_proj.weight": "model-00007-of-00101.safetensors", + "model.layers.8.mlp.experts.43.gate_proj.weight": "model-00007-of-00101.safetensors", + "model.layers.8.mlp.experts.43.up_proj.weight": "model-00007-of-00101.safetensors", + "model.layers.8.mlp.experts.43.down_proj.weight": "model-00007-of-00101.safetensors", + "model.layers.8.mlp.experts.44.gate_proj.weight": "model-00007-of-00101.safetensors", + "model.layers.8.mlp.experts.44.up_proj.weight": "model-00007-of-00101.safetensors", + "model.layers.8.mlp.experts.44.down_proj.weight": "model-00007-of-00101.safetensors", + "model.layers.8.mlp.experts.45.gate_proj.weight": "model-00007-of-00101.safetensors", + "model.layers.8.mlp.experts.45.up_proj.weight": "model-00007-of-00101.safetensors", + "model.layers.8.mlp.experts.45.down_proj.weight": "model-00007-of-00101.safetensors", + "model.layers.8.mlp.experts.46.gate_proj.weight": "model-00007-of-00101.safetensors", + "model.layers.8.mlp.experts.46.up_proj.weight": "model-00007-of-00101.safetensors", + "model.layers.8.mlp.experts.46.down_proj.weight": "model-00007-of-00101.safetensors", + "model.layers.8.mlp.experts.47.gate_proj.weight": "model-00007-of-00101.safetensors", + "model.layers.8.mlp.experts.47.up_proj.weight": "model-00007-of-00101.safetensors", + "model.layers.8.mlp.experts.47.down_proj.weight": "model-00007-of-00101.safetensors", + "model.layers.8.mlp.experts.48.gate_proj.weight": "model-00007-of-00101.safetensors", + "model.layers.8.mlp.experts.48.up_proj.weight": "model-00007-of-00101.safetensors", + "model.layers.8.mlp.experts.48.down_proj.weight": "model-00007-of-00101.safetensors", + "model.layers.8.mlp.experts.49.gate_proj.weight": "model-00007-of-00101.safetensors", + "model.layers.8.mlp.experts.49.up_proj.weight": "model-00007-of-00101.safetensors", + "model.layers.8.mlp.experts.49.down_proj.weight": "model-00007-of-00101.safetensors", + "model.layers.8.mlp.experts.50.gate_proj.weight": "model-00007-of-00101.safetensors", + "model.layers.8.mlp.experts.50.up_proj.weight": "model-00007-of-00101.safetensors", + "model.layers.8.mlp.experts.50.down_proj.weight": "model-00007-of-00101.safetensors", + "model.layers.8.mlp.experts.51.gate_proj.weight": "model-00007-of-00101.safetensors", + "model.layers.8.mlp.experts.51.up_proj.weight": "model-00007-of-00101.safetensors", + "model.layers.8.mlp.experts.51.down_proj.weight": "model-00007-of-00101.safetensors", + "model.layers.8.mlp.experts.52.gate_proj.weight": "model-00007-of-00101.safetensors", + "model.layers.8.mlp.experts.52.up_proj.weight": "model-00007-of-00101.safetensors", + "model.layers.8.mlp.experts.52.down_proj.weight": "model-00007-of-00101.safetensors", + "model.layers.8.mlp.experts.53.gate_proj.weight": "model-00007-of-00101.safetensors", + "model.layers.8.mlp.experts.53.up_proj.weight": "model-00007-of-00101.safetensors", + "model.layers.8.mlp.experts.53.down_proj.weight": "model-00007-of-00101.safetensors", + "model.layers.8.mlp.experts.54.gate_proj.weight": "model-00007-of-00101.safetensors", + "model.layers.8.mlp.experts.54.up_proj.weight": "model-00007-of-00101.safetensors", + "model.layers.8.mlp.experts.54.down_proj.weight": "model-00007-of-00101.safetensors", + "model.layers.8.mlp.experts.55.gate_proj.weight": "model-00007-of-00101.safetensors", + "model.layers.8.mlp.experts.55.up_proj.weight": "model-00007-of-00101.safetensors", + "model.layers.8.mlp.experts.55.down_proj.weight": "model-00007-of-00101.safetensors", + "model.layers.8.mlp.experts.56.gate_proj.weight": "model-00007-of-00101.safetensors", + "model.layers.8.mlp.experts.56.up_proj.weight": "model-00007-of-00101.safetensors", + "model.layers.8.mlp.experts.56.down_proj.weight": "model-00007-of-00101.safetensors", + "model.layers.8.mlp.experts.57.gate_proj.weight": "model-00007-of-00101.safetensors", + "model.layers.8.mlp.experts.57.up_proj.weight": "model-00007-of-00101.safetensors", + "model.layers.8.mlp.experts.57.down_proj.weight": "model-00007-of-00101.safetensors", + "model.layers.8.mlp.experts.58.gate_proj.weight": "model-00007-of-00101.safetensors", + "model.layers.8.mlp.experts.58.up_proj.weight": "model-00007-of-00101.safetensors", + "model.layers.8.mlp.experts.58.down_proj.weight": "model-00007-of-00101.safetensors", + "model.layers.8.mlp.experts.59.gate_proj.weight": "model-00007-of-00101.safetensors", + "model.layers.8.mlp.experts.59.up_proj.weight": "model-00007-of-00101.safetensors", + "model.layers.8.mlp.experts.59.down_proj.weight": "model-00007-of-00101.safetensors", + "model.layers.8.mlp.experts.60.gate_proj.weight": "model-00007-of-00101.safetensors", + "model.layers.8.mlp.experts.60.up_proj.weight": "model-00007-of-00101.safetensors", + "model.layers.8.mlp.experts.60.down_proj.weight": "model-00007-of-00101.safetensors", + "model.layers.8.mlp.experts.61.gate_proj.weight": "model-00007-of-00101.safetensors", + "model.layers.8.mlp.experts.61.up_proj.weight": "model-00007-of-00101.safetensors", + "model.layers.8.mlp.experts.61.down_proj.weight": "model-00007-of-00101.safetensors", + "model.layers.8.mlp.experts.62.gate_proj.weight": "model-00007-of-00101.safetensors", + "model.layers.8.mlp.experts.62.up_proj.weight": "model-00007-of-00101.safetensors", + "model.layers.8.mlp.experts.62.down_proj.weight": "model-00007-of-00101.safetensors", + "model.layers.8.mlp.experts.63.gate_proj.weight": "model-00007-of-00101.safetensors", + "model.layers.8.mlp.experts.63.up_proj.weight": "model-00007-of-00101.safetensors", + "model.layers.8.mlp.experts.63.down_proj.weight": "model-00007-of-00101.safetensors", + "model.layers.8.mlp.experts.64.gate_proj.weight": "model-00007-of-00101.safetensors", + "model.layers.8.mlp.experts.64.up_proj.weight": "model-00007-of-00101.safetensors", + "model.layers.8.mlp.experts.64.down_proj.weight": "model-00007-of-00101.safetensors", + "model.layers.8.mlp.experts.65.gate_proj.weight": "model-00007-of-00101.safetensors", + "model.layers.8.mlp.experts.65.up_proj.weight": "model-00007-of-00101.safetensors", + "model.layers.8.mlp.experts.65.down_proj.weight": "model-00007-of-00101.safetensors", + "model.layers.8.mlp.experts.66.gate_proj.weight": "model-00007-of-00101.safetensors", + "model.layers.8.mlp.experts.66.up_proj.weight": "model-00007-of-00101.safetensors", + "model.layers.8.mlp.experts.66.down_proj.weight": "model-00007-of-00101.safetensors", + "model.layers.8.mlp.experts.67.gate_proj.weight": "model-00007-of-00101.safetensors", + "model.layers.8.mlp.experts.67.up_proj.weight": "model-00007-of-00101.safetensors", + "model.layers.8.mlp.experts.67.down_proj.weight": "model-00007-of-00101.safetensors", + "model.layers.8.mlp.experts.68.gate_proj.weight": "model-00007-of-00101.safetensors", + "model.layers.8.mlp.experts.68.up_proj.weight": "model-00007-of-00101.safetensors", + "model.layers.8.mlp.experts.68.down_proj.weight": "model-00007-of-00101.safetensors", + "model.layers.8.mlp.experts.69.gate_proj.weight": "model-00007-of-00101.safetensors", + "model.layers.8.mlp.experts.69.up_proj.weight": "model-00007-of-00101.safetensors", + "model.layers.8.mlp.experts.69.down_proj.weight": "model-00007-of-00101.safetensors", + "model.layers.8.mlp.experts.70.gate_proj.weight": "model-00007-of-00101.safetensors", + "model.layers.8.mlp.experts.70.up_proj.weight": "model-00007-of-00101.safetensors", + "model.layers.8.mlp.experts.70.down_proj.weight": "model-00007-of-00101.safetensors", + "model.layers.8.mlp.experts.71.gate_proj.weight": "model-00007-of-00101.safetensors", + "model.layers.8.mlp.experts.71.up_proj.weight": "model-00007-of-00101.safetensors", + "model.layers.8.mlp.experts.71.down_proj.weight": "model-00007-of-00101.safetensors", + "model.layers.8.mlp.experts.72.gate_proj.weight": "model-00007-of-00101.safetensors", + "model.layers.8.mlp.experts.72.up_proj.weight": "model-00007-of-00101.safetensors", + "model.layers.8.mlp.experts.72.down_proj.weight": "model-00007-of-00101.safetensors", + "model.layers.8.mlp.experts.73.gate_proj.weight": "model-00007-of-00101.safetensors", + "model.layers.8.mlp.experts.73.up_proj.weight": "model-00007-of-00101.safetensors", + "model.layers.8.mlp.experts.73.down_proj.weight": "model-00007-of-00101.safetensors", + "model.layers.8.mlp.experts.74.gate_proj.weight": "model-00007-of-00101.safetensors", + "model.layers.8.mlp.experts.74.up_proj.weight": "model-00007-of-00101.safetensors", + "model.layers.8.mlp.experts.74.down_proj.weight": "model-00007-of-00101.safetensors", + "model.layers.8.mlp.experts.75.gate_proj.weight": "model-00007-of-00101.safetensors", + "model.layers.8.mlp.experts.75.up_proj.weight": "model-00007-of-00101.safetensors", + "model.layers.8.mlp.experts.75.down_proj.weight": "model-00007-of-00101.safetensors", + "model.layers.8.mlp.experts.76.gate_proj.weight": "model-00007-of-00101.safetensors", + "model.layers.8.mlp.experts.76.up_proj.weight": "model-00007-of-00101.safetensors", + "model.layers.8.mlp.experts.76.down_proj.weight": "model-00007-of-00101.safetensors", + "model.layers.8.mlp.experts.77.gate_proj.weight": "model-00007-of-00101.safetensors", + "model.layers.8.mlp.experts.77.up_proj.weight": "model-00007-of-00101.safetensors", + "model.layers.8.mlp.experts.77.down_proj.weight": "model-00007-of-00101.safetensors", + "model.layers.8.mlp.experts.78.gate_proj.weight": "model-00007-of-00101.safetensors", + "model.layers.8.mlp.experts.78.up_proj.weight": "model-00007-of-00101.safetensors", + "model.layers.8.mlp.experts.78.down_proj.weight": "model-00007-of-00101.safetensors", + "model.layers.8.mlp.experts.79.gate_proj.weight": "model-00007-of-00101.safetensors", + "model.layers.8.mlp.experts.79.up_proj.weight": "model-00007-of-00101.safetensors", + "model.layers.8.mlp.experts.79.down_proj.weight": "model-00007-of-00101.safetensors", + "model.layers.8.mlp.experts.80.gate_proj.weight": "model-00007-of-00101.safetensors", + "model.layers.8.mlp.experts.80.up_proj.weight": "model-00007-of-00101.safetensors", + "model.layers.8.mlp.experts.80.down_proj.weight": "model-00008-of-00101.safetensors", + "model.layers.8.mlp.experts.81.gate_proj.weight": "model-00008-of-00101.safetensors", + "model.layers.8.mlp.experts.81.up_proj.weight": "model-00008-of-00101.safetensors", + "model.layers.8.mlp.experts.81.down_proj.weight": "model-00008-of-00101.safetensors", + "model.layers.8.mlp.experts.82.gate_proj.weight": "model-00008-of-00101.safetensors", + "model.layers.8.mlp.experts.82.up_proj.weight": "model-00008-of-00101.safetensors", + "model.layers.8.mlp.experts.82.down_proj.weight": "model-00008-of-00101.safetensors", + "model.layers.8.mlp.experts.83.gate_proj.weight": "model-00008-of-00101.safetensors", + "model.layers.8.mlp.experts.83.up_proj.weight": "model-00008-of-00101.safetensors", + "model.layers.8.mlp.experts.83.down_proj.weight": "model-00008-of-00101.safetensors", + "model.layers.8.mlp.experts.84.gate_proj.weight": "model-00008-of-00101.safetensors", + "model.layers.8.mlp.experts.84.up_proj.weight": "model-00008-of-00101.safetensors", + "model.layers.8.mlp.experts.84.down_proj.weight": "model-00008-of-00101.safetensors", + "model.layers.8.mlp.experts.85.gate_proj.weight": "model-00008-of-00101.safetensors", + "model.layers.8.mlp.experts.85.up_proj.weight": "model-00008-of-00101.safetensors", + "model.layers.8.mlp.experts.85.down_proj.weight": "model-00008-of-00101.safetensors", + "model.layers.8.mlp.experts.86.gate_proj.weight": "model-00008-of-00101.safetensors", + "model.layers.8.mlp.experts.86.up_proj.weight": "model-00008-of-00101.safetensors", + "model.layers.8.mlp.experts.86.down_proj.weight": "model-00008-of-00101.safetensors", + "model.layers.8.mlp.experts.87.gate_proj.weight": "model-00008-of-00101.safetensors", + "model.layers.8.mlp.experts.87.up_proj.weight": "model-00008-of-00101.safetensors", + "model.layers.8.mlp.experts.87.down_proj.weight": "model-00008-of-00101.safetensors", + "model.layers.8.mlp.experts.88.gate_proj.weight": "model-00008-of-00101.safetensors", + "model.layers.8.mlp.experts.88.up_proj.weight": "model-00008-of-00101.safetensors", + "model.layers.8.mlp.experts.88.down_proj.weight": "model-00008-of-00101.safetensors", + "model.layers.8.mlp.experts.89.gate_proj.weight": "model-00008-of-00101.safetensors", + "model.layers.8.mlp.experts.89.up_proj.weight": "model-00008-of-00101.safetensors", + "model.layers.8.mlp.experts.89.down_proj.weight": "model-00008-of-00101.safetensors", + "model.layers.8.mlp.experts.90.gate_proj.weight": "model-00008-of-00101.safetensors", + "model.layers.8.mlp.experts.90.up_proj.weight": "model-00008-of-00101.safetensors", + "model.layers.8.mlp.experts.90.down_proj.weight": "model-00008-of-00101.safetensors", + "model.layers.8.mlp.experts.91.gate_proj.weight": "model-00008-of-00101.safetensors", + "model.layers.8.mlp.experts.91.up_proj.weight": "model-00008-of-00101.safetensors", + "model.layers.8.mlp.experts.91.down_proj.weight": "model-00008-of-00101.safetensors", + "model.layers.8.mlp.experts.92.gate_proj.weight": "model-00008-of-00101.safetensors", + "model.layers.8.mlp.experts.92.up_proj.weight": "model-00008-of-00101.safetensors", + "model.layers.8.mlp.experts.92.down_proj.weight": "model-00008-of-00101.safetensors", + "model.layers.8.mlp.experts.93.gate_proj.weight": "model-00008-of-00101.safetensors", + "model.layers.8.mlp.experts.93.up_proj.weight": "model-00008-of-00101.safetensors", + "model.layers.8.mlp.experts.93.down_proj.weight": "model-00008-of-00101.safetensors", + "model.layers.8.mlp.experts.94.gate_proj.weight": "model-00008-of-00101.safetensors", + "model.layers.8.mlp.experts.94.up_proj.weight": "model-00008-of-00101.safetensors", + "model.layers.8.mlp.experts.94.down_proj.weight": "model-00008-of-00101.safetensors", + "model.layers.8.mlp.experts.95.gate_proj.weight": "model-00008-of-00101.safetensors", + "model.layers.8.mlp.experts.95.up_proj.weight": "model-00008-of-00101.safetensors", + "model.layers.8.mlp.experts.95.down_proj.weight": "model-00008-of-00101.safetensors", + "model.layers.8.mlp.experts.96.gate_proj.weight": "model-00008-of-00101.safetensors", + "model.layers.8.mlp.experts.96.up_proj.weight": "model-00008-of-00101.safetensors", + "model.layers.8.mlp.experts.96.down_proj.weight": "model-00008-of-00101.safetensors", + "model.layers.8.mlp.experts.97.gate_proj.weight": "model-00008-of-00101.safetensors", + "model.layers.8.mlp.experts.97.up_proj.weight": "model-00008-of-00101.safetensors", + "model.layers.8.mlp.experts.97.down_proj.weight": "model-00008-of-00101.safetensors", + "model.layers.8.mlp.experts.98.gate_proj.weight": "model-00008-of-00101.safetensors", + "model.layers.8.mlp.experts.98.up_proj.weight": "model-00008-of-00101.safetensors", + "model.layers.8.mlp.experts.98.down_proj.weight": "model-00008-of-00101.safetensors", + "model.layers.8.mlp.experts.99.gate_proj.weight": "model-00008-of-00101.safetensors", + "model.layers.8.mlp.experts.99.up_proj.weight": "model-00008-of-00101.safetensors", + "model.layers.8.mlp.experts.99.down_proj.weight": "model-00008-of-00101.safetensors", + "model.layers.8.mlp.experts.100.gate_proj.weight": "model-00008-of-00101.safetensors", + "model.layers.8.mlp.experts.100.up_proj.weight": "model-00008-of-00101.safetensors", + "model.layers.8.mlp.experts.100.down_proj.weight": "model-00008-of-00101.safetensors", + "model.layers.8.mlp.experts.101.gate_proj.weight": "model-00008-of-00101.safetensors", + "model.layers.8.mlp.experts.101.up_proj.weight": "model-00008-of-00101.safetensors", + "model.layers.8.mlp.experts.101.down_proj.weight": "model-00008-of-00101.safetensors", + "model.layers.8.mlp.experts.102.gate_proj.weight": "model-00008-of-00101.safetensors", + "model.layers.8.mlp.experts.102.up_proj.weight": "model-00008-of-00101.safetensors", + "model.layers.8.mlp.experts.102.down_proj.weight": "model-00008-of-00101.safetensors", + "model.layers.8.mlp.experts.103.gate_proj.weight": "model-00008-of-00101.safetensors", + "model.layers.8.mlp.experts.103.up_proj.weight": "model-00008-of-00101.safetensors", + "model.layers.8.mlp.experts.103.down_proj.weight": "model-00008-of-00101.safetensors", + "model.layers.8.mlp.experts.104.gate_proj.weight": "model-00008-of-00101.safetensors", + "model.layers.8.mlp.experts.104.up_proj.weight": "model-00008-of-00101.safetensors", + "model.layers.8.mlp.experts.104.down_proj.weight": "model-00008-of-00101.safetensors", + "model.layers.8.mlp.experts.105.gate_proj.weight": "model-00008-of-00101.safetensors", + "model.layers.8.mlp.experts.105.up_proj.weight": "model-00008-of-00101.safetensors", + "model.layers.8.mlp.experts.105.down_proj.weight": "model-00008-of-00101.safetensors", + "model.layers.8.mlp.experts.106.gate_proj.weight": "model-00008-of-00101.safetensors", + "model.layers.8.mlp.experts.106.up_proj.weight": "model-00008-of-00101.safetensors", + "model.layers.8.mlp.experts.106.down_proj.weight": "model-00008-of-00101.safetensors", + "model.layers.8.mlp.experts.107.gate_proj.weight": "model-00008-of-00101.safetensors", + "model.layers.8.mlp.experts.107.up_proj.weight": "model-00008-of-00101.safetensors", + "model.layers.8.mlp.experts.107.down_proj.weight": "model-00008-of-00101.safetensors", + "model.layers.8.mlp.experts.108.gate_proj.weight": "model-00008-of-00101.safetensors", + "model.layers.8.mlp.experts.108.up_proj.weight": "model-00008-of-00101.safetensors", + "model.layers.8.mlp.experts.108.down_proj.weight": "model-00008-of-00101.safetensors", + "model.layers.8.mlp.experts.109.gate_proj.weight": "model-00008-of-00101.safetensors", + "model.layers.8.mlp.experts.109.up_proj.weight": "model-00008-of-00101.safetensors", + "model.layers.8.mlp.experts.109.down_proj.weight": "model-00008-of-00101.safetensors", + "model.layers.8.mlp.experts.110.gate_proj.weight": "model-00008-of-00101.safetensors", + "model.layers.8.mlp.experts.110.up_proj.weight": "model-00008-of-00101.safetensors", + "model.layers.8.mlp.experts.110.down_proj.weight": "model-00008-of-00101.safetensors", + "model.layers.8.mlp.experts.111.gate_proj.weight": "model-00008-of-00101.safetensors", + "model.layers.8.mlp.experts.111.up_proj.weight": "model-00008-of-00101.safetensors", + "model.layers.8.mlp.experts.111.down_proj.weight": "model-00008-of-00101.safetensors", + "model.layers.8.mlp.experts.112.gate_proj.weight": "model-00008-of-00101.safetensors", + "model.layers.8.mlp.experts.112.up_proj.weight": "model-00008-of-00101.safetensors", + "model.layers.8.mlp.experts.112.down_proj.weight": "model-00008-of-00101.safetensors", + "model.layers.8.mlp.experts.113.gate_proj.weight": "model-00008-of-00101.safetensors", + "model.layers.8.mlp.experts.113.up_proj.weight": "model-00008-of-00101.safetensors", + "model.layers.8.mlp.experts.113.down_proj.weight": "model-00008-of-00101.safetensors", + "model.layers.8.mlp.experts.114.gate_proj.weight": "model-00008-of-00101.safetensors", + "model.layers.8.mlp.experts.114.up_proj.weight": "model-00008-of-00101.safetensors", + "model.layers.8.mlp.experts.114.down_proj.weight": "model-00008-of-00101.safetensors", + "model.layers.8.mlp.experts.115.gate_proj.weight": "model-00008-of-00101.safetensors", + "model.layers.8.mlp.experts.115.up_proj.weight": "model-00008-of-00101.safetensors", + "model.layers.8.mlp.experts.115.down_proj.weight": "model-00008-of-00101.safetensors", + "model.layers.8.mlp.experts.116.gate_proj.weight": "model-00008-of-00101.safetensors", + "model.layers.8.mlp.experts.116.up_proj.weight": "model-00008-of-00101.safetensors", + "model.layers.8.mlp.experts.116.down_proj.weight": "model-00008-of-00101.safetensors", + "model.layers.8.mlp.experts.117.gate_proj.weight": "model-00008-of-00101.safetensors", + "model.layers.8.mlp.experts.117.up_proj.weight": "model-00008-of-00101.safetensors", + "model.layers.8.mlp.experts.117.down_proj.weight": "model-00008-of-00101.safetensors", + "model.layers.8.mlp.experts.118.gate_proj.weight": "model-00008-of-00101.safetensors", + "model.layers.8.mlp.experts.118.up_proj.weight": "model-00008-of-00101.safetensors", + "model.layers.8.mlp.experts.118.down_proj.weight": "model-00008-of-00101.safetensors", + "model.layers.8.mlp.experts.119.gate_proj.weight": "model-00008-of-00101.safetensors", + "model.layers.8.mlp.experts.119.up_proj.weight": "model-00008-of-00101.safetensors", + "model.layers.8.mlp.experts.119.down_proj.weight": "model-00008-of-00101.safetensors", + "model.layers.8.mlp.gate.weight": "model-00008-of-00101.safetensors", + "model.layers.8.mlp.gate.e_score_correction_bias": "model-00008-of-00101.safetensors", + "model.layers.8.mlp.shared_experts.gate_proj.weight": "model-00008-of-00101.safetensors", + "model.layers.8.mlp.shared_experts.up_proj.weight": "model-00008-of-00101.safetensors", + "model.layers.8.mlp.shared_experts.down_proj.weight": "model-00008-of-00101.safetensors", + "model.layers.8.input_layernorm.weight": "model-00008-of-00101.safetensors", + "model.layers.8.post_attention_layernorm.weight": "model-00008-of-00101.safetensors", + "model.layers.9.self_attn.q_proj.weight": "model-00008-of-00101.safetensors", + "model.layers.9.self_attn.q_proj.bias": "model-00008-of-00101.safetensors", + "model.layers.9.self_attn.k_proj.weight": "model-00008-of-00101.safetensors", + "model.layers.9.self_attn.k_proj.bias": "model-00008-of-00101.safetensors", + "model.layers.9.self_attn.v_proj.weight": "model-00008-of-00101.safetensors", + "model.layers.9.self_attn.v_proj.bias": "model-00008-of-00101.safetensors", + "model.layers.9.self_attn.o_proj.weight": "model-00008-of-00101.safetensors", + "model.layers.9.self_attn.q_norm.weight": "model-00008-of-00101.safetensors", + "model.layers.9.self_attn.k_norm.weight": "model-00008-of-00101.safetensors", + "model.layers.9.mlp.experts.0.gate_proj.weight": "model-00008-of-00101.safetensors", + "model.layers.9.mlp.experts.0.up_proj.weight": "model-00008-of-00101.safetensors", + "model.layers.9.mlp.experts.0.down_proj.weight": "model-00008-of-00101.safetensors", + "model.layers.9.mlp.experts.1.gate_proj.weight": "model-00008-of-00101.safetensors", + "model.layers.9.mlp.experts.1.up_proj.weight": "model-00008-of-00101.safetensors", + "model.layers.9.mlp.experts.1.down_proj.weight": "model-00008-of-00101.safetensors", + "model.layers.9.mlp.experts.2.gate_proj.weight": "model-00008-of-00101.safetensors", + "model.layers.9.mlp.experts.2.up_proj.weight": "model-00008-of-00101.safetensors", + "model.layers.9.mlp.experts.2.down_proj.weight": "model-00008-of-00101.safetensors", + "model.layers.9.mlp.experts.3.gate_proj.weight": "model-00008-of-00101.safetensors", + "model.layers.9.mlp.experts.3.up_proj.weight": "model-00008-of-00101.safetensors", + "model.layers.9.mlp.experts.3.down_proj.weight": "model-00008-of-00101.safetensors", + "model.layers.9.mlp.experts.4.gate_proj.weight": "model-00008-of-00101.safetensors", + "model.layers.9.mlp.experts.4.up_proj.weight": "model-00008-of-00101.safetensors", + "model.layers.9.mlp.experts.4.down_proj.weight": "model-00008-of-00101.safetensors", + "model.layers.9.mlp.experts.5.gate_proj.weight": "model-00008-of-00101.safetensors", + "model.layers.9.mlp.experts.5.up_proj.weight": "model-00008-of-00101.safetensors", + "model.layers.9.mlp.experts.5.down_proj.weight": "model-00008-of-00101.safetensors", + "model.layers.9.mlp.experts.6.gate_proj.weight": "model-00008-of-00101.safetensors", + "model.layers.9.mlp.experts.6.up_proj.weight": "model-00008-of-00101.safetensors", + "model.layers.9.mlp.experts.6.down_proj.weight": "model-00008-of-00101.safetensors", + "model.layers.9.mlp.experts.7.gate_proj.weight": "model-00008-of-00101.safetensors", + "model.layers.9.mlp.experts.7.up_proj.weight": "model-00008-of-00101.safetensors", + "model.layers.9.mlp.experts.7.down_proj.weight": "model-00008-of-00101.safetensors", + "model.layers.9.mlp.experts.8.gate_proj.weight": "model-00008-of-00101.safetensors", + "model.layers.9.mlp.experts.8.up_proj.weight": "model-00008-of-00101.safetensors", + "model.layers.9.mlp.experts.8.down_proj.weight": "model-00008-of-00101.safetensors", + "model.layers.9.mlp.experts.9.gate_proj.weight": "model-00008-of-00101.safetensors", + "model.layers.9.mlp.experts.9.up_proj.weight": "model-00008-of-00101.safetensors", + "model.layers.9.mlp.experts.9.down_proj.weight": "model-00008-of-00101.safetensors", + "model.layers.9.mlp.experts.10.gate_proj.weight": "model-00008-of-00101.safetensors", + "model.layers.9.mlp.experts.10.up_proj.weight": "model-00008-of-00101.safetensors", + "model.layers.9.mlp.experts.10.down_proj.weight": "model-00008-of-00101.safetensors", + "model.layers.9.mlp.experts.11.gate_proj.weight": "model-00008-of-00101.safetensors", + "model.layers.9.mlp.experts.11.up_proj.weight": "model-00008-of-00101.safetensors", + "model.layers.9.mlp.experts.11.down_proj.weight": "model-00008-of-00101.safetensors", + "model.layers.9.mlp.experts.12.gate_proj.weight": "model-00008-of-00101.safetensors", + "model.layers.9.mlp.experts.12.up_proj.weight": "model-00008-of-00101.safetensors", + "model.layers.9.mlp.experts.12.down_proj.weight": "model-00008-of-00101.safetensors", + "model.layers.9.mlp.experts.13.gate_proj.weight": "model-00008-of-00101.safetensors", + "model.layers.9.mlp.experts.13.up_proj.weight": "model-00008-of-00101.safetensors", + "model.layers.9.mlp.experts.13.down_proj.weight": "model-00008-of-00101.safetensors", + "model.layers.9.mlp.experts.14.gate_proj.weight": "model-00008-of-00101.safetensors", + "model.layers.9.mlp.experts.14.up_proj.weight": "model-00008-of-00101.safetensors", + "model.layers.9.mlp.experts.14.down_proj.weight": "model-00008-of-00101.safetensors", + "model.layers.9.mlp.experts.15.gate_proj.weight": "model-00008-of-00101.safetensors", + "model.layers.9.mlp.experts.15.up_proj.weight": "model-00008-of-00101.safetensors", + "model.layers.9.mlp.experts.15.down_proj.weight": "model-00008-of-00101.safetensors", + "model.layers.9.mlp.experts.16.gate_proj.weight": "model-00008-of-00101.safetensors", + "model.layers.9.mlp.experts.16.up_proj.weight": "model-00008-of-00101.safetensors", + "model.layers.9.mlp.experts.16.down_proj.weight": "model-00008-of-00101.safetensors", + "model.layers.9.mlp.experts.17.gate_proj.weight": "model-00008-of-00101.safetensors", + "model.layers.9.mlp.experts.17.up_proj.weight": "model-00008-of-00101.safetensors", + "model.layers.9.mlp.experts.17.down_proj.weight": "model-00008-of-00101.safetensors", + "model.layers.9.mlp.experts.18.gate_proj.weight": "model-00008-of-00101.safetensors", + "model.layers.9.mlp.experts.18.up_proj.weight": "model-00008-of-00101.safetensors", + "model.layers.9.mlp.experts.18.down_proj.weight": "model-00008-of-00101.safetensors", + "model.layers.9.mlp.experts.19.gate_proj.weight": "model-00008-of-00101.safetensors", + "model.layers.9.mlp.experts.19.up_proj.weight": "model-00008-of-00101.safetensors", + "model.layers.9.mlp.experts.19.down_proj.weight": "model-00008-of-00101.safetensors", + "model.layers.9.mlp.experts.20.gate_proj.weight": "model-00008-of-00101.safetensors", + "model.layers.9.mlp.experts.20.up_proj.weight": "model-00008-of-00101.safetensors", + "model.layers.9.mlp.experts.20.down_proj.weight": "model-00008-of-00101.safetensors", + "model.layers.9.mlp.experts.21.gate_proj.weight": "model-00008-of-00101.safetensors", + "model.layers.9.mlp.experts.21.up_proj.weight": "model-00008-of-00101.safetensors", + "model.layers.9.mlp.experts.21.down_proj.weight": "model-00008-of-00101.safetensors", + "model.layers.9.mlp.experts.22.gate_proj.weight": "model-00008-of-00101.safetensors", + "model.layers.9.mlp.experts.22.up_proj.weight": "model-00008-of-00101.safetensors", + "model.layers.9.mlp.experts.22.down_proj.weight": "model-00008-of-00101.safetensors", + "model.layers.9.mlp.experts.23.gate_proj.weight": "model-00008-of-00101.safetensors", + "model.layers.9.mlp.experts.23.up_proj.weight": "model-00008-of-00101.safetensors", + "model.layers.9.mlp.experts.23.down_proj.weight": "model-00008-of-00101.safetensors", + "model.layers.9.mlp.experts.24.gate_proj.weight": "model-00008-of-00101.safetensors", + "model.layers.9.mlp.experts.24.up_proj.weight": "model-00008-of-00101.safetensors", + "model.layers.9.mlp.experts.24.down_proj.weight": "model-00008-of-00101.safetensors", + "model.layers.9.mlp.experts.25.gate_proj.weight": "model-00008-of-00101.safetensors", + "model.layers.9.mlp.experts.25.up_proj.weight": "model-00008-of-00101.safetensors", + "model.layers.9.mlp.experts.25.down_proj.weight": "model-00008-of-00101.safetensors", + "model.layers.9.mlp.experts.26.gate_proj.weight": "model-00008-of-00101.safetensors", + "model.layers.9.mlp.experts.26.up_proj.weight": "model-00008-of-00101.safetensors", + "model.layers.9.mlp.experts.26.down_proj.weight": "model-00008-of-00101.safetensors", + "model.layers.9.mlp.experts.27.gate_proj.weight": "model-00008-of-00101.safetensors", + "model.layers.9.mlp.experts.27.up_proj.weight": "model-00008-of-00101.safetensors", + "model.layers.9.mlp.experts.27.down_proj.weight": "model-00008-of-00101.safetensors", + "model.layers.9.mlp.experts.28.gate_proj.weight": "model-00008-of-00101.safetensors", + "model.layers.9.mlp.experts.28.up_proj.weight": "model-00008-of-00101.safetensors", + "model.layers.9.mlp.experts.28.down_proj.weight": "model-00008-of-00101.safetensors", + "model.layers.9.mlp.experts.29.gate_proj.weight": "model-00008-of-00101.safetensors", + "model.layers.9.mlp.experts.29.up_proj.weight": "model-00008-of-00101.safetensors", + "model.layers.9.mlp.experts.29.down_proj.weight": "model-00008-of-00101.safetensors", + "model.layers.9.mlp.experts.30.gate_proj.weight": "model-00008-of-00101.safetensors", + "model.layers.9.mlp.experts.30.up_proj.weight": "model-00008-of-00101.safetensors", + "model.layers.9.mlp.experts.30.down_proj.weight": "model-00008-of-00101.safetensors", + "model.layers.9.mlp.experts.31.gate_proj.weight": "model-00008-of-00101.safetensors", + "model.layers.9.mlp.experts.31.up_proj.weight": "model-00008-of-00101.safetensors", + "model.layers.9.mlp.experts.31.down_proj.weight": "model-00008-of-00101.safetensors", + "model.layers.9.mlp.experts.32.gate_proj.weight": "model-00008-of-00101.safetensors", + "model.layers.9.mlp.experts.32.up_proj.weight": "model-00008-of-00101.safetensors", + "model.layers.9.mlp.experts.32.down_proj.weight": "model-00008-of-00101.safetensors", + "model.layers.9.mlp.experts.33.gate_proj.weight": "model-00008-of-00101.safetensors", + "model.layers.9.mlp.experts.33.up_proj.weight": "model-00008-of-00101.safetensors", + "model.layers.9.mlp.experts.33.down_proj.weight": "model-00008-of-00101.safetensors", + "model.layers.9.mlp.experts.34.gate_proj.weight": "model-00008-of-00101.safetensors", + "model.layers.9.mlp.experts.34.up_proj.weight": "model-00008-of-00101.safetensors", + "model.layers.9.mlp.experts.34.down_proj.weight": "model-00008-of-00101.safetensors", + "model.layers.9.mlp.experts.35.gate_proj.weight": "model-00008-of-00101.safetensors", + "model.layers.9.mlp.experts.35.up_proj.weight": "model-00008-of-00101.safetensors", + "model.layers.9.mlp.experts.35.down_proj.weight": "model-00008-of-00101.safetensors", + "model.layers.9.mlp.experts.36.gate_proj.weight": "model-00008-of-00101.safetensors", + "model.layers.9.mlp.experts.36.up_proj.weight": "model-00008-of-00101.safetensors", + "model.layers.9.mlp.experts.36.down_proj.weight": "model-00008-of-00101.safetensors", + "model.layers.9.mlp.experts.37.gate_proj.weight": "model-00008-of-00101.safetensors", + "model.layers.9.mlp.experts.37.up_proj.weight": "model-00008-of-00101.safetensors", + "model.layers.9.mlp.experts.37.down_proj.weight": "model-00008-of-00101.safetensors", + "model.layers.9.mlp.experts.38.gate_proj.weight": "model-00008-of-00101.safetensors", + "model.layers.9.mlp.experts.38.up_proj.weight": "model-00008-of-00101.safetensors", + "model.layers.9.mlp.experts.38.down_proj.weight": "model-00008-of-00101.safetensors", + "model.layers.9.mlp.experts.39.gate_proj.weight": "model-00008-of-00101.safetensors", + "model.layers.9.mlp.experts.39.up_proj.weight": "model-00008-of-00101.safetensors", + "model.layers.9.mlp.experts.39.down_proj.weight": "model-00008-of-00101.safetensors", + "model.layers.9.mlp.experts.40.gate_proj.weight": "model-00008-of-00101.safetensors", + "model.layers.9.mlp.experts.40.up_proj.weight": "model-00008-of-00101.safetensors", + "model.layers.9.mlp.experts.40.down_proj.weight": "model-00008-of-00101.safetensors", + "model.layers.9.mlp.experts.41.gate_proj.weight": "model-00008-of-00101.safetensors", + "model.layers.9.mlp.experts.41.up_proj.weight": "model-00008-of-00101.safetensors", + "model.layers.9.mlp.experts.41.down_proj.weight": "model-00008-of-00101.safetensors", + "model.layers.9.mlp.experts.42.gate_proj.weight": "model-00008-of-00101.safetensors", + "model.layers.9.mlp.experts.42.up_proj.weight": "model-00008-of-00101.safetensors", + "model.layers.9.mlp.experts.42.down_proj.weight": "model-00008-of-00101.safetensors", + "model.layers.9.mlp.experts.43.gate_proj.weight": "model-00008-of-00101.safetensors", + "model.layers.9.mlp.experts.43.up_proj.weight": "model-00008-of-00101.safetensors", + "model.layers.9.mlp.experts.43.down_proj.weight": "model-00008-of-00101.safetensors", + "model.layers.9.mlp.experts.44.gate_proj.weight": "model-00008-of-00101.safetensors", + "model.layers.9.mlp.experts.44.up_proj.weight": "model-00008-of-00101.safetensors", + "model.layers.9.mlp.experts.44.down_proj.weight": "model-00008-of-00101.safetensors", + "model.layers.9.mlp.experts.45.gate_proj.weight": "model-00008-of-00101.safetensors", + "model.layers.9.mlp.experts.45.up_proj.weight": "model-00008-of-00101.safetensors", + "model.layers.9.mlp.experts.45.down_proj.weight": "model-00008-of-00101.safetensors", + "model.layers.9.mlp.experts.46.gate_proj.weight": "model-00008-of-00101.safetensors", + "model.layers.9.mlp.experts.46.up_proj.weight": "model-00008-of-00101.safetensors", + "model.layers.9.mlp.experts.46.down_proj.weight": "model-00008-of-00101.safetensors", + "model.layers.9.mlp.experts.47.gate_proj.weight": "model-00008-of-00101.safetensors", + "model.layers.9.mlp.experts.47.up_proj.weight": "model-00008-of-00101.safetensors", + "model.layers.9.mlp.experts.47.down_proj.weight": "model-00008-of-00101.safetensors", + "model.layers.9.mlp.experts.48.gate_proj.weight": "model-00008-of-00101.safetensors", + "model.layers.9.mlp.experts.48.up_proj.weight": "model-00008-of-00101.safetensors", + "model.layers.9.mlp.experts.48.down_proj.weight": "model-00008-of-00101.safetensors", + "model.layers.9.mlp.experts.49.gate_proj.weight": "model-00008-of-00101.safetensors", + "model.layers.9.mlp.experts.49.up_proj.weight": "model-00008-of-00101.safetensors", + "model.layers.9.mlp.experts.49.down_proj.weight": "model-00008-of-00101.safetensors", + "model.layers.9.mlp.experts.50.gate_proj.weight": "model-00008-of-00101.safetensors", + "model.layers.9.mlp.experts.50.up_proj.weight": "model-00008-of-00101.safetensors", + "model.layers.9.mlp.experts.50.down_proj.weight": "model-00008-of-00101.safetensors", + "model.layers.9.mlp.experts.51.gate_proj.weight": "model-00008-of-00101.safetensors", + "model.layers.9.mlp.experts.51.up_proj.weight": "model-00008-of-00101.safetensors", + "model.layers.9.mlp.experts.51.down_proj.weight": "model-00008-of-00101.safetensors", + "model.layers.9.mlp.experts.52.gate_proj.weight": "model-00008-of-00101.safetensors", + "model.layers.9.mlp.experts.52.up_proj.weight": "model-00008-of-00101.safetensors", + "model.layers.9.mlp.experts.52.down_proj.weight": "model-00008-of-00101.safetensors", + "model.layers.9.mlp.experts.53.gate_proj.weight": "model-00008-of-00101.safetensors", + "model.layers.9.mlp.experts.53.up_proj.weight": "model-00008-of-00101.safetensors", + "model.layers.9.mlp.experts.53.down_proj.weight": "model-00008-of-00101.safetensors", + "model.layers.9.mlp.experts.54.gate_proj.weight": "model-00008-of-00101.safetensors", + "model.layers.9.mlp.experts.54.up_proj.weight": "model-00008-of-00101.safetensors", + "model.layers.9.mlp.experts.54.down_proj.weight": "model-00008-of-00101.safetensors", + "model.layers.9.mlp.experts.55.gate_proj.weight": "model-00008-of-00101.safetensors", + "model.layers.9.mlp.experts.55.up_proj.weight": "model-00008-of-00101.safetensors", + "model.layers.9.mlp.experts.55.down_proj.weight": "model-00008-of-00101.safetensors", + "model.layers.9.mlp.experts.56.gate_proj.weight": "model-00008-of-00101.safetensors", + "model.layers.9.mlp.experts.56.up_proj.weight": "model-00008-of-00101.safetensors", + "model.layers.9.mlp.experts.56.down_proj.weight": "model-00008-of-00101.safetensors", + "model.layers.9.mlp.experts.57.gate_proj.weight": "model-00008-of-00101.safetensors", + "model.layers.9.mlp.experts.57.up_proj.weight": "model-00008-of-00101.safetensors", + "model.layers.9.mlp.experts.57.down_proj.weight": "model-00008-of-00101.safetensors", + "model.layers.9.mlp.experts.58.gate_proj.weight": "model-00008-of-00101.safetensors", + "model.layers.9.mlp.experts.58.up_proj.weight": "model-00008-of-00101.safetensors", + "model.layers.9.mlp.experts.58.down_proj.weight": "model-00008-of-00101.safetensors", + "model.layers.9.mlp.experts.59.gate_proj.weight": "model-00008-of-00101.safetensors", + "model.layers.9.mlp.experts.59.up_proj.weight": "model-00008-of-00101.safetensors", + "model.layers.9.mlp.experts.59.down_proj.weight": "model-00008-of-00101.safetensors", + "model.layers.9.mlp.experts.60.gate_proj.weight": "model-00008-of-00101.safetensors", + "model.layers.9.mlp.experts.60.up_proj.weight": "model-00008-of-00101.safetensors", + "model.layers.9.mlp.experts.60.down_proj.weight": "model-00008-of-00101.safetensors", + "model.layers.9.mlp.experts.61.gate_proj.weight": "model-00008-of-00101.safetensors", + "model.layers.9.mlp.experts.61.up_proj.weight": "model-00008-of-00101.safetensors", + "model.layers.9.mlp.experts.61.down_proj.weight": "model-00008-of-00101.safetensors", + "model.layers.9.mlp.experts.62.gate_proj.weight": "model-00008-of-00101.safetensors", + "model.layers.9.mlp.experts.62.up_proj.weight": "model-00008-of-00101.safetensors", + "model.layers.9.mlp.experts.62.down_proj.weight": "model-00008-of-00101.safetensors", + "model.layers.9.mlp.experts.63.gate_proj.weight": "model-00008-of-00101.safetensors", + "model.layers.9.mlp.experts.63.up_proj.weight": "model-00008-of-00101.safetensors", + "model.layers.9.mlp.experts.63.down_proj.weight": "model-00008-of-00101.safetensors", + "model.layers.9.mlp.experts.64.gate_proj.weight": "model-00008-of-00101.safetensors", + "model.layers.9.mlp.experts.64.up_proj.weight": "model-00008-of-00101.safetensors", + "model.layers.9.mlp.experts.64.down_proj.weight": "model-00008-of-00101.safetensors", + "model.layers.9.mlp.experts.65.gate_proj.weight": "model-00008-of-00101.safetensors", + "model.layers.9.mlp.experts.65.up_proj.weight": "model-00008-of-00101.safetensors", + "model.layers.9.mlp.experts.65.down_proj.weight": "model-00008-of-00101.safetensors", + "model.layers.9.mlp.experts.66.gate_proj.weight": "model-00008-of-00101.safetensors", + "model.layers.9.mlp.experts.66.up_proj.weight": "model-00008-of-00101.safetensors", + "model.layers.9.mlp.experts.66.down_proj.weight": "model-00008-of-00101.safetensors", + "model.layers.9.mlp.experts.67.gate_proj.weight": "model-00008-of-00101.safetensors", + "model.layers.9.mlp.experts.67.up_proj.weight": "model-00009-of-00101.safetensors", + "model.layers.9.mlp.experts.67.down_proj.weight": "model-00009-of-00101.safetensors", + "model.layers.9.mlp.experts.68.gate_proj.weight": "model-00009-of-00101.safetensors", + "model.layers.9.mlp.experts.68.up_proj.weight": "model-00009-of-00101.safetensors", + "model.layers.9.mlp.experts.68.down_proj.weight": "model-00009-of-00101.safetensors", + "model.layers.9.mlp.experts.69.gate_proj.weight": "model-00009-of-00101.safetensors", + "model.layers.9.mlp.experts.69.up_proj.weight": "model-00009-of-00101.safetensors", + "model.layers.9.mlp.experts.69.down_proj.weight": "model-00009-of-00101.safetensors", + "model.layers.9.mlp.experts.70.gate_proj.weight": "model-00009-of-00101.safetensors", + "model.layers.9.mlp.experts.70.up_proj.weight": "model-00009-of-00101.safetensors", + "model.layers.9.mlp.experts.70.down_proj.weight": "model-00009-of-00101.safetensors", + "model.layers.9.mlp.experts.71.gate_proj.weight": "model-00009-of-00101.safetensors", + "model.layers.9.mlp.experts.71.up_proj.weight": "model-00009-of-00101.safetensors", + "model.layers.9.mlp.experts.71.down_proj.weight": "model-00009-of-00101.safetensors", + "model.layers.9.mlp.experts.72.gate_proj.weight": "model-00009-of-00101.safetensors", + "model.layers.9.mlp.experts.72.up_proj.weight": "model-00009-of-00101.safetensors", + "model.layers.9.mlp.experts.72.down_proj.weight": "model-00009-of-00101.safetensors", + "model.layers.9.mlp.experts.73.gate_proj.weight": "model-00009-of-00101.safetensors", + "model.layers.9.mlp.experts.73.up_proj.weight": "model-00009-of-00101.safetensors", + "model.layers.9.mlp.experts.73.down_proj.weight": "model-00009-of-00101.safetensors", + "model.layers.9.mlp.experts.74.gate_proj.weight": "model-00009-of-00101.safetensors", + "model.layers.9.mlp.experts.74.up_proj.weight": "model-00009-of-00101.safetensors", + "model.layers.9.mlp.experts.74.down_proj.weight": "model-00009-of-00101.safetensors", + "model.layers.9.mlp.experts.75.gate_proj.weight": "model-00009-of-00101.safetensors", + "model.layers.9.mlp.experts.75.up_proj.weight": "model-00009-of-00101.safetensors", + "model.layers.9.mlp.experts.75.down_proj.weight": "model-00009-of-00101.safetensors", + "model.layers.9.mlp.experts.76.gate_proj.weight": "model-00009-of-00101.safetensors", + "model.layers.9.mlp.experts.76.up_proj.weight": "model-00009-of-00101.safetensors", + "model.layers.9.mlp.experts.76.down_proj.weight": "model-00009-of-00101.safetensors", + "model.layers.9.mlp.experts.77.gate_proj.weight": "model-00009-of-00101.safetensors", + "model.layers.9.mlp.experts.77.up_proj.weight": "model-00009-of-00101.safetensors", + "model.layers.9.mlp.experts.77.down_proj.weight": "model-00009-of-00101.safetensors", + "model.layers.9.mlp.experts.78.gate_proj.weight": "model-00009-of-00101.safetensors", + "model.layers.9.mlp.experts.78.up_proj.weight": "model-00009-of-00101.safetensors", + "model.layers.9.mlp.experts.78.down_proj.weight": "model-00009-of-00101.safetensors", + "model.layers.9.mlp.experts.79.gate_proj.weight": "model-00009-of-00101.safetensors", + "model.layers.9.mlp.experts.79.up_proj.weight": "model-00009-of-00101.safetensors", + "model.layers.9.mlp.experts.79.down_proj.weight": "model-00009-of-00101.safetensors", + "model.layers.9.mlp.experts.80.gate_proj.weight": "model-00009-of-00101.safetensors", + "model.layers.9.mlp.experts.80.up_proj.weight": "model-00009-of-00101.safetensors", + "model.layers.9.mlp.experts.80.down_proj.weight": "model-00009-of-00101.safetensors", + "model.layers.9.mlp.experts.81.gate_proj.weight": "model-00009-of-00101.safetensors", + "model.layers.9.mlp.experts.81.up_proj.weight": "model-00009-of-00101.safetensors", + "model.layers.9.mlp.experts.81.down_proj.weight": "model-00009-of-00101.safetensors", + "model.layers.9.mlp.experts.82.gate_proj.weight": "model-00009-of-00101.safetensors", + "model.layers.9.mlp.experts.82.up_proj.weight": "model-00009-of-00101.safetensors", + "model.layers.9.mlp.experts.82.down_proj.weight": "model-00009-of-00101.safetensors", + "model.layers.9.mlp.experts.83.gate_proj.weight": "model-00009-of-00101.safetensors", + "model.layers.9.mlp.experts.83.up_proj.weight": "model-00009-of-00101.safetensors", + "model.layers.9.mlp.experts.83.down_proj.weight": "model-00009-of-00101.safetensors", + "model.layers.9.mlp.experts.84.gate_proj.weight": "model-00009-of-00101.safetensors", + "model.layers.9.mlp.experts.84.up_proj.weight": "model-00009-of-00101.safetensors", + "model.layers.9.mlp.experts.84.down_proj.weight": "model-00009-of-00101.safetensors", + "model.layers.9.mlp.experts.85.gate_proj.weight": "model-00009-of-00101.safetensors", + "model.layers.9.mlp.experts.85.up_proj.weight": "model-00009-of-00101.safetensors", + "model.layers.9.mlp.experts.85.down_proj.weight": "model-00009-of-00101.safetensors", + "model.layers.9.mlp.experts.86.gate_proj.weight": "model-00009-of-00101.safetensors", + "model.layers.9.mlp.experts.86.up_proj.weight": "model-00009-of-00101.safetensors", + "model.layers.9.mlp.experts.86.down_proj.weight": "model-00009-of-00101.safetensors", + "model.layers.9.mlp.experts.87.gate_proj.weight": "model-00009-of-00101.safetensors", + "model.layers.9.mlp.experts.87.up_proj.weight": "model-00009-of-00101.safetensors", + "model.layers.9.mlp.experts.87.down_proj.weight": "model-00009-of-00101.safetensors", + "model.layers.9.mlp.experts.88.gate_proj.weight": "model-00009-of-00101.safetensors", + "model.layers.9.mlp.experts.88.up_proj.weight": "model-00009-of-00101.safetensors", + "model.layers.9.mlp.experts.88.down_proj.weight": "model-00009-of-00101.safetensors", + "model.layers.9.mlp.experts.89.gate_proj.weight": "model-00009-of-00101.safetensors", + "model.layers.9.mlp.experts.89.up_proj.weight": "model-00009-of-00101.safetensors", + "model.layers.9.mlp.experts.89.down_proj.weight": "model-00009-of-00101.safetensors", + "model.layers.9.mlp.experts.90.gate_proj.weight": "model-00009-of-00101.safetensors", + "model.layers.9.mlp.experts.90.up_proj.weight": "model-00009-of-00101.safetensors", + "model.layers.9.mlp.experts.90.down_proj.weight": "model-00009-of-00101.safetensors", + "model.layers.9.mlp.experts.91.gate_proj.weight": "model-00009-of-00101.safetensors", + "model.layers.9.mlp.experts.91.up_proj.weight": "model-00009-of-00101.safetensors", + "model.layers.9.mlp.experts.91.down_proj.weight": "model-00009-of-00101.safetensors", + "model.layers.9.mlp.experts.92.gate_proj.weight": "model-00009-of-00101.safetensors", + "model.layers.9.mlp.experts.92.up_proj.weight": "model-00009-of-00101.safetensors", + "model.layers.9.mlp.experts.92.down_proj.weight": "model-00009-of-00101.safetensors", + "model.layers.9.mlp.experts.93.gate_proj.weight": "model-00009-of-00101.safetensors", + "model.layers.9.mlp.experts.93.up_proj.weight": "model-00009-of-00101.safetensors", + "model.layers.9.mlp.experts.93.down_proj.weight": "model-00009-of-00101.safetensors", + "model.layers.9.mlp.experts.94.gate_proj.weight": "model-00009-of-00101.safetensors", + "model.layers.9.mlp.experts.94.up_proj.weight": "model-00009-of-00101.safetensors", + "model.layers.9.mlp.experts.94.down_proj.weight": "model-00009-of-00101.safetensors", + "model.layers.9.mlp.experts.95.gate_proj.weight": "model-00009-of-00101.safetensors", + "model.layers.9.mlp.experts.95.up_proj.weight": "model-00009-of-00101.safetensors", + "model.layers.9.mlp.experts.95.down_proj.weight": "model-00009-of-00101.safetensors", + "model.layers.9.mlp.experts.96.gate_proj.weight": "model-00009-of-00101.safetensors", + "model.layers.9.mlp.experts.96.up_proj.weight": "model-00009-of-00101.safetensors", + "model.layers.9.mlp.experts.96.down_proj.weight": "model-00009-of-00101.safetensors", + "model.layers.9.mlp.experts.97.gate_proj.weight": "model-00009-of-00101.safetensors", + "model.layers.9.mlp.experts.97.up_proj.weight": "model-00009-of-00101.safetensors", + "model.layers.9.mlp.experts.97.down_proj.weight": "model-00009-of-00101.safetensors", + "model.layers.9.mlp.experts.98.gate_proj.weight": "model-00009-of-00101.safetensors", + "model.layers.9.mlp.experts.98.up_proj.weight": "model-00009-of-00101.safetensors", + "model.layers.9.mlp.experts.98.down_proj.weight": "model-00009-of-00101.safetensors", + "model.layers.9.mlp.experts.99.gate_proj.weight": "model-00009-of-00101.safetensors", + "model.layers.9.mlp.experts.99.up_proj.weight": "model-00009-of-00101.safetensors", + "model.layers.9.mlp.experts.99.down_proj.weight": "model-00009-of-00101.safetensors", + "model.layers.9.mlp.experts.100.gate_proj.weight": "model-00009-of-00101.safetensors", + "model.layers.9.mlp.experts.100.up_proj.weight": "model-00009-of-00101.safetensors", + "model.layers.9.mlp.experts.100.down_proj.weight": "model-00009-of-00101.safetensors", + "model.layers.9.mlp.experts.101.gate_proj.weight": "model-00009-of-00101.safetensors", + "model.layers.9.mlp.experts.101.up_proj.weight": "model-00009-of-00101.safetensors", + "model.layers.9.mlp.experts.101.down_proj.weight": "model-00009-of-00101.safetensors", + "model.layers.9.mlp.experts.102.gate_proj.weight": "model-00009-of-00101.safetensors", + "model.layers.9.mlp.experts.102.up_proj.weight": "model-00009-of-00101.safetensors", + "model.layers.9.mlp.experts.102.down_proj.weight": "model-00009-of-00101.safetensors", + "model.layers.9.mlp.experts.103.gate_proj.weight": "model-00009-of-00101.safetensors", + "model.layers.9.mlp.experts.103.up_proj.weight": "model-00009-of-00101.safetensors", + "model.layers.9.mlp.experts.103.down_proj.weight": "model-00009-of-00101.safetensors", + "model.layers.9.mlp.experts.104.gate_proj.weight": "model-00009-of-00101.safetensors", + "model.layers.9.mlp.experts.104.up_proj.weight": "model-00009-of-00101.safetensors", + "model.layers.9.mlp.experts.104.down_proj.weight": "model-00009-of-00101.safetensors", + "model.layers.9.mlp.experts.105.gate_proj.weight": "model-00009-of-00101.safetensors", + "model.layers.9.mlp.experts.105.up_proj.weight": "model-00009-of-00101.safetensors", + "model.layers.9.mlp.experts.105.down_proj.weight": "model-00009-of-00101.safetensors", + "model.layers.9.mlp.experts.106.gate_proj.weight": "model-00009-of-00101.safetensors", + "model.layers.9.mlp.experts.106.up_proj.weight": "model-00009-of-00101.safetensors", + "model.layers.9.mlp.experts.106.down_proj.weight": "model-00009-of-00101.safetensors", + "model.layers.9.mlp.experts.107.gate_proj.weight": "model-00009-of-00101.safetensors", + "model.layers.9.mlp.experts.107.up_proj.weight": "model-00009-of-00101.safetensors", + "model.layers.9.mlp.experts.107.down_proj.weight": "model-00009-of-00101.safetensors", + "model.layers.9.mlp.experts.108.gate_proj.weight": "model-00009-of-00101.safetensors", + "model.layers.9.mlp.experts.108.up_proj.weight": "model-00009-of-00101.safetensors", + "model.layers.9.mlp.experts.108.down_proj.weight": "model-00009-of-00101.safetensors", + "model.layers.9.mlp.experts.109.gate_proj.weight": "model-00009-of-00101.safetensors", + "model.layers.9.mlp.experts.109.up_proj.weight": "model-00009-of-00101.safetensors", + "model.layers.9.mlp.experts.109.down_proj.weight": "model-00009-of-00101.safetensors", + "model.layers.9.mlp.experts.110.gate_proj.weight": "model-00009-of-00101.safetensors", + "model.layers.9.mlp.experts.110.up_proj.weight": "model-00009-of-00101.safetensors", + "model.layers.9.mlp.experts.110.down_proj.weight": "model-00009-of-00101.safetensors", + "model.layers.9.mlp.experts.111.gate_proj.weight": "model-00009-of-00101.safetensors", + "model.layers.9.mlp.experts.111.up_proj.weight": "model-00009-of-00101.safetensors", + "model.layers.9.mlp.experts.111.down_proj.weight": "model-00009-of-00101.safetensors", + "model.layers.9.mlp.experts.112.gate_proj.weight": "model-00009-of-00101.safetensors", + "model.layers.9.mlp.experts.112.up_proj.weight": "model-00009-of-00101.safetensors", + "model.layers.9.mlp.experts.112.down_proj.weight": "model-00009-of-00101.safetensors", + "model.layers.9.mlp.experts.113.gate_proj.weight": "model-00009-of-00101.safetensors", + "model.layers.9.mlp.experts.113.up_proj.weight": "model-00009-of-00101.safetensors", + "model.layers.9.mlp.experts.113.down_proj.weight": "model-00009-of-00101.safetensors", + "model.layers.9.mlp.experts.114.gate_proj.weight": "model-00009-of-00101.safetensors", + "model.layers.9.mlp.experts.114.up_proj.weight": "model-00009-of-00101.safetensors", + "model.layers.9.mlp.experts.114.down_proj.weight": "model-00009-of-00101.safetensors", + "model.layers.9.mlp.experts.115.gate_proj.weight": "model-00009-of-00101.safetensors", + "model.layers.9.mlp.experts.115.up_proj.weight": "model-00009-of-00101.safetensors", + "model.layers.9.mlp.experts.115.down_proj.weight": "model-00009-of-00101.safetensors", + "model.layers.9.mlp.experts.116.gate_proj.weight": "model-00009-of-00101.safetensors", + "model.layers.9.mlp.experts.116.up_proj.weight": "model-00009-of-00101.safetensors", + "model.layers.9.mlp.experts.116.down_proj.weight": "model-00009-of-00101.safetensors", + "model.layers.9.mlp.experts.117.gate_proj.weight": "model-00009-of-00101.safetensors", + "model.layers.9.mlp.experts.117.up_proj.weight": "model-00009-of-00101.safetensors", + "model.layers.9.mlp.experts.117.down_proj.weight": "model-00009-of-00101.safetensors", + "model.layers.9.mlp.experts.118.gate_proj.weight": "model-00009-of-00101.safetensors", + "model.layers.9.mlp.experts.118.up_proj.weight": "model-00009-of-00101.safetensors", + "model.layers.9.mlp.experts.118.down_proj.weight": "model-00009-of-00101.safetensors", + "model.layers.9.mlp.experts.119.gate_proj.weight": "model-00009-of-00101.safetensors", + "model.layers.9.mlp.experts.119.up_proj.weight": "model-00009-of-00101.safetensors", + "model.layers.9.mlp.experts.119.down_proj.weight": "model-00009-of-00101.safetensors", + "model.layers.9.mlp.gate.weight": "model-00009-of-00101.safetensors", + "model.layers.9.mlp.gate.e_score_correction_bias": "model-00009-of-00101.safetensors", + "model.layers.9.mlp.shared_experts.gate_proj.weight": "model-00009-of-00101.safetensors", + "model.layers.9.mlp.shared_experts.up_proj.weight": "model-00009-of-00101.safetensors", + "model.layers.9.mlp.shared_experts.down_proj.weight": "model-00009-of-00101.safetensors", + "model.layers.9.input_layernorm.weight": "model-00009-of-00101.safetensors", + "model.layers.9.post_attention_layernorm.weight": "model-00009-of-00101.safetensors", + "model.layers.10.self_attn.q_proj.weight": "model-00009-of-00101.safetensors", + "model.layers.10.self_attn.q_proj.bias": "model-00009-of-00101.safetensors", + "model.layers.10.self_attn.k_proj.weight": "model-00009-of-00101.safetensors", + "model.layers.10.self_attn.k_proj.bias": "model-00009-of-00101.safetensors", + "model.layers.10.self_attn.v_proj.weight": "model-00009-of-00101.safetensors", + "model.layers.10.self_attn.v_proj.bias": "model-00009-of-00101.safetensors", + "model.layers.10.self_attn.o_proj.weight": "model-00009-of-00101.safetensors", + "model.layers.10.self_attn.q_norm.weight": "model-00009-of-00101.safetensors", + "model.layers.10.self_attn.k_norm.weight": "model-00009-of-00101.safetensors", + "model.layers.10.mlp.experts.0.gate_proj.weight": "model-00009-of-00101.safetensors", + "model.layers.10.mlp.experts.0.up_proj.weight": "model-00009-of-00101.safetensors", + "model.layers.10.mlp.experts.0.down_proj.weight": "model-00009-of-00101.safetensors", + "model.layers.10.mlp.experts.1.gate_proj.weight": "model-00009-of-00101.safetensors", + "model.layers.10.mlp.experts.1.up_proj.weight": "model-00009-of-00101.safetensors", + "model.layers.10.mlp.experts.1.down_proj.weight": "model-00009-of-00101.safetensors", + "model.layers.10.mlp.experts.2.gate_proj.weight": "model-00009-of-00101.safetensors", + "model.layers.10.mlp.experts.2.up_proj.weight": "model-00009-of-00101.safetensors", + "model.layers.10.mlp.experts.2.down_proj.weight": "model-00009-of-00101.safetensors", + "model.layers.10.mlp.experts.3.gate_proj.weight": "model-00009-of-00101.safetensors", + "model.layers.10.mlp.experts.3.up_proj.weight": "model-00009-of-00101.safetensors", + "model.layers.10.mlp.experts.3.down_proj.weight": "model-00009-of-00101.safetensors", + "model.layers.10.mlp.experts.4.gate_proj.weight": "model-00009-of-00101.safetensors", + "model.layers.10.mlp.experts.4.up_proj.weight": "model-00009-of-00101.safetensors", + "model.layers.10.mlp.experts.4.down_proj.weight": "model-00009-of-00101.safetensors", + "model.layers.10.mlp.experts.5.gate_proj.weight": "model-00009-of-00101.safetensors", + "model.layers.10.mlp.experts.5.up_proj.weight": "model-00009-of-00101.safetensors", + "model.layers.10.mlp.experts.5.down_proj.weight": "model-00009-of-00101.safetensors", + "model.layers.10.mlp.experts.6.gate_proj.weight": "model-00009-of-00101.safetensors", + "model.layers.10.mlp.experts.6.up_proj.weight": "model-00009-of-00101.safetensors", + "model.layers.10.mlp.experts.6.down_proj.weight": "model-00009-of-00101.safetensors", + "model.layers.10.mlp.experts.7.gate_proj.weight": "model-00009-of-00101.safetensors", + "model.layers.10.mlp.experts.7.up_proj.weight": "model-00009-of-00101.safetensors", + "model.layers.10.mlp.experts.7.down_proj.weight": "model-00009-of-00101.safetensors", + "model.layers.10.mlp.experts.8.gate_proj.weight": "model-00009-of-00101.safetensors", + "model.layers.10.mlp.experts.8.up_proj.weight": "model-00009-of-00101.safetensors", + "model.layers.10.mlp.experts.8.down_proj.weight": "model-00009-of-00101.safetensors", + "model.layers.10.mlp.experts.9.gate_proj.weight": "model-00009-of-00101.safetensors", + "model.layers.10.mlp.experts.9.up_proj.weight": "model-00009-of-00101.safetensors", + "model.layers.10.mlp.experts.9.down_proj.weight": "model-00009-of-00101.safetensors", + "model.layers.10.mlp.experts.10.gate_proj.weight": "model-00009-of-00101.safetensors", + "model.layers.10.mlp.experts.10.up_proj.weight": "model-00009-of-00101.safetensors", + "model.layers.10.mlp.experts.10.down_proj.weight": "model-00009-of-00101.safetensors", + "model.layers.10.mlp.experts.11.gate_proj.weight": "model-00009-of-00101.safetensors", + "model.layers.10.mlp.experts.11.up_proj.weight": "model-00009-of-00101.safetensors", + "model.layers.10.mlp.experts.11.down_proj.weight": "model-00009-of-00101.safetensors", + "model.layers.10.mlp.experts.12.gate_proj.weight": "model-00009-of-00101.safetensors", + "model.layers.10.mlp.experts.12.up_proj.weight": "model-00009-of-00101.safetensors", + "model.layers.10.mlp.experts.12.down_proj.weight": "model-00009-of-00101.safetensors", + "model.layers.10.mlp.experts.13.gate_proj.weight": "model-00009-of-00101.safetensors", + "model.layers.10.mlp.experts.13.up_proj.weight": "model-00009-of-00101.safetensors", + "model.layers.10.mlp.experts.13.down_proj.weight": "model-00009-of-00101.safetensors", + "model.layers.10.mlp.experts.14.gate_proj.weight": "model-00009-of-00101.safetensors", + "model.layers.10.mlp.experts.14.up_proj.weight": "model-00009-of-00101.safetensors", + "model.layers.10.mlp.experts.14.down_proj.weight": "model-00009-of-00101.safetensors", + "model.layers.10.mlp.experts.15.gate_proj.weight": "model-00009-of-00101.safetensors", + "model.layers.10.mlp.experts.15.up_proj.weight": "model-00009-of-00101.safetensors", + "model.layers.10.mlp.experts.15.down_proj.weight": "model-00009-of-00101.safetensors", + "model.layers.10.mlp.experts.16.gate_proj.weight": "model-00009-of-00101.safetensors", + "model.layers.10.mlp.experts.16.up_proj.weight": "model-00009-of-00101.safetensors", + "model.layers.10.mlp.experts.16.down_proj.weight": "model-00009-of-00101.safetensors", + "model.layers.10.mlp.experts.17.gate_proj.weight": "model-00009-of-00101.safetensors", + "model.layers.10.mlp.experts.17.up_proj.weight": "model-00009-of-00101.safetensors", + "model.layers.10.mlp.experts.17.down_proj.weight": "model-00009-of-00101.safetensors", + "model.layers.10.mlp.experts.18.gate_proj.weight": "model-00009-of-00101.safetensors", + "model.layers.10.mlp.experts.18.up_proj.weight": "model-00009-of-00101.safetensors", + "model.layers.10.mlp.experts.18.down_proj.weight": "model-00009-of-00101.safetensors", + "model.layers.10.mlp.experts.19.gate_proj.weight": "model-00009-of-00101.safetensors", + "model.layers.10.mlp.experts.19.up_proj.weight": "model-00009-of-00101.safetensors", + "model.layers.10.mlp.experts.19.down_proj.weight": "model-00009-of-00101.safetensors", + "model.layers.10.mlp.experts.20.gate_proj.weight": "model-00009-of-00101.safetensors", + "model.layers.10.mlp.experts.20.up_proj.weight": "model-00009-of-00101.safetensors", + "model.layers.10.mlp.experts.20.down_proj.weight": "model-00009-of-00101.safetensors", + "model.layers.10.mlp.experts.21.gate_proj.weight": "model-00009-of-00101.safetensors", + "model.layers.10.mlp.experts.21.up_proj.weight": "model-00009-of-00101.safetensors", + "model.layers.10.mlp.experts.21.down_proj.weight": "model-00009-of-00101.safetensors", + "model.layers.10.mlp.experts.22.gate_proj.weight": "model-00009-of-00101.safetensors", + "model.layers.10.mlp.experts.22.up_proj.weight": "model-00009-of-00101.safetensors", + "model.layers.10.mlp.experts.22.down_proj.weight": "model-00009-of-00101.safetensors", + "model.layers.10.mlp.experts.23.gate_proj.weight": "model-00009-of-00101.safetensors", + "model.layers.10.mlp.experts.23.up_proj.weight": "model-00009-of-00101.safetensors", + "model.layers.10.mlp.experts.23.down_proj.weight": "model-00009-of-00101.safetensors", + "model.layers.10.mlp.experts.24.gate_proj.weight": "model-00009-of-00101.safetensors", + "model.layers.10.mlp.experts.24.up_proj.weight": "model-00009-of-00101.safetensors", + "model.layers.10.mlp.experts.24.down_proj.weight": "model-00009-of-00101.safetensors", + "model.layers.10.mlp.experts.25.gate_proj.weight": "model-00009-of-00101.safetensors", + "model.layers.10.mlp.experts.25.up_proj.weight": "model-00009-of-00101.safetensors", + "model.layers.10.mlp.experts.25.down_proj.weight": "model-00009-of-00101.safetensors", + "model.layers.10.mlp.experts.26.gate_proj.weight": "model-00009-of-00101.safetensors", + "model.layers.10.mlp.experts.26.up_proj.weight": "model-00009-of-00101.safetensors", + "model.layers.10.mlp.experts.26.down_proj.weight": "model-00009-of-00101.safetensors", + "model.layers.10.mlp.experts.27.gate_proj.weight": "model-00009-of-00101.safetensors", + "model.layers.10.mlp.experts.27.up_proj.weight": "model-00009-of-00101.safetensors", + "model.layers.10.mlp.experts.27.down_proj.weight": "model-00009-of-00101.safetensors", + "model.layers.10.mlp.experts.28.gate_proj.weight": "model-00009-of-00101.safetensors", + "model.layers.10.mlp.experts.28.up_proj.weight": "model-00009-of-00101.safetensors", + "model.layers.10.mlp.experts.28.down_proj.weight": "model-00009-of-00101.safetensors", + "model.layers.10.mlp.experts.29.gate_proj.weight": "model-00009-of-00101.safetensors", + "model.layers.10.mlp.experts.29.up_proj.weight": "model-00009-of-00101.safetensors", + "model.layers.10.mlp.experts.29.down_proj.weight": "model-00009-of-00101.safetensors", + "model.layers.10.mlp.experts.30.gate_proj.weight": "model-00009-of-00101.safetensors", + "model.layers.10.mlp.experts.30.up_proj.weight": "model-00009-of-00101.safetensors", + "model.layers.10.mlp.experts.30.down_proj.weight": "model-00009-of-00101.safetensors", + "model.layers.10.mlp.experts.31.gate_proj.weight": "model-00009-of-00101.safetensors", + "model.layers.10.mlp.experts.31.up_proj.weight": "model-00009-of-00101.safetensors", + "model.layers.10.mlp.experts.31.down_proj.weight": "model-00009-of-00101.safetensors", + "model.layers.10.mlp.experts.32.gate_proj.weight": "model-00009-of-00101.safetensors", + "model.layers.10.mlp.experts.32.up_proj.weight": "model-00009-of-00101.safetensors", + "model.layers.10.mlp.experts.32.down_proj.weight": "model-00009-of-00101.safetensors", + "model.layers.10.mlp.experts.33.gate_proj.weight": "model-00009-of-00101.safetensors", + "model.layers.10.mlp.experts.33.up_proj.weight": "model-00009-of-00101.safetensors", + "model.layers.10.mlp.experts.33.down_proj.weight": "model-00009-of-00101.safetensors", + "model.layers.10.mlp.experts.34.gate_proj.weight": "model-00009-of-00101.safetensors", + "model.layers.10.mlp.experts.34.up_proj.weight": "model-00009-of-00101.safetensors", + "model.layers.10.mlp.experts.34.down_proj.weight": "model-00009-of-00101.safetensors", + "model.layers.10.mlp.experts.35.gate_proj.weight": "model-00009-of-00101.safetensors", + "model.layers.10.mlp.experts.35.up_proj.weight": "model-00009-of-00101.safetensors", + "model.layers.10.mlp.experts.35.down_proj.weight": "model-00009-of-00101.safetensors", + "model.layers.10.mlp.experts.36.gate_proj.weight": "model-00009-of-00101.safetensors", + "model.layers.10.mlp.experts.36.up_proj.weight": "model-00009-of-00101.safetensors", + "model.layers.10.mlp.experts.36.down_proj.weight": "model-00009-of-00101.safetensors", + "model.layers.10.mlp.experts.37.gate_proj.weight": "model-00009-of-00101.safetensors", + "model.layers.10.mlp.experts.37.up_proj.weight": "model-00009-of-00101.safetensors", + "model.layers.10.mlp.experts.37.down_proj.weight": "model-00009-of-00101.safetensors", + "model.layers.10.mlp.experts.38.gate_proj.weight": "model-00009-of-00101.safetensors", + "model.layers.10.mlp.experts.38.up_proj.weight": "model-00009-of-00101.safetensors", + "model.layers.10.mlp.experts.38.down_proj.weight": "model-00009-of-00101.safetensors", + "model.layers.10.mlp.experts.39.gate_proj.weight": "model-00009-of-00101.safetensors", + "model.layers.10.mlp.experts.39.up_proj.weight": "model-00009-of-00101.safetensors", + "model.layers.10.mlp.experts.39.down_proj.weight": "model-00009-of-00101.safetensors", + "model.layers.10.mlp.experts.40.gate_proj.weight": "model-00009-of-00101.safetensors", + "model.layers.10.mlp.experts.40.up_proj.weight": "model-00009-of-00101.safetensors", + "model.layers.10.mlp.experts.40.down_proj.weight": "model-00009-of-00101.safetensors", + "model.layers.10.mlp.experts.41.gate_proj.weight": "model-00009-of-00101.safetensors", + "model.layers.10.mlp.experts.41.up_proj.weight": "model-00009-of-00101.safetensors", + "model.layers.10.mlp.experts.41.down_proj.weight": "model-00009-of-00101.safetensors", + "model.layers.10.mlp.experts.42.gate_proj.weight": "model-00009-of-00101.safetensors", + "model.layers.10.mlp.experts.42.up_proj.weight": "model-00009-of-00101.safetensors", + "model.layers.10.mlp.experts.42.down_proj.weight": "model-00009-of-00101.safetensors", + "model.layers.10.mlp.experts.43.gate_proj.weight": "model-00009-of-00101.safetensors", + "model.layers.10.mlp.experts.43.up_proj.weight": "model-00009-of-00101.safetensors", + "model.layers.10.mlp.experts.43.down_proj.weight": "model-00009-of-00101.safetensors", + "model.layers.10.mlp.experts.44.gate_proj.weight": "model-00009-of-00101.safetensors", + "model.layers.10.mlp.experts.44.up_proj.weight": "model-00009-of-00101.safetensors", + "model.layers.10.mlp.experts.44.down_proj.weight": "model-00009-of-00101.safetensors", + "model.layers.10.mlp.experts.45.gate_proj.weight": "model-00009-of-00101.safetensors", + "model.layers.10.mlp.experts.45.up_proj.weight": "model-00009-of-00101.safetensors", + "model.layers.10.mlp.experts.45.down_proj.weight": "model-00009-of-00101.safetensors", + "model.layers.10.mlp.experts.46.gate_proj.weight": "model-00009-of-00101.safetensors", + "model.layers.10.mlp.experts.46.up_proj.weight": "model-00009-of-00101.safetensors", + "model.layers.10.mlp.experts.46.down_proj.weight": "model-00009-of-00101.safetensors", + "model.layers.10.mlp.experts.47.gate_proj.weight": "model-00009-of-00101.safetensors", + "model.layers.10.mlp.experts.47.up_proj.weight": "model-00009-of-00101.safetensors", + "model.layers.10.mlp.experts.47.down_proj.weight": "model-00009-of-00101.safetensors", + "model.layers.10.mlp.experts.48.gate_proj.weight": "model-00009-of-00101.safetensors", + "model.layers.10.mlp.experts.48.up_proj.weight": "model-00009-of-00101.safetensors", + "model.layers.10.mlp.experts.48.down_proj.weight": "model-00009-of-00101.safetensors", + "model.layers.10.mlp.experts.49.gate_proj.weight": "model-00009-of-00101.safetensors", + "model.layers.10.mlp.experts.49.up_proj.weight": "model-00009-of-00101.safetensors", + "model.layers.10.mlp.experts.49.down_proj.weight": "model-00009-of-00101.safetensors", + "model.layers.10.mlp.experts.50.gate_proj.weight": "model-00009-of-00101.safetensors", + "model.layers.10.mlp.experts.50.up_proj.weight": "model-00009-of-00101.safetensors", + "model.layers.10.mlp.experts.50.down_proj.weight": "model-00009-of-00101.safetensors", + "model.layers.10.mlp.experts.51.gate_proj.weight": "model-00009-of-00101.safetensors", + "model.layers.10.mlp.experts.51.up_proj.weight": "model-00009-of-00101.safetensors", + "model.layers.10.mlp.experts.51.down_proj.weight": "model-00009-of-00101.safetensors", + "model.layers.10.mlp.experts.52.gate_proj.weight": "model-00009-of-00101.safetensors", + "model.layers.10.mlp.experts.52.up_proj.weight": "model-00009-of-00101.safetensors", + "model.layers.10.mlp.experts.52.down_proj.weight": "model-00009-of-00101.safetensors", + "model.layers.10.mlp.experts.53.gate_proj.weight": "model-00009-of-00101.safetensors", + "model.layers.10.mlp.experts.53.up_proj.weight": "model-00009-of-00101.safetensors", + "model.layers.10.mlp.experts.53.down_proj.weight": "model-00009-of-00101.safetensors", + "model.layers.10.mlp.experts.54.gate_proj.weight": "model-00010-of-00101.safetensors", + "model.layers.10.mlp.experts.54.up_proj.weight": "model-00010-of-00101.safetensors", + "model.layers.10.mlp.experts.54.down_proj.weight": "model-00010-of-00101.safetensors", + "model.layers.10.mlp.experts.55.gate_proj.weight": "model-00010-of-00101.safetensors", + "model.layers.10.mlp.experts.55.up_proj.weight": "model-00010-of-00101.safetensors", + "model.layers.10.mlp.experts.55.down_proj.weight": "model-00010-of-00101.safetensors", + "model.layers.10.mlp.experts.56.gate_proj.weight": "model-00010-of-00101.safetensors", + "model.layers.10.mlp.experts.56.up_proj.weight": "model-00010-of-00101.safetensors", + "model.layers.10.mlp.experts.56.down_proj.weight": "model-00010-of-00101.safetensors", + "model.layers.10.mlp.experts.57.gate_proj.weight": "model-00010-of-00101.safetensors", + "model.layers.10.mlp.experts.57.up_proj.weight": "model-00010-of-00101.safetensors", + "model.layers.10.mlp.experts.57.down_proj.weight": "model-00010-of-00101.safetensors", + "model.layers.10.mlp.experts.58.gate_proj.weight": "model-00010-of-00101.safetensors", + "model.layers.10.mlp.experts.58.up_proj.weight": "model-00010-of-00101.safetensors", + "model.layers.10.mlp.experts.58.down_proj.weight": "model-00010-of-00101.safetensors", + "model.layers.10.mlp.experts.59.gate_proj.weight": "model-00010-of-00101.safetensors", + "model.layers.10.mlp.experts.59.up_proj.weight": "model-00010-of-00101.safetensors", + "model.layers.10.mlp.experts.59.down_proj.weight": "model-00010-of-00101.safetensors", + "model.layers.10.mlp.experts.60.gate_proj.weight": "model-00010-of-00101.safetensors", + "model.layers.10.mlp.experts.60.up_proj.weight": "model-00010-of-00101.safetensors", + "model.layers.10.mlp.experts.60.down_proj.weight": "model-00010-of-00101.safetensors", + "model.layers.10.mlp.experts.61.gate_proj.weight": "model-00010-of-00101.safetensors", + "model.layers.10.mlp.experts.61.up_proj.weight": "model-00010-of-00101.safetensors", + "model.layers.10.mlp.experts.61.down_proj.weight": "model-00010-of-00101.safetensors", + "model.layers.10.mlp.experts.62.gate_proj.weight": "model-00010-of-00101.safetensors", + "model.layers.10.mlp.experts.62.up_proj.weight": "model-00010-of-00101.safetensors", + "model.layers.10.mlp.experts.62.down_proj.weight": "model-00010-of-00101.safetensors", + "model.layers.10.mlp.experts.63.gate_proj.weight": "model-00010-of-00101.safetensors", + "model.layers.10.mlp.experts.63.up_proj.weight": "model-00010-of-00101.safetensors", + "model.layers.10.mlp.experts.63.down_proj.weight": "model-00010-of-00101.safetensors", + "model.layers.10.mlp.experts.64.gate_proj.weight": "model-00010-of-00101.safetensors", + "model.layers.10.mlp.experts.64.up_proj.weight": "model-00010-of-00101.safetensors", + "model.layers.10.mlp.experts.64.down_proj.weight": "model-00010-of-00101.safetensors", + "model.layers.10.mlp.experts.65.gate_proj.weight": "model-00010-of-00101.safetensors", + "model.layers.10.mlp.experts.65.up_proj.weight": "model-00010-of-00101.safetensors", + "model.layers.10.mlp.experts.65.down_proj.weight": "model-00010-of-00101.safetensors", + "model.layers.10.mlp.experts.66.gate_proj.weight": "model-00010-of-00101.safetensors", + "model.layers.10.mlp.experts.66.up_proj.weight": "model-00010-of-00101.safetensors", + "model.layers.10.mlp.experts.66.down_proj.weight": "model-00010-of-00101.safetensors", + "model.layers.10.mlp.experts.67.gate_proj.weight": "model-00010-of-00101.safetensors", + "model.layers.10.mlp.experts.67.up_proj.weight": "model-00010-of-00101.safetensors", + "model.layers.10.mlp.experts.67.down_proj.weight": "model-00010-of-00101.safetensors", + "model.layers.10.mlp.experts.68.gate_proj.weight": "model-00010-of-00101.safetensors", + "model.layers.10.mlp.experts.68.up_proj.weight": "model-00010-of-00101.safetensors", + "model.layers.10.mlp.experts.68.down_proj.weight": "model-00010-of-00101.safetensors", + "model.layers.10.mlp.experts.69.gate_proj.weight": "model-00010-of-00101.safetensors", + "model.layers.10.mlp.experts.69.up_proj.weight": "model-00010-of-00101.safetensors", + "model.layers.10.mlp.experts.69.down_proj.weight": "model-00010-of-00101.safetensors", + "model.layers.10.mlp.experts.70.gate_proj.weight": "model-00010-of-00101.safetensors", + "model.layers.10.mlp.experts.70.up_proj.weight": "model-00010-of-00101.safetensors", + "model.layers.10.mlp.experts.70.down_proj.weight": "model-00010-of-00101.safetensors", + "model.layers.10.mlp.experts.71.gate_proj.weight": "model-00010-of-00101.safetensors", + "model.layers.10.mlp.experts.71.up_proj.weight": "model-00010-of-00101.safetensors", + "model.layers.10.mlp.experts.71.down_proj.weight": "model-00010-of-00101.safetensors", + "model.layers.10.mlp.experts.72.gate_proj.weight": "model-00010-of-00101.safetensors", + "model.layers.10.mlp.experts.72.up_proj.weight": "model-00010-of-00101.safetensors", + "model.layers.10.mlp.experts.72.down_proj.weight": "model-00010-of-00101.safetensors", + "model.layers.10.mlp.experts.73.gate_proj.weight": "model-00010-of-00101.safetensors", + "model.layers.10.mlp.experts.73.up_proj.weight": "model-00010-of-00101.safetensors", + "model.layers.10.mlp.experts.73.down_proj.weight": "model-00010-of-00101.safetensors", + "model.layers.10.mlp.experts.74.gate_proj.weight": "model-00010-of-00101.safetensors", + "model.layers.10.mlp.experts.74.up_proj.weight": "model-00010-of-00101.safetensors", + "model.layers.10.mlp.experts.74.down_proj.weight": "model-00010-of-00101.safetensors", + "model.layers.10.mlp.experts.75.gate_proj.weight": "model-00010-of-00101.safetensors", + "model.layers.10.mlp.experts.75.up_proj.weight": "model-00010-of-00101.safetensors", + "model.layers.10.mlp.experts.75.down_proj.weight": "model-00010-of-00101.safetensors", + "model.layers.10.mlp.experts.76.gate_proj.weight": "model-00010-of-00101.safetensors", + "model.layers.10.mlp.experts.76.up_proj.weight": "model-00010-of-00101.safetensors", + "model.layers.10.mlp.experts.76.down_proj.weight": "model-00010-of-00101.safetensors", + "model.layers.10.mlp.experts.77.gate_proj.weight": "model-00010-of-00101.safetensors", + "model.layers.10.mlp.experts.77.up_proj.weight": "model-00010-of-00101.safetensors", + "model.layers.10.mlp.experts.77.down_proj.weight": "model-00010-of-00101.safetensors", + "model.layers.10.mlp.experts.78.gate_proj.weight": "model-00010-of-00101.safetensors", + "model.layers.10.mlp.experts.78.up_proj.weight": "model-00010-of-00101.safetensors", + "model.layers.10.mlp.experts.78.down_proj.weight": "model-00010-of-00101.safetensors", + "model.layers.10.mlp.experts.79.gate_proj.weight": "model-00010-of-00101.safetensors", + "model.layers.10.mlp.experts.79.up_proj.weight": "model-00010-of-00101.safetensors", + "model.layers.10.mlp.experts.79.down_proj.weight": "model-00010-of-00101.safetensors", + "model.layers.10.mlp.experts.80.gate_proj.weight": "model-00010-of-00101.safetensors", + "model.layers.10.mlp.experts.80.up_proj.weight": "model-00010-of-00101.safetensors", + "model.layers.10.mlp.experts.80.down_proj.weight": "model-00010-of-00101.safetensors", + "model.layers.10.mlp.experts.81.gate_proj.weight": "model-00010-of-00101.safetensors", + "model.layers.10.mlp.experts.81.up_proj.weight": "model-00010-of-00101.safetensors", + "model.layers.10.mlp.experts.81.down_proj.weight": "model-00010-of-00101.safetensors", + "model.layers.10.mlp.experts.82.gate_proj.weight": "model-00010-of-00101.safetensors", + "model.layers.10.mlp.experts.82.up_proj.weight": "model-00010-of-00101.safetensors", + "model.layers.10.mlp.experts.82.down_proj.weight": "model-00010-of-00101.safetensors", + "model.layers.10.mlp.experts.83.gate_proj.weight": "model-00010-of-00101.safetensors", + "model.layers.10.mlp.experts.83.up_proj.weight": "model-00010-of-00101.safetensors", + "model.layers.10.mlp.experts.83.down_proj.weight": "model-00010-of-00101.safetensors", + "model.layers.10.mlp.experts.84.gate_proj.weight": "model-00010-of-00101.safetensors", + "model.layers.10.mlp.experts.84.up_proj.weight": "model-00010-of-00101.safetensors", + "model.layers.10.mlp.experts.84.down_proj.weight": "model-00010-of-00101.safetensors", + "model.layers.10.mlp.experts.85.gate_proj.weight": "model-00010-of-00101.safetensors", + "model.layers.10.mlp.experts.85.up_proj.weight": "model-00010-of-00101.safetensors", + "model.layers.10.mlp.experts.85.down_proj.weight": "model-00010-of-00101.safetensors", + "model.layers.10.mlp.experts.86.gate_proj.weight": "model-00010-of-00101.safetensors", + "model.layers.10.mlp.experts.86.up_proj.weight": "model-00010-of-00101.safetensors", + "model.layers.10.mlp.experts.86.down_proj.weight": "model-00010-of-00101.safetensors", + "model.layers.10.mlp.experts.87.gate_proj.weight": "model-00010-of-00101.safetensors", + "model.layers.10.mlp.experts.87.up_proj.weight": "model-00010-of-00101.safetensors", + "model.layers.10.mlp.experts.87.down_proj.weight": "model-00010-of-00101.safetensors", + "model.layers.10.mlp.experts.88.gate_proj.weight": "model-00010-of-00101.safetensors", + "model.layers.10.mlp.experts.88.up_proj.weight": "model-00010-of-00101.safetensors", + "model.layers.10.mlp.experts.88.down_proj.weight": "model-00010-of-00101.safetensors", + "model.layers.10.mlp.experts.89.gate_proj.weight": "model-00010-of-00101.safetensors", + "model.layers.10.mlp.experts.89.up_proj.weight": "model-00010-of-00101.safetensors", + "model.layers.10.mlp.experts.89.down_proj.weight": "model-00010-of-00101.safetensors", + "model.layers.10.mlp.experts.90.gate_proj.weight": "model-00010-of-00101.safetensors", + "model.layers.10.mlp.experts.90.up_proj.weight": "model-00010-of-00101.safetensors", + "model.layers.10.mlp.experts.90.down_proj.weight": "model-00010-of-00101.safetensors", + "model.layers.10.mlp.experts.91.gate_proj.weight": "model-00010-of-00101.safetensors", + "model.layers.10.mlp.experts.91.up_proj.weight": "model-00010-of-00101.safetensors", + "model.layers.10.mlp.experts.91.down_proj.weight": "model-00010-of-00101.safetensors", + "model.layers.10.mlp.experts.92.gate_proj.weight": "model-00010-of-00101.safetensors", + "model.layers.10.mlp.experts.92.up_proj.weight": "model-00010-of-00101.safetensors", + "model.layers.10.mlp.experts.92.down_proj.weight": "model-00010-of-00101.safetensors", + "model.layers.10.mlp.experts.93.gate_proj.weight": "model-00010-of-00101.safetensors", + "model.layers.10.mlp.experts.93.up_proj.weight": "model-00010-of-00101.safetensors", + "model.layers.10.mlp.experts.93.down_proj.weight": "model-00010-of-00101.safetensors", + "model.layers.10.mlp.experts.94.gate_proj.weight": "model-00010-of-00101.safetensors", + "model.layers.10.mlp.experts.94.up_proj.weight": "model-00010-of-00101.safetensors", + "model.layers.10.mlp.experts.94.down_proj.weight": "model-00010-of-00101.safetensors", + "model.layers.10.mlp.experts.95.gate_proj.weight": "model-00010-of-00101.safetensors", + "model.layers.10.mlp.experts.95.up_proj.weight": "model-00010-of-00101.safetensors", + "model.layers.10.mlp.experts.95.down_proj.weight": "model-00010-of-00101.safetensors", + "model.layers.10.mlp.experts.96.gate_proj.weight": "model-00010-of-00101.safetensors", + "model.layers.10.mlp.experts.96.up_proj.weight": "model-00010-of-00101.safetensors", + "model.layers.10.mlp.experts.96.down_proj.weight": "model-00010-of-00101.safetensors", + "model.layers.10.mlp.experts.97.gate_proj.weight": "model-00010-of-00101.safetensors", + "model.layers.10.mlp.experts.97.up_proj.weight": "model-00010-of-00101.safetensors", + "model.layers.10.mlp.experts.97.down_proj.weight": "model-00010-of-00101.safetensors", + "model.layers.10.mlp.experts.98.gate_proj.weight": "model-00010-of-00101.safetensors", + "model.layers.10.mlp.experts.98.up_proj.weight": "model-00010-of-00101.safetensors", + "model.layers.10.mlp.experts.98.down_proj.weight": "model-00010-of-00101.safetensors", + "model.layers.10.mlp.experts.99.gate_proj.weight": "model-00010-of-00101.safetensors", + "model.layers.10.mlp.experts.99.up_proj.weight": "model-00010-of-00101.safetensors", + "model.layers.10.mlp.experts.99.down_proj.weight": "model-00010-of-00101.safetensors", + "model.layers.10.mlp.experts.100.gate_proj.weight": "model-00010-of-00101.safetensors", + "model.layers.10.mlp.experts.100.up_proj.weight": "model-00010-of-00101.safetensors", + "model.layers.10.mlp.experts.100.down_proj.weight": "model-00010-of-00101.safetensors", + "model.layers.10.mlp.experts.101.gate_proj.weight": "model-00010-of-00101.safetensors", + "model.layers.10.mlp.experts.101.up_proj.weight": "model-00010-of-00101.safetensors", + "model.layers.10.mlp.experts.101.down_proj.weight": "model-00010-of-00101.safetensors", + "model.layers.10.mlp.experts.102.gate_proj.weight": "model-00010-of-00101.safetensors", + "model.layers.10.mlp.experts.102.up_proj.weight": "model-00010-of-00101.safetensors", + "model.layers.10.mlp.experts.102.down_proj.weight": "model-00010-of-00101.safetensors", + "model.layers.10.mlp.experts.103.gate_proj.weight": "model-00010-of-00101.safetensors", + "model.layers.10.mlp.experts.103.up_proj.weight": "model-00010-of-00101.safetensors", + "model.layers.10.mlp.experts.103.down_proj.weight": "model-00010-of-00101.safetensors", + "model.layers.10.mlp.experts.104.gate_proj.weight": "model-00010-of-00101.safetensors", + "model.layers.10.mlp.experts.104.up_proj.weight": "model-00010-of-00101.safetensors", + "model.layers.10.mlp.experts.104.down_proj.weight": "model-00010-of-00101.safetensors", + "model.layers.10.mlp.experts.105.gate_proj.weight": "model-00010-of-00101.safetensors", + "model.layers.10.mlp.experts.105.up_proj.weight": "model-00010-of-00101.safetensors", + "model.layers.10.mlp.experts.105.down_proj.weight": "model-00010-of-00101.safetensors", + "model.layers.10.mlp.experts.106.gate_proj.weight": "model-00010-of-00101.safetensors", + "model.layers.10.mlp.experts.106.up_proj.weight": "model-00010-of-00101.safetensors", + "model.layers.10.mlp.experts.106.down_proj.weight": "model-00010-of-00101.safetensors", + "model.layers.10.mlp.experts.107.gate_proj.weight": "model-00010-of-00101.safetensors", + "model.layers.10.mlp.experts.107.up_proj.weight": "model-00010-of-00101.safetensors", + "model.layers.10.mlp.experts.107.down_proj.weight": "model-00010-of-00101.safetensors", + "model.layers.10.mlp.experts.108.gate_proj.weight": "model-00010-of-00101.safetensors", + "model.layers.10.mlp.experts.108.up_proj.weight": "model-00010-of-00101.safetensors", + "model.layers.10.mlp.experts.108.down_proj.weight": "model-00010-of-00101.safetensors", + "model.layers.10.mlp.experts.109.gate_proj.weight": "model-00010-of-00101.safetensors", + "model.layers.10.mlp.experts.109.up_proj.weight": "model-00010-of-00101.safetensors", + "model.layers.10.mlp.experts.109.down_proj.weight": "model-00010-of-00101.safetensors", + "model.layers.10.mlp.experts.110.gate_proj.weight": "model-00010-of-00101.safetensors", + "model.layers.10.mlp.experts.110.up_proj.weight": "model-00010-of-00101.safetensors", + "model.layers.10.mlp.experts.110.down_proj.weight": "model-00010-of-00101.safetensors", + "model.layers.10.mlp.experts.111.gate_proj.weight": "model-00010-of-00101.safetensors", + "model.layers.10.mlp.experts.111.up_proj.weight": "model-00010-of-00101.safetensors", + "model.layers.10.mlp.experts.111.down_proj.weight": "model-00010-of-00101.safetensors", + "model.layers.10.mlp.experts.112.gate_proj.weight": "model-00010-of-00101.safetensors", + "model.layers.10.mlp.experts.112.up_proj.weight": "model-00010-of-00101.safetensors", + "model.layers.10.mlp.experts.112.down_proj.weight": "model-00010-of-00101.safetensors", + "model.layers.10.mlp.experts.113.gate_proj.weight": "model-00010-of-00101.safetensors", + "model.layers.10.mlp.experts.113.up_proj.weight": "model-00010-of-00101.safetensors", + "model.layers.10.mlp.experts.113.down_proj.weight": "model-00010-of-00101.safetensors", + "model.layers.10.mlp.experts.114.gate_proj.weight": "model-00010-of-00101.safetensors", + "model.layers.10.mlp.experts.114.up_proj.weight": "model-00010-of-00101.safetensors", + "model.layers.10.mlp.experts.114.down_proj.weight": "model-00010-of-00101.safetensors", + "model.layers.10.mlp.experts.115.gate_proj.weight": "model-00010-of-00101.safetensors", + "model.layers.10.mlp.experts.115.up_proj.weight": "model-00010-of-00101.safetensors", + "model.layers.10.mlp.experts.115.down_proj.weight": "model-00010-of-00101.safetensors", + "model.layers.10.mlp.experts.116.gate_proj.weight": "model-00010-of-00101.safetensors", + "model.layers.10.mlp.experts.116.up_proj.weight": "model-00010-of-00101.safetensors", + "model.layers.10.mlp.experts.116.down_proj.weight": "model-00010-of-00101.safetensors", + "model.layers.10.mlp.experts.117.gate_proj.weight": "model-00010-of-00101.safetensors", + "model.layers.10.mlp.experts.117.up_proj.weight": "model-00010-of-00101.safetensors", + "model.layers.10.mlp.experts.117.down_proj.weight": "model-00010-of-00101.safetensors", + "model.layers.10.mlp.experts.118.gate_proj.weight": "model-00010-of-00101.safetensors", + "model.layers.10.mlp.experts.118.up_proj.weight": "model-00010-of-00101.safetensors", + "model.layers.10.mlp.experts.118.down_proj.weight": "model-00010-of-00101.safetensors", + "model.layers.10.mlp.experts.119.gate_proj.weight": "model-00010-of-00101.safetensors", + "model.layers.10.mlp.experts.119.up_proj.weight": "model-00010-of-00101.safetensors", + "model.layers.10.mlp.experts.119.down_proj.weight": "model-00010-of-00101.safetensors", + "model.layers.10.mlp.gate.weight": "model-00010-of-00101.safetensors", + "model.layers.10.mlp.gate.e_score_correction_bias": "model-00010-of-00101.safetensors", + "model.layers.10.mlp.shared_experts.gate_proj.weight": "model-00010-of-00101.safetensors", + "model.layers.10.mlp.shared_experts.up_proj.weight": "model-00010-of-00101.safetensors", + "model.layers.10.mlp.shared_experts.down_proj.weight": "model-00010-of-00101.safetensors", + "model.layers.10.input_layernorm.weight": "model-00010-of-00101.safetensors", + "model.layers.10.post_attention_layernorm.weight": "model-00010-of-00101.safetensors", + "model.layers.11.self_attn.q_proj.weight": "model-00010-of-00101.safetensors", + "model.layers.11.self_attn.q_proj.bias": "model-00010-of-00101.safetensors", + "model.layers.11.self_attn.k_proj.weight": "model-00010-of-00101.safetensors", + "model.layers.11.self_attn.k_proj.bias": "model-00010-of-00101.safetensors", + "model.layers.11.self_attn.v_proj.weight": "model-00010-of-00101.safetensors", + "model.layers.11.self_attn.v_proj.bias": "model-00010-of-00101.safetensors", + "model.layers.11.self_attn.o_proj.weight": "model-00010-of-00101.safetensors", + "model.layers.11.self_attn.q_norm.weight": "model-00010-of-00101.safetensors", + "model.layers.11.self_attn.k_norm.weight": "model-00010-of-00101.safetensors", + "model.layers.11.mlp.experts.0.gate_proj.weight": "model-00010-of-00101.safetensors", + "model.layers.11.mlp.experts.0.up_proj.weight": "model-00010-of-00101.safetensors", + "model.layers.11.mlp.experts.0.down_proj.weight": "model-00010-of-00101.safetensors", + "model.layers.11.mlp.experts.1.gate_proj.weight": "model-00010-of-00101.safetensors", + "model.layers.11.mlp.experts.1.up_proj.weight": "model-00010-of-00101.safetensors", + "model.layers.11.mlp.experts.1.down_proj.weight": "model-00010-of-00101.safetensors", + "model.layers.11.mlp.experts.2.gate_proj.weight": "model-00010-of-00101.safetensors", + "model.layers.11.mlp.experts.2.up_proj.weight": "model-00010-of-00101.safetensors", + "model.layers.11.mlp.experts.2.down_proj.weight": "model-00010-of-00101.safetensors", + "model.layers.11.mlp.experts.3.gate_proj.weight": "model-00010-of-00101.safetensors", + "model.layers.11.mlp.experts.3.up_proj.weight": "model-00010-of-00101.safetensors", + "model.layers.11.mlp.experts.3.down_proj.weight": "model-00010-of-00101.safetensors", + "model.layers.11.mlp.experts.4.gate_proj.weight": "model-00010-of-00101.safetensors", + "model.layers.11.mlp.experts.4.up_proj.weight": "model-00010-of-00101.safetensors", + "model.layers.11.mlp.experts.4.down_proj.weight": "model-00010-of-00101.safetensors", + "model.layers.11.mlp.experts.5.gate_proj.weight": "model-00010-of-00101.safetensors", + "model.layers.11.mlp.experts.5.up_proj.weight": "model-00010-of-00101.safetensors", + "model.layers.11.mlp.experts.5.down_proj.weight": "model-00010-of-00101.safetensors", + "model.layers.11.mlp.experts.6.gate_proj.weight": "model-00010-of-00101.safetensors", + "model.layers.11.mlp.experts.6.up_proj.weight": "model-00010-of-00101.safetensors", + "model.layers.11.mlp.experts.6.down_proj.weight": "model-00010-of-00101.safetensors", + "model.layers.11.mlp.experts.7.gate_proj.weight": "model-00010-of-00101.safetensors", + "model.layers.11.mlp.experts.7.up_proj.weight": "model-00010-of-00101.safetensors", + "model.layers.11.mlp.experts.7.down_proj.weight": "model-00010-of-00101.safetensors", + "model.layers.11.mlp.experts.8.gate_proj.weight": "model-00010-of-00101.safetensors", + "model.layers.11.mlp.experts.8.up_proj.weight": "model-00010-of-00101.safetensors", + "model.layers.11.mlp.experts.8.down_proj.weight": "model-00010-of-00101.safetensors", + "model.layers.11.mlp.experts.9.gate_proj.weight": "model-00010-of-00101.safetensors", + "model.layers.11.mlp.experts.9.up_proj.weight": "model-00010-of-00101.safetensors", + "model.layers.11.mlp.experts.9.down_proj.weight": "model-00010-of-00101.safetensors", + "model.layers.11.mlp.experts.10.gate_proj.weight": "model-00010-of-00101.safetensors", + "model.layers.11.mlp.experts.10.up_proj.weight": "model-00010-of-00101.safetensors", + "model.layers.11.mlp.experts.10.down_proj.weight": "model-00010-of-00101.safetensors", + "model.layers.11.mlp.experts.11.gate_proj.weight": "model-00010-of-00101.safetensors", + "model.layers.11.mlp.experts.11.up_proj.weight": "model-00010-of-00101.safetensors", + "model.layers.11.mlp.experts.11.down_proj.weight": "model-00010-of-00101.safetensors", + "model.layers.11.mlp.experts.12.gate_proj.weight": "model-00010-of-00101.safetensors", + "model.layers.11.mlp.experts.12.up_proj.weight": "model-00010-of-00101.safetensors", + "model.layers.11.mlp.experts.12.down_proj.weight": "model-00010-of-00101.safetensors", + "model.layers.11.mlp.experts.13.gate_proj.weight": "model-00010-of-00101.safetensors", + "model.layers.11.mlp.experts.13.up_proj.weight": "model-00010-of-00101.safetensors", + "model.layers.11.mlp.experts.13.down_proj.weight": "model-00010-of-00101.safetensors", + "model.layers.11.mlp.experts.14.gate_proj.weight": "model-00010-of-00101.safetensors", + "model.layers.11.mlp.experts.14.up_proj.weight": "model-00010-of-00101.safetensors", + "model.layers.11.mlp.experts.14.down_proj.weight": "model-00010-of-00101.safetensors", + "model.layers.11.mlp.experts.15.gate_proj.weight": "model-00010-of-00101.safetensors", + "model.layers.11.mlp.experts.15.up_proj.weight": "model-00010-of-00101.safetensors", + "model.layers.11.mlp.experts.15.down_proj.weight": "model-00010-of-00101.safetensors", + "model.layers.11.mlp.experts.16.gate_proj.weight": "model-00010-of-00101.safetensors", + "model.layers.11.mlp.experts.16.up_proj.weight": "model-00010-of-00101.safetensors", + "model.layers.11.mlp.experts.16.down_proj.weight": "model-00010-of-00101.safetensors", + "model.layers.11.mlp.experts.17.gate_proj.weight": "model-00010-of-00101.safetensors", + "model.layers.11.mlp.experts.17.up_proj.weight": "model-00010-of-00101.safetensors", + "model.layers.11.mlp.experts.17.down_proj.weight": "model-00010-of-00101.safetensors", + "model.layers.11.mlp.experts.18.gate_proj.weight": "model-00010-of-00101.safetensors", + "model.layers.11.mlp.experts.18.up_proj.weight": "model-00010-of-00101.safetensors", + "model.layers.11.mlp.experts.18.down_proj.weight": "model-00010-of-00101.safetensors", + "model.layers.11.mlp.experts.19.gate_proj.weight": "model-00010-of-00101.safetensors", + "model.layers.11.mlp.experts.19.up_proj.weight": "model-00010-of-00101.safetensors", + "model.layers.11.mlp.experts.19.down_proj.weight": "model-00010-of-00101.safetensors", + "model.layers.11.mlp.experts.20.gate_proj.weight": "model-00010-of-00101.safetensors", + "model.layers.11.mlp.experts.20.up_proj.weight": "model-00010-of-00101.safetensors", + "model.layers.11.mlp.experts.20.down_proj.weight": "model-00010-of-00101.safetensors", + "model.layers.11.mlp.experts.21.gate_proj.weight": "model-00010-of-00101.safetensors", + "model.layers.11.mlp.experts.21.up_proj.weight": "model-00010-of-00101.safetensors", + "model.layers.11.mlp.experts.21.down_proj.weight": "model-00010-of-00101.safetensors", + "model.layers.11.mlp.experts.22.gate_proj.weight": "model-00010-of-00101.safetensors", + "model.layers.11.mlp.experts.22.up_proj.weight": "model-00010-of-00101.safetensors", + "model.layers.11.mlp.experts.22.down_proj.weight": "model-00010-of-00101.safetensors", + "model.layers.11.mlp.experts.23.gate_proj.weight": "model-00010-of-00101.safetensors", + "model.layers.11.mlp.experts.23.up_proj.weight": "model-00010-of-00101.safetensors", + "model.layers.11.mlp.experts.23.down_proj.weight": "model-00010-of-00101.safetensors", + "model.layers.11.mlp.experts.24.gate_proj.weight": "model-00010-of-00101.safetensors", + "model.layers.11.mlp.experts.24.up_proj.weight": "model-00010-of-00101.safetensors", + "model.layers.11.mlp.experts.24.down_proj.weight": "model-00010-of-00101.safetensors", + "model.layers.11.mlp.experts.25.gate_proj.weight": "model-00010-of-00101.safetensors", + "model.layers.11.mlp.experts.25.up_proj.weight": "model-00010-of-00101.safetensors", + "model.layers.11.mlp.experts.25.down_proj.weight": "model-00010-of-00101.safetensors", + "model.layers.11.mlp.experts.26.gate_proj.weight": "model-00010-of-00101.safetensors", + "model.layers.11.mlp.experts.26.up_proj.weight": "model-00010-of-00101.safetensors", + "model.layers.11.mlp.experts.26.down_proj.weight": "model-00010-of-00101.safetensors", + "model.layers.11.mlp.experts.27.gate_proj.weight": "model-00010-of-00101.safetensors", + "model.layers.11.mlp.experts.27.up_proj.weight": "model-00010-of-00101.safetensors", + "model.layers.11.mlp.experts.27.down_proj.weight": "model-00010-of-00101.safetensors", + "model.layers.11.mlp.experts.28.gate_proj.weight": "model-00010-of-00101.safetensors", + "model.layers.11.mlp.experts.28.up_proj.weight": "model-00010-of-00101.safetensors", + "model.layers.11.mlp.experts.28.down_proj.weight": "model-00010-of-00101.safetensors", + "model.layers.11.mlp.experts.29.gate_proj.weight": "model-00010-of-00101.safetensors", + "model.layers.11.mlp.experts.29.up_proj.weight": "model-00010-of-00101.safetensors", + "model.layers.11.mlp.experts.29.down_proj.weight": "model-00010-of-00101.safetensors", + "model.layers.11.mlp.experts.30.gate_proj.weight": "model-00010-of-00101.safetensors", + "model.layers.11.mlp.experts.30.up_proj.weight": "model-00010-of-00101.safetensors", + "model.layers.11.mlp.experts.30.down_proj.weight": "model-00010-of-00101.safetensors", + "model.layers.11.mlp.experts.31.gate_proj.weight": "model-00010-of-00101.safetensors", + "model.layers.11.mlp.experts.31.up_proj.weight": "model-00010-of-00101.safetensors", + "model.layers.11.mlp.experts.31.down_proj.weight": "model-00010-of-00101.safetensors", + "model.layers.11.mlp.experts.32.gate_proj.weight": "model-00010-of-00101.safetensors", + "model.layers.11.mlp.experts.32.up_proj.weight": "model-00010-of-00101.safetensors", + "model.layers.11.mlp.experts.32.down_proj.weight": "model-00010-of-00101.safetensors", + "model.layers.11.mlp.experts.33.gate_proj.weight": "model-00010-of-00101.safetensors", + "model.layers.11.mlp.experts.33.up_proj.weight": "model-00010-of-00101.safetensors", + "model.layers.11.mlp.experts.33.down_proj.weight": "model-00010-of-00101.safetensors", + "model.layers.11.mlp.experts.34.gate_proj.weight": "model-00010-of-00101.safetensors", + "model.layers.11.mlp.experts.34.up_proj.weight": "model-00010-of-00101.safetensors", + "model.layers.11.mlp.experts.34.down_proj.weight": "model-00010-of-00101.safetensors", + "model.layers.11.mlp.experts.35.gate_proj.weight": "model-00010-of-00101.safetensors", + "model.layers.11.mlp.experts.35.up_proj.weight": "model-00010-of-00101.safetensors", + "model.layers.11.mlp.experts.35.down_proj.weight": "model-00010-of-00101.safetensors", + "model.layers.11.mlp.experts.36.gate_proj.weight": "model-00010-of-00101.safetensors", + "model.layers.11.mlp.experts.36.up_proj.weight": "model-00010-of-00101.safetensors", + "model.layers.11.mlp.experts.36.down_proj.weight": "model-00010-of-00101.safetensors", + "model.layers.11.mlp.experts.37.gate_proj.weight": "model-00010-of-00101.safetensors", + "model.layers.11.mlp.experts.37.up_proj.weight": "model-00010-of-00101.safetensors", + "model.layers.11.mlp.experts.37.down_proj.weight": "model-00010-of-00101.safetensors", + "model.layers.11.mlp.experts.38.gate_proj.weight": "model-00010-of-00101.safetensors", + "model.layers.11.mlp.experts.38.up_proj.weight": "model-00010-of-00101.safetensors", + "model.layers.11.mlp.experts.38.down_proj.weight": "model-00010-of-00101.safetensors", + "model.layers.11.mlp.experts.39.gate_proj.weight": "model-00010-of-00101.safetensors", + "model.layers.11.mlp.experts.39.up_proj.weight": "model-00010-of-00101.safetensors", + "model.layers.11.mlp.experts.39.down_proj.weight": "model-00010-of-00101.safetensors", + "model.layers.11.mlp.experts.40.gate_proj.weight": "model-00010-of-00101.safetensors", + "model.layers.11.mlp.experts.40.up_proj.weight": "model-00010-of-00101.safetensors", + "model.layers.11.mlp.experts.40.down_proj.weight": "model-00011-of-00101.safetensors", + "model.layers.11.mlp.experts.41.gate_proj.weight": "model-00011-of-00101.safetensors", + "model.layers.11.mlp.experts.41.up_proj.weight": "model-00011-of-00101.safetensors", + "model.layers.11.mlp.experts.41.down_proj.weight": "model-00011-of-00101.safetensors", + "model.layers.11.mlp.experts.42.gate_proj.weight": "model-00011-of-00101.safetensors", + "model.layers.11.mlp.experts.42.up_proj.weight": "model-00011-of-00101.safetensors", + "model.layers.11.mlp.experts.42.down_proj.weight": "model-00011-of-00101.safetensors", + "model.layers.11.mlp.experts.43.gate_proj.weight": "model-00011-of-00101.safetensors", + "model.layers.11.mlp.experts.43.up_proj.weight": "model-00011-of-00101.safetensors", + "model.layers.11.mlp.experts.43.down_proj.weight": "model-00011-of-00101.safetensors", + "model.layers.11.mlp.experts.44.gate_proj.weight": "model-00011-of-00101.safetensors", + "model.layers.11.mlp.experts.44.up_proj.weight": "model-00011-of-00101.safetensors", + "model.layers.11.mlp.experts.44.down_proj.weight": "model-00011-of-00101.safetensors", + "model.layers.11.mlp.experts.45.gate_proj.weight": "model-00011-of-00101.safetensors", + "model.layers.11.mlp.experts.45.up_proj.weight": "model-00011-of-00101.safetensors", + "model.layers.11.mlp.experts.45.down_proj.weight": "model-00011-of-00101.safetensors", + "model.layers.11.mlp.experts.46.gate_proj.weight": "model-00011-of-00101.safetensors", + "model.layers.11.mlp.experts.46.up_proj.weight": "model-00011-of-00101.safetensors", + "model.layers.11.mlp.experts.46.down_proj.weight": "model-00011-of-00101.safetensors", + "model.layers.11.mlp.experts.47.gate_proj.weight": "model-00011-of-00101.safetensors", + "model.layers.11.mlp.experts.47.up_proj.weight": "model-00011-of-00101.safetensors", + "model.layers.11.mlp.experts.47.down_proj.weight": "model-00011-of-00101.safetensors", + "model.layers.11.mlp.experts.48.gate_proj.weight": "model-00011-of-00101.safetensors", + "model.layers.11.mlp.experts.48.up_proj.weight": "model-00011-of-00101.safetensors", + "model.layers.11.mlp.experts.48.down_proj.weight": "model-00011-of-00101.safetensors", + "model.layers.11.mlp.experts.49.gate_proj.weight": "model-00011-of-00101.safetensors", + "model.layers.11.mlp.experts.49.up_proj.weight": "model-00011-of-00101.safetensors", + "model.layers.11.mlp.experts.49.down_proj.weight": "model-00011-of-00101.safetensors", + "model.layers.11.mlp.experts.50.gate_proj.weight": "model-00011-of-00101.safetensors", + "model.layers.11.mlp.experts.50.up_proj.weight": "model-00011-of-00101.safetensors", + "model.layers.11.mlp.experts.50.down_proj.weight": "model-00011-of-00101.safetensors", + "model.layers.11.mlp.experts.51.gate_proj.weight": "model-00011-of-00101.safetensors", + "model.layers.11.mlp.experts.51.up_proj.weight": "model-00011-of-00101.safetensors", + "model.layers.11.mlp.experts.51.down_proj.weight": "model-00011-of-00101.safetensors", + "model.layers.11.mlp.experts.52.gate_proj.weight": "model-00011-of-00101.safetensors", + "model.layers.11.mlp.experts.52.up_proj.weight": "model-00011-of-00101.safetensors", + "model.layers.11.mlp.experts.52.down_proj.weight": "model-00011-of-00101.safetensors", + "model.layers.11.mlp.experts.53.gate_proj.weight": "model-00011-of-00101.safetensors", + "model.layers.11.mlp.experts.53.up_proj.weight": "model-00011-of-00101.safetensors", + "model.layers.11.mlp.experts.53.down_proj.weight": "model-00011-of-00101.safetensors", + "model.layers.11.mlp.experts.54.gate_proj.weight": "model-00011-of-00101.safetensors", + "model.layers.11.mlp.experts.54.up_proj.weight": "model-00011-of-00101.safetensors", + "model.layers.11.mlp.experts.54.down_proj.weight": "model-00011-of-00101.safetensors", + "model.layers.11.mlp.experts.55.gate_proj.weight": "model-00011-of-00101.safetensors", + "model.layers.11.mlp.experts.55.up_proj.weight": "model-00011-of-00101.safetensors", + "model.layers.11.mlp.experts.55.down_proj.weight": "model-00011-of-00101.safetensors", + "model.layers.11.mlp.experts.56.gate_proj.weight": "model-00011-of-00101.safetensors", + "model.layers.11.mlp.experts.56.up_proj.weight": "model-00011-of-00101.safetensors", + "model.layers.11.mlp.experts.56.down_proj.weight": "model-00011-of-00101.safetensors", + "model.layers.11.mlp.experts.57.gate_proj.weight": "model-00011-of-00101.safetensors", + "model.layers.11.mlp.experts.57.up_proj.weight": "model-00011-of-00101.safetensors", + "model.layers.11.mlp.experts.57.down_proj.weight": "model-00011-of-00101.safetensors", + "model.layers.11.mlp.experts.58.gate_proj.weight": "model-00011-of-00101.safetensors", + "model.layers.11.mlp.experts.58.up_proj.weight": "model-00011-of-00101.safetensors", + "model.layers.11.mlp.experts.58.down_proj.weight": "model-00011-of-00101.safetensors", + "model.layers.11.mlp.experts.59.gate_proj.weight": "model-00011-of-00101.safetensors", + "model.layers.11.mlp.experts.59.up_proj.weight": "model-00011-of-00101.safetensors", + "model.layers.11.mlp.experts.59.down_proj.weight": "model-00011-of-00101.safetensors", + "model.layers.11.mlp.experts.60.gate_proj.weight": "model-00011-of-00101.safetensors", + "model.layers.11.mlp.experts.60.up_proj.weight": "model-00011-of-00101.safetensors", + "model.layers.11.mlp.experts.60.down_proj.weight": "model-00011-of-00101.safetensors", + "model.layers.11.mlp.experts.61.gate_proj.weight": "model-00011-of-00101.safetensors", + "model.layers.11.mlp.experts.61.up_proj.weight": "model-00011-of-00101.safetensors", + "model.layers.11.mlp.experts.61.down_proj.weight": "model-00011-of-00101.safetensors", + "model.layers.11.mlp.experts.62.gate_proj.weight": "model-00011-of-00101.safetensors", + "model.layers.11.mlp.experts.62.up_proj.weight": "model-00011-of-00101.safetensors", + "model.layers.11.mlp.experts.62.down_proj.weight": "model-00011-of-00101.safetensors", + "model.layers.11.mlp.experts.63.gate_proj.weight": "model-00011-of-00101.safetensors", + "model.layers.11.mlp.experts.63.up_proj.weight": "model-00011-of-00101.safetensors", + "model.layers.11.mlp.experts.63.down_proj.weight": "model-00011-of-00101.safetensors", + "model.layers.11.mlp.experts.64.gate_proj.weight": "model-00011-of-00101.safetensors", + "model.layers.11.mlp.experts.64.up_proj.weight": "model-00011-of-00101.safetensors", + "model.layers.11.mlp.experts.64.down_proj.weight": "model-00011-of-00101.safetensors", + "model.layers.11.mlp.experts.65.gate_proj.weight": "model-00011-of-00101.safetensors", + "model.layers.11.mlp.experts.65.up_proj.weight": "model-00011-of-00101.safetensors", + "model.layers.11.mlp.experts.65.down_proj.weight": "model-00011-of-00101.safetensors", + "model.layers.11.mlp.experts.66.gate_proj.weight": "model-00011-of-00101.safetensors", + "model.layers.11.mlp.experts.66.up_proj.weight": "model-00011-of-00101.safetensors", + "model.layers.11.mlp.experts.66.down_proj.weight": "model-00011-of-00101.safetensors", + "model.layers.11.mlp.experts.67.gate_proj.weight": "model-00011-of-00101.safetensors", + "model.layers.11.mlp.experts.67.up_proj.weight": "model-00011-of-00101.safetensors", + "model.layers.11.mlp.experts.67.down_proj.weight": "model-00011-of-00101.safetensors", + "model.layers.11.mlp.experts.68.gate_proj.weight": "model-00011-of-00101.safetensors", + "model.layers.11.mlp.experts.68.up_proj.weight": "model-00011-of-00101.safetensors", + "model.layers.11.mlp.experts.68.down_proj.weight": "model-00011-of-00101.safetensors", + "model.layers.11.mlp.experts.69.gate_proj.weight": "model-00011-of-00101.safetensors", + "model.layers.11.mlp.experts.69.up_proj.weight": "model-00011-of-00101.safetensors", + "model.layers.11.mlp.experts.69.down_proj.weight": "model-00011-of-00101.safetensors", + "model.layers.11.mlp.experts.70.gate_proj.weight": "model-00011-of-00101.safetensors", + "model.layers.11.mlp.experts.70.up_proj.weight": "model-00011-of-00101.safetensors", + "model.layers.11.mlp.experts.70.down_proj.weight": "model-00011-of-00101.safetensors", + "model.layers.11.mlp.experts.71.gate_proj.weight": "model-00011-of-00101.safetensors", + "model.layers.11.mlp.experts.71.up_proj.weight": "model-00011-of-00101.safetensors", + "model.layers.11.mlp.experts.71.down_proj.weight": "model-00011-of-00101.safetensors", + "model.layers.11.mlp.experts.72.gate_proj.weight": "model-00011-of-00101.safetensors", + "model.layers.11.mlp.experts.72.up_proj.weight": "model-00011-of-00101.safetensors", + "model.layers.11.mlp.experts.72.down_proj.weight": "model-00011-of-00101.safetensors", + "model.layers.11.mlp.experts.73.gate_proj.weight": "model-00011-of-00101.safetensors", + "model.layers.11.mlp.experts.73.up_proj.weight": "model-00011-of-00101.safetensors", + "model.layers.11.mlp.experts.73.down_proj.weight": "model-00011-of-00101.safetensors", + "model.layers.11.mlp.experts.74.gate_proj.weight": "model-00011-of-00101.safetensors", + "model.layers.11.mlp.experts.74.up_proj.weight": "model-00011-of-00101.safetensors", + "model.layers.11.mlp.experts.74.down_proj.weight": "model-00011-of-00101.safetensors", + "model.layers.11.mlp.experts.75.gate_proj.weight": "model-00011-of-00101.safetensors", + "model.layers.11.mlp.experts.75.up_proj.weight": "model-00011-of-00101.safetensors", + "model.layers.11.mlp.experts.75.down_proj.weight": "model-00011-of-00101.safetensors", + "model.layers.11.mlp.experts.76.gate_proj.weight": "model-00011-of-00101.safetensors", + "model.layers.11.mlp.experts.76.up_proj.weight": "model-00011-of-00101.safetensors", + "model.layers.11.mlp.experts.76.down_proj.weight": "model-00011-of-00101.safetensors", + "model.layers.11.mlp.experts.77.gate_proj.weight": "model-00011-of-00101.safetensors", + "model.layers.11.mlp.experts.77.up_proj.weight": "model-00011-of-00101.safetensors", + "model.layers.11.mlp.experts.77.down_proj.weight": "model-00011-of-00101.safetensors", + "model.layers.11.mlp.experts.78.gate_proj.weight": "model-00011-of-00101.safetensors", + "model.layers.11.mlp.experts.78.up_proj.weight": "model-00011-of-00101.safetensors", + "model.layers.11.mlp.experts.78.down_proj.weight": "model-00011-of-00101.safetensors", + "model.layers.11.mlp.experts.79.gate_proj.weight": "model-00011-of-00101.safetensors", + "model.layers.11.mlp.experts.79.up_proj.weight": "model-00011-of-00101.safetensors", + "model.layers.11.mlp.experts.79.down_proj.weight": "model-00011-of-00101.safetensors", + "model.layers.11.mlp.experts.80.gate_proj.weight": "model-00011-of-00101.safetensors", + "model.layers.11.mlp.experts.80.up_proj.weight": "model-00011-of-00101.safetensors", + "model.layers.11.mlp.experts.80.down_proj.weight": "model-00011-of-00101.safetensors", + "model.layers.11.mlp.experts.81.gate_proj.weight": "model-00011-of-00101.safetensors", + "model.layers.11.mlp.experts.81.up_proj.weight": "model-00011-of-00101.safetensors", + "model.layers.11.mlp.experts.81.down_proj.weight": "model-00011-of-00101.safetensors", + "model.layers.11.mlp.experts.82.gate_proj.weight": "model-00011-of-00101.safetensors", + "model.layers.11.mlp.experts.82.up_proj.weight": "model-00011-of-00101.safetensors", + "model.layers.11.mlp.experts.82.down_proj.weight": "model-00011-of-00101.safetensors", + "model.layers.11.mlp.experts.83.gate_proj.weight": "model-00011-of-00101.safetensors", + "model.layers.11.mlp.experts.83.up_proj.weight": "model-00011-of-00101.safetensors", + "model.layers.11.mlp.experts.83.down_proj.weight": "model-00011-of-00101.safetensors", + "model.layers.11.mlp.experts.84.gate_proj.weight": "model-00011-of-00101.safetensors", + "model.layers.11.mlp.experts.84.up_proj.weight": "model-00011-of-00101.safetensors", + "model.layers.11.mlp.experts.84.down_proj.weight": "model-00011-of-00101.safetensors", + "model.layers.11.mlp.experts.85.gate_proj.weight": "model-00011-of-00101.safetensors", + "model.layers.11.mlp.experts.85.up_proj.weight": "model-00011-of-00101.safetensors", + "model.layers.11.mlp.experts.85.down_proj.weight": "model-00011-of-00101.safetensors", + "model.layers.11.mlp.experts.86.gate_proj.weight": "model-00011-of-00101.safetensors", + "model.layers.11.mlp.experts.86.up_proj.weight": "model-00011-of-00101.safetensors", + "model.layers.11.mlp.experts.86.down_proj.weight": "model-00011-of-00101.safetensors", + "model.layers.11.mlp.experts.87.gate_proj.weight": "model-00011-of-00101.safetensors", + "model.layers.11.mlp.experts.87.up_proj.weight": "model-00011-of-00101.safetensors", + "model.layers.11.mlp.experts.87.down_proj.weight": "model-00011-of-00101.safetensors", + "model.layers.11.mlp.experts.88.gate_proj.weight": "model-00011-of-00101.safetensors", + "model.layers.11.mlp.experts.88.up_proj.weight": "model-00011-of-00101.safetensors", + "model.layers.11.mlp.experts.88.down_proj.weight": "model-00011-of-00101.safetensors", + "model.layers.11.mlp.experts.89.gate_proj.weight": "model-00011-of-00101.safetensors", + "model.layers.11.mlp.experts.89.up_proj.weight": "model-00011-of-00101.safetensors", + "model.layers.11.mlp.experts.89.down_proj.weight": "model-00011-of-00101.safetensors", + "model.layers.11.mlp.experts.90.gate_proj.weight": "model-00011-of-00101.safetensors", + "model.layers.11.mlp.experts.90.up_proj.weight": "model-00011-of-00101.safetensors", + "model.layers.11.mlp.experts.90.down_proj.weight": "model-00011-of-00101.safetensors", + "model.layers.11.mlp.experts.91.gate_proj.weight": "model-00011-of-00101.safetensors", + "model.layers.11.mlp.experts.91.up_proj.weight": "model-00011-of-00101.safetensors", + "model.layers.11.mlp.experts.91.down_proj.weight": "model-00011-of-00101.safetensors", + "model.layers.11.mlp.experts.92.gate_proj.weight": "model-00011-of-00101.safetensors", + "model.layers.11.mlp.experts.92.up_proj.weight": "model-00011-of-00101.safetensors", + "model.layers.11.mlp.experts.92.down_proj.weight": "model-00011-of-00101.safetensors", + "model.layers.11.mlp.experts.93.gate_proj.weight": "model-00011-of-00101.safetensors", + "model.layers.11.mlp.experts.93.up_proj.weight": "model-00011-of-00101.safetensors", + "model.layers.11.mlp.experts.93.down_proj.weight": "model-00011-of-00101.safetensors", + "model.layers.11.mlp.experts.94.gate_proj.weight": "model-00011-of-00101.safetensors", + "model.layers.11.mlp.experts.94.up_proj.weight": "model-00011-of-00101.safetensors", + "model.layers.11.mlp.experts.94.down_proj.weight": "model-00011-of-00101.safetensors", + "model.layers.11.mlp.experts.95.gate_proj.weight": "model-00011-of-00101.safetensors", + "model.layers.11.mlp.experts.95.up_proj.weight": "model-00011-of-00101.safetensors", + "model.layers.11.mlp.experts.95.down_proj.weight": "model-00011-of-00101.safetensors", + "model.layers.11.mlp.experts.96.gate_proj.weight": "model-00011-of-00101.safetensors", + "model.layers.11.mlp.experts.96.up_proj.weight": "model-00011-of-00101.safetensors", + "model.layers.11.mlp.experts.96.down_proj.weight": "model-00011-of-00101.safetensors", + "model.layers.11.mlp.experts.97.gate_proj.weight": "model-00011-of-00101.safetensors", + "model.layers.11.mlp.experts.97.up_proj.weight": "model-00011-of-00101.safetensors", + "model.layers.11.mlp.experts.97.down_proj.weight": "model-00011-of-00101.safetensors", + "model.layers.11.mlp.experts.98.gate_proj.weight": "model-00011-of-00101.safetensors", + "model.layers.11.mlp.experts.98.up_proj.weight": "model-00011-of-00101.safetensors", + "model.layers.11.mlp.experts.98.down_proj.weight": "model-00011-of-00101.safetensors", + "model.layers.11.mlp.experts.99.gate_proj.weight": "model-00011-of-00101.safetensors", + "model.layers.11.mlp.experts.99.up_proj.weight": "model-00011-of-00101.safetensors", + "model.layers.11.mlp.experts.99.down_proj.weight": "model-00011-of-00101.safetensors", + "model.layers.11.mlp.experts.100.gate_proj.weight": "model-00011-of-00101.safetensors", + "model.layers.11.mlp.experts.100.up_proj.weight": "model-00011-of-00101.safetensors", + "model.layers.11.mlp.experts.100.down_proj.weight": "model-00011-of-00101.safetensors", + "model.layers.11.mlp.experts.101.gate_proj.weight": "model-00011-of-00101.safetensors", + "model.layers.11.mlp.experts.101.up_proj.weight": "model-00011-of-00101.safetensors", + "model.layers.11.mlp.experts.101.down_proj.weight": "model-00011-of-00101.safetensors", + "model.layers.11.mlp.experts.102.gate_proj.weight": "model-00011-of-00101.safetensors", + "model.layers.11.mlp.experts.102.up_proj.weight": "model-00011-of-00101.safetensors", + "model.layers.11.mlp.experts.102.down_proj.weight": "model-00011-of-00101.safetensors", + "model.layers.11.mlp.experts.103.gate_proj.weight": "model-00011-of-00101.safetensors", + "model.layers.11.mlp.experts.103.up_proj.weight": "model-00011-of-00101.safetensors", + "model.layers.11.mlp.experts.103.down_proj.weight": "model-00011-of-00101.safetensors", + "model.layers.11.mlp.experts.104.gate_proj.weight": "model-00011-of-00101.safetensors", + "model.layers.11.mlp.experts.104.up_proj.weight": "model-00011-of-00101.safetensors", + "model.layers.11.mlp.experts.104.down_proj.weight": "model-00011-of-00101.safetensors", + "model.layers.11.mlp.experts.105.gate_proj.weight": "model-00011-of-00101.safetensors", + "model.layers.11.mlp.experts.105.up_proj.weight": "model-00011-of-00101.safetensors", + "model.layers.11.mlp.experts.105.down_proj.weight": "model-00011-of-00101.safetensors", + "model.layers.11.mlp.experts.106.gate_proj.weight": "model-00011-of-00101.safetensors", + "model.layers.11.mlp.experts.106.up_proj.weight": "model-00011-of-00101.safetensors", + "model.layers.11.mlp.experts.106.down_proj.weight": "model-00011-of-00101.safetensors", + "model.layers.11.mlp.experts.107.gate_proj.weight": "model-00011-of-00101.safetensors", + "model.layers.11.mlp.experts.107.up_proj.weight": "model-00011-of-00101.safetensors", + "model.layers.11.mlp.experts.107.down_proj.weight": "model-00011-of-00101.safetensors", + "model.layers.11.mlp.experts.108.gate_proj.weight": "model-00011-of-00101.safetensors", + "model.layers.11.mlp.experts.108.up_proj.weight": "model-00011-of-00101.safetensors", + "model.layers.11.mlp.experts.108.down_proj.weight": "model-00011-of-00101.safetensors", + "model.layers.11.mlp.experts.109.gate_proj.weight": "model-00011-of-00101.safetensors", + "model.layers.11.mlp.experts.109.up_proj.weight": "model-00011-of-00101.safetensors", + "model.layers.11.mlp.experts.109.down_proj.weight": "model-00011-of-00101.safetensors", + "model.layers.11.mlp.experts.110.gate_proj.weight": "model-00011-of-00101.safetensors", + "model.layers.11.mlp.experts.110.up_proj.weight": "model-00011-of-00101.safetensors", + "model.layers.11.mlp.experts.110.down_proj.weight": "model-00011-of-00101.safetensors", + "model.layers.11.mlp.experts.111.gate_proj.weight": "model-00011-of-00101.safetensors", + "model.layers.11.mlp.experts.111.up_proj.weight": "model-00011-of-00101.safetensors", + "model.layers.11.mlp.experts.111.down_proj.weight": "model-00011-of-00101.safetensors", + "model.layers.11.mlp.experts.112.gate_proj.weight": "model-00011-of-00101.safetensors", + "model.layers.11.mlp.experts.112.up_proj.weight": "model-00011-of-00101.safetensors", + "model.layers.11.mlp.experts.112.down_proj.weight": "model-00011-of-00101.safetensors", + "model.layers.11.mlp.experts.113.gate_proj.weight": "model-00011-of-00101.safetensors", + "model.layers.11.mlp.experts.113.up_proj.weight": "model-00011-of-00101.safetensors", + "model.layers.11.mlp.experts.113.down_proj.weight": "model-00011-of-00101.safetensors", + "model.layers.11.mlp.experts.114.gate_proj.weight": "model-00011-of-00101.safetensors", + "model.layers.11.mlp.experts.114.up_proj.weight": "model-00011-of-00101.safetensors", + "model.layers.11.mlp.experts.114.down_proj.weight": "model-00011-of-00101.safetensors", + "model.layers.11.mlp.experts.115.gate_proj.weight": "model-00011-of-00101.safetensors", + "model.layers.11.mlp.experts.115.up_proj.weight": "model-00011-of-00101.safetensors", + "model.layers.11.mlp.experts.115.down_proj.weight": "model-00011-of-00101.safetensors", + "model.layers.11.mlp.experts.116.gate_proj.weight": "model-00011-of-00101.safetensors", + "model.layers.11.mlp.experts.116.up_proj.weight": "model-00011-of-00101.safetensors", + "model.layers.11.mlp.experts.116.down_proj.weight": "model-00011-of-00101.safetensors", + "model.layers.11.mlp.experts.117.gate_proj.weight": "model-00011-of-00101.safetensors", + "model.layers.11.mlp.experts.117.up_proj.weight": "model-00011-of-00101.safetensors", + "model.layers.11.mlp.experts.117.down_proj.weight": "model-00011-of-00101.safetensors", + "model.layers.11.mlp.experts.118.gate_proj.weight": "model-00011-of-00101.safetensors", + "model.layers.11.mlp.experts.118.up_proj.weight": "model-00011-of-00101.safetensors", + "model.layers.11.mlp.experts.118.down_proj.weight": "model-00011-of-00101.safetensors", + "model.layers.11.mlp.experts.119.gate_proj.weight": "model-00011-of-00101.safetensors", + "model.layers.11.mlp.experts.119.up_proj.weight": "model-00011-of-00101.safetensors", + "model.layers.11.mlp.experts.119.down_proj.weight": "model-00011-of-00101.safetensors", + "model.layers.11.mlp.gate.weight": "model-00011-of-00101.safetensors", + "model.layers.11.mlp.gate.e_score_correction_bias": "model-00011-of-00101.safetensors", + "model.layers.11.mlp.shared_experts.gate_proj.weight": "model-00011-of-00101.safetensors", + "model.layers.11.mlp.shared_experts.up_proj.weight": "model-00011-of-00101.safetensors", + "model.layers.11.mlp.shared_experts.down_proj.weight": "model-00011-of-00101.safetensors", + "model.layers.11.input_layernorm.weight": "model-00011-of-00101.safetensors", + "model.layers.11.post_attention_layernorm.weight": "model-00011-of-00101.safetensors", + "model.layers.12.self_attn.q_proj.weight": "model-00011-of-00101.safetensors", + "model.layers.12.self_attn.q_proj.bias": "model-00011-of-00101.safetensors", + "model.layers.12.self_attn.k_proj.weight": "model-00011-of-00101.safetensors", + "model.layers.12.self_attn.k_proj.bias": "model-00011-of-00101.safetensors", + "model.layers.12.self_attn.v_proj.weight": "model-00011-of-00101.safetensors", + "model.layers.12.self_attn.v_proj.bias": "model-00011-of-00101.safetensors", + "model.layers.12.self_attn.o_proj.weight": "model-00011-of-00101.safetensors", + "model.layers.12.self_attn.q_norm.weight": "model-00011-of-00101.safetensors", + "model.layers.12.self_attn.k_norm.weight": "model-00011-of-00101.safetensors", + "model.layers.12.mlp.experts.0.gate_proj.weight": "model-00011-of-00101.safetensors", + "model.layers.12.mlp.experts.0.up_proj.weight": "model-00011-of-00101.safetensors", + "model.layers.12.mlp.experts.0.down_proj.weight": "model-00011-of-00101.safetensors", + "model.layers.12.mlp.experts.1.gate_proj.weight": "model-00011-of-00101.safetensors", + "model.layers.12.mlp.experts.1.up_proj.weight": "model-00011-of-00101.safetensors", + "model.layers.12.mlp.experts.1.down_proj.weight": "model-00011-of-00101.safetensors", + "model.layers.12.mlp.experts.2.gate_proj.weight": "model-00011-of-00101.safetensors", + "model.layers.12.mlp.experts.2.up_proj.weight": "model-00011-of-00101.safetensors", + "model.layers.12.mlp.experts.2.down_proj.weight": "model-00011-of-00101.safetensors", + "model.layers.12.mlp.experts.3.gate_proj.weight": "model-00011-of-00101.safetensors", + "model.layers.12.mlp.experts.3.up_proj.weight": "model-00011-of-00101.safetensors", + "model.layers.12.mlp.experts.3.down_proj.weight": "model-00011-of-00101.safetensors", + "model.layers.12.mlp.experts.4.gate_proj.weight": "model-00011-of-00101.safetensors", + "model.layers.12.mlp.experts.4.up_proj.weight": "model-00011-of-00101.safetensors", + "model.layers.12.mlp.experts.4.down_proj.weight": "model-00011-of-00101.safetensors", + "model.layers.12.mlp.experts.5.gate_proj.weight": "model-00011-of-00101.safetensors", + "model.layers.12.mlp.experts.5.up_proj.weight": "model-00011-of-00101.safetensors", + "model.layers.12.mlp.experts.5.down_proj.weight": "model-00011-of-00101.safetensors", + "model.layers.12.mlp.experts.6.gate_proj.weight": "model-00011-of-00101.safetensors", + "model.layers.12.mlp.experts.6.up_proj.weight": "model-00011-of-00101.safetensors", + "model.layers.12.mlp.experts.6.down_proj.weight": "model-00011-of-00101.safetensors", + "model.layers.12.mlp.experts.7.gate_proj.weight": "model-00011-of-00101.safetensors", + "model.layers.12.mlp.experts.7.up_proj.weight": "model-00011-of-00101.safetensors", + "model.layers.12.mlp.experts.7.down_proj.weight": "model-00011-of-00101.safetensors", + "model.layers.12.mlp.experts.8.gate_proj.weight": "model-00011-of-00101.safetensors", + "model.layers.12.mlp.experts.8.up_proj.weight": "model-00011-of-00101.safetensors", + "model.layers.12.mlp.experts.8.down_proj.weight": "model-00011-of-00101.safetensors", + "model.layers.12.mlp.experts.9.gate_proj.weight": "model-00011-of-00101.safetensors", + "model.layers.12.mlp.experts.9.up_proj.weight": "model-00011-of-00101.safetensors", + "model.layers.12.mlp.experts.9.down_proj.weight": "model-00011-of-00101.safetensors", + "model.layers.12.mlp.experts.10.gate_proj.weight": "model-00011-of-00101.safetensors", + "model.layers.12.mlp.experts.10.up_proj.weight": "model-00011-of-00101.safetensors", + "model.layers.12.mlp.experts.10.down_proj.weight": "model-00011-of-00101.safetensors", + "model.layers.12.mlp.experts.11.gate_proj.weight": "model-00011-of-00101.safetensors", + "model.layers.12.mlp.experts.11.up_proj.weight": "model-00011-of-00101.safetensors", + "model.layers.12.mlp.experts.11.down_proj.weight": "model-00011-of-00101.safetensors", + "model.layers.12.mlp.experts.12.gate_proj.weight": "model-00011-of-00101.safetensors", + "model.layers.12.mlp.experts.12.up_proj.weight": "model-00011-of-00101.safetensors", + "model.layers.12.mlp.experts.12.down_proj.weight": "model-00011-of-00101.safetensors", + "model.layers.12.mlp.experts.13.gate_proj.weight": "model-00011-of-00101.safetensors", + "model.layers.12.mlp.experts.13.up_proj.weight": "model-00011-of-00101.safetensors", + "model.layers.12.mlp.experts.13.down_proj.weight": "model-00011-of-00101.safetensors", + "model.layers.12.mlp.experts.14.gate_proj.weight": "model-00011-of-00101.safetensors", + "model.layers.12.mlp.experts.14.up_proj.weight": "model-00011-of-00101.safetensors", + "model.layers.12.mlp.experts.14.down_proj.weight": "model-00011-of-00101.safetensors", + "model.layers.12.mlp.experts.15.gate_proj.weight": "model-00011-of-00101.safetensors", + "model.layers.12.mlp.experts.15.up_proj.weight": "model-00011-of-00101.safetensors", + "model.layers.12.mlp.experts.15.down_proj.weight": "model-00011-of-00101.safetensors", + "model.layers.12.mlp.experts.16.gate_proj.weight": "model-00011-of-00101.safetensors", + "model.layers.12.mlp.experts.16.up_proj.weight": "model-00011-of-00101.safetensors", + "model.layers.12.mlp.experts.16.down_proj.weight": "model-00011-of-00101.safetensors", + "model.layers.12.mlp.experts.17.gate_proj.weight": "model-00011-of-00101.safetensors", + "model.layers.12.mlp.experts.17.up_proj.weight": "model-00011-of-00101.safetensors", + "model.layers.12.mlp.experts.17.down_proj.weight": "model-00011-of-00101.safetensors", + "model.layers.12.mlp.experts.18.gate_proj.weight": "model-00011-of-00101.safetensors", + "model.layers.12.mlp.experts.18.up_proj.weight": "model-00011-of-00101.safetensors", + "model.layers.12.mlp.experts.18.down_proj.weight": "model-00011-of-00101.safetensors", + "model.layers.12.mlp.experts.19.gate_proj.weight": "model-00011-of-00101.safetensors", + "model.layers.12.mlp.experts.19.up_proj.weight": "model-00011-of-00101.safetensors", + "model.layers.12.mlp.experts.19.down_proj.weight": "model-00011-of-00101.safetensors", + "model.layers.12.mlp.experts.20.gate_proj.weight": "model-00011-of-00101.safetensors", + "model.layers.12.mlp.experts.20.up_proj.weight": "model-00011-of-00101.safetensors", + "model.layers.12.mlp.experts.20.down_proj.weight": "model-00011-of-00101.safetensors", + "model.layers.12.mlp.experts.21.gate_proj.weight": "model-00011-of-00101.safetensors", + "model.layers.12.mlp.experts.21.up_proj.weight": "model-00011-of-00101.safetensors", + "model.layers.12.mlp.experts.21.down_proj.weight": "model-00011-of-00101.safetensors", + "model.layers.12.mlp.experts.22.gate_proj.weight": "model-00011-of-00101.safetensors", + "model.layers.12.mlp.experts.22.up_proj.weight": "model-00011-of-00101.safetensors", + "model.layers.12.mlp.experts.22.down_proj.weight": "model-00011-of-00101.safetensors", + "model.layers.12.mlp.experts.23.gate_proj.weight": "model-00011-of-00101.safetensors", + "model.layers.12.mlp.experts.23.up_proj.weight": "model-00011-of-00101.safetensors", + "model.layers.12.mlp.experts.23.down_proj.weight": "model-00011-of-00101.safetensors", + "model.layers.12.mlp.experts.24.gate_proj.weight": "model-00011-of-00101.safetensors", + "model.layers.12.mlp.experts.24.up_proj.weight": "model-00011-of-00101.safetensors", + "model.layers.12.mlp.experts.24.down_proj.weight": "model-00011-of-00101.safetensors", + "model.layers.12.mlp.experts.25.gate_proj.weight": "model-00011-of-00101.safetensors", + "model.layers.12.mlp.experts.25.up_proj.weight": "model-00011-of-00101.safetensors", + "model.layers.12.mlp.experts.25.down_proj.weight": "model-00011-of-00101.safetensors", + "model.layers.12.mlp.experts.26.gate_proj.weight": "model-00011-of-00101.safetensors", + "model.layers.12.mlp.experts.26.up_proj.weight": "model-00011-of-00101.safetensors", + "model.layers.12.mlp.experts.26.down_proj.weight": "model-00011-of-00101.safetensors", + "model.layers.12.mlp.experts.27.gate_proj.weight": "model-00011-of-00101.safetensors", + "model.layers.12.mlp.experts.27.up_proj.weight": "model-00012-of-00101.safetensors", + "model.layers.12.mlp.experts.27.down_proj.weight": "model-00012-of-00101.safetensors", + "model.layers.12.mlp.experts.28.gate_proj.weight": "model-00012-of-00101.safetensors", + "model.layers.12.mlp.experts.28.up_proj.weight": "model-00012-of-00101.safetensors", + "model.layers.12.mlp.experts.28.down_proj.weight": "model-00012-of-00101.safetensors", + "model.layers.12.mlp.experts.29.gate_proj.weight": "model-00012-of-00101.safetensors", + "model.layers.12.mlp.experts.29.up_proj.weight": "model-00012-of-00101.safetensors", + "model.layers.12.mlp.experts.29.down_proj.weight": "model-00012-of-00101.safetensors", + "model.layers.12.mlp.experts.30.gate_proj.weight": "model-00012-of-00101.safetensors", + "model.layers.12.mlp.experts.30.up_proj.weight": "model-00012-of-00101.safetensors", + "model.layers.12.mlp.experts.30.down_proj.weight": "model-00012-of-00101.safetensors", + "model.layers.12.mlp.experts.31.gate_proj.weight": "model-00012-of-00101.safetensors", + "model.layers.12.mlp.experts.31.up_proj.weight": "model-00012-of-00101.safetensors", + "model.layers.12.mlp.experts.31.down_proj.weight": "model-00012-of-00101.safetensors", + "model.layers.12.mlp.experts.32.gate_proj.weight": "model-00012-of-00101.safetensors", + "model.layers.12.mlp.experts.32.up_proj.weight": "model-00012-of-00101.safetensors", + "model.layers.12.mlp.experts.32.down_proj.weight": "model-00012-of-00101.safetensors", + "model.layers.12.mlp.experts.33.gate_proj.weight": "model-00012-of-00101.safetensors", + "model.layers.12.mlp.experts.33.up_proj.weight": "model-00012-of-00101.safetensors", + "model.layers.12.mlp.experts.33.down_proj.weight": "model-00012-of-00101.safetensors", + "model.layers.12.mlp.experts.34.gate_proj.weight": "model-00012-of-00101.safetensors", + "model.layers.12.mlp.experts.34.up_proj.weight": "model-00012-of-00101.safetensors", + "model.layers.12.mlp.experts.34.down_proj.weight": "model-00012-of-00101.safetensors", + "model.layers.12.mlp.experts.35.gate_proj.weight": "model-00012-of-00101.safetensors", + "model.layers.12.mlp.experts.35.up_proj.weight": "model-00012-of-00101.safetensors", + "model.layers.12.mlp.experts.35.down_proj.weight": "model-00012-of-00101.safetensors", + "model.layers.12.mlp.experts.36.gate_proj.weight": "model-00012-of-00101.safetensors", + "model.layers.12.mlp.experts.36.up_proj.weight": "model-00012-of-00101.safetensors", + "model.layers.12.mlp.experts.36.down_proj.weight": "model-00012-of-00101.safetensors", + "model.layers.12.mlp.experts.37.gate_proj.weight": "model-00012-of-00101.safetensors", + "model.layers.12.mlp.experts.37.up_proj.weight": "model-00012-of-00101.safetensors", + "model.layers.12.mlp.experts.37.down_proj.weight": "model-00012-of-00101.safetensors", + "model.layers.12.mlp.experts.38.gate_proj.weight": "model-00012-of-00101.safetensors", + "model.layers.12.mlp.experts.38.up_proj.weight": "model-00012-of-00101.safetensors", + "model.layers.12.mlp.experts.38.down_proj.weight": "model-00012-of-00101.safetensors", + "model.layers.12.mlp.experts.39.gate_proj.weight": "model-00012-of-00101.safetensors", + "model.layers.12.mlp.experts.39.up_proj.weight": "model-00012-of-00101.safetensors", + "model.layers.12.mlp.experts.39.down_proj.weight": "model-00012-of-00101.safetensors", + "model.layers.12.mlp.experts.40.gate_proj.weight": "model-00012-of-00101.safetensors", + "model.layers.12.mlp.experts.40.up_proj.weight": "model-00012-of-00101.safetensors", + "model.layers.12.mlp.experts.40.down_proj.weight": "model-00012-of-00101.safetensors", + "model.layers.12.mlp.experts.41.gate_proj.weight": "model-00012-of-00101.safetensors", + "model.layers.12.mlp.experts.41.up_proj.weight": "model-00012-of-00101.safetensors", + "model.layers.12.mlp.experts.41.down_proj.weight": "model-00012-of-00101.safetensors", + "model.layers.12.mlp.experts.42.gate_proj.weight": "model-00012-of-00101.safetensors", + "model.layers.12.mlp.experts.42.up_proj.weight": "model-00012-of-00101.safetensors", + "model.layers.12.mlp.experts.42.down_proj.weight": "model-00012-of-00101.safetensors", + "model.layers.12.mlp.experts.43.gate_proj.weight": "model-00012-of-00101.safetensors", + "model.layers.12.mlp.experts.43.up_proj.weight": "model-00012-of-00101.safetensors", + "model.layers.12.mlp.experts.43.down_proj.weight": "model-00012-of-00101.safetensors", + "model.layers.12.mlp.experts.44.gate_proj.weight": "model-00012-of-00101.safetensors", + "model.layers.12.mlp.experts.44.up_proj.weight": "model-00012-of-00101.safetensors", + "model.layers.12.mlp.experts.44.down_proj.weight": "model-00012-of-00101.safetensors", + "model.layers.12.mlp.experts.45.gate_proj.weight": "model-00012-of-00101.safetensors", + "model.layers.12.mlp.experts.45.up_proj.weight": "model-00012-of-00101.safetensors", + "model.layers.12.mlp.experts.45.down_proj.weight": "model-00012-of-00101.safetensors", + "model.layers.12.mlp.experts.46.gate_proj.weight": "model-00012-of-00101.safetensors", + "model.layers.12.mlp.experts.46.up_proj.weight": "model-00012-of-00101.safetensors", + "model.layers.12.mlp.experts.46.down_proj.weight": "model-00012-of-00101.safetensors", + "model.layers.12.mlp.experts.47.gate_proj.weight": "model-00012-of-00101.safetensors", + "model.layers.12.mlp.experts.47.up_proj.weight": "model-00012-of-00101.safetensors", + "model.layers.12.mlp.experts.47.down_proj.weight": "model-00012-of-00101.safetensors", + "model.layers.12.mlp.experts.48.gate_proj.weight": "model-00012-of-00101.safetensors", + "model.layers.12.mlp.experts.48.up_proj.weight": "model-00012-of-00101.safetensors", + "model.layers.12.mlp.experts.48.down_proj.weight": "model-00012-of-00101.safetensors", + "model.layers.12.mlp.experts.49.gate_proj.weight": "model-00012-of-00101.safetensors", + "model.layers.12.mlp.experts.49.up_proj.weight": "model-00012-of-00101.safetensors", + "model.layers.12.mlp.experts.49.down_proj.weight": "model-00012-of-00101.safetensors", + "model.layers.12.mlp.experts.50.gate_proj.weight": "model-00012-of-00101.safetensors", + "model.layers.12.mlp.experts.50.up_proj.weight": "model-00012-of-00101.safetensors", + "model.layers.12.mlp.experts.50.down_proj.weight": "model-00012-of-00101.safetensors", + "model.layers.12.mlp.experts.51.gate_proj.weight": "model-00012-of-00101.safetensors", + "model.layers.12.mlp.experts.51.up_proj.weight": "model-00012-of-00101.safetensors", + "model.layers.12.mlp.experts.51.down_proj.weight": "model-00012-of-00101.safetensors", + "model.layers.12.mlp.experts.52.gate_proj.weight": "model-00012-of-00101.safetensors", + "model.layers.12.mlp.experts.52.up_proj.weight": "model-00012-of-00101.safetensors", + "model.layers.12.mlp.experts.52.down_proj.weight": "model-00012-of-00101.safetensors", + "model.layers.12.mlp.experts.53.gate_proj.weight": "model-00012-of-00101.safetensors", + "model.layers.12.mlp.experts.53.up_proj.weight": "model-00012-of-00101.safetensors", + "model.layers.12.mlp.experts.53.down_proj.weight": "model-00012-of-00101.safetensors", + "model.layers.12.mlp.experts.54.gate_proj.weight": "model-00012-of-00101.safetensors", + "model.layers.12.mlp.experts.54.up_proj.weight": "model-00012-of-00101.safetensors", + "model.layers.12.mlp.experts.54.down_proj.weight": "model-00012-of-00101.safetensors", + "model.layers.12.mlp.experts.55.gate_proj.weight": "model-00012-of-00101.safetensors", + "model.layers.12.mlp.experts.55.up_proj.weight": "model-00012-of-00101.safetensors", + "model.layers.12.mlp.experts.55.down_proj.weight": "model-00012-of-00101.safetensors", + "model.layers.12.mlp.experts.56.gate_proj.weight": "model-00012-of-00101.safetensors", + "model.layers.12.mlp.experts.56.up_proj.weight": "model-00012-of-00101.safetensors", + "model.layers.12.mlp.experts.56.down_proj.weight": "model-00012-of-00101.safetensors", + "model.layers.12.mlp.experts.57.gate_proj.weight": "model-00012-of-00101.safetensors", + "model.layers.12.mlp.experts.57.up_proj.weight": "model-00012-of-00101.safetensors", + "model.layers.12.mlp.experts.57.down_proj.weight": "model-00012-of-00101.safetensors", + "model.layers.12.mlp.experts.58.gate_proj.weight": "model-00012-of-00101.safetensors", + "model.layers.12.mlp.experts.58.up_proj.weight": "model-00012-of-00101.safetensors", + "model.layers.12.mlp.experts.58.down_proj.weight": "model-00012-of-00101.safetensors", + "model.layers.12.mlp.experts.59.gate_proj.weight": "model-00012-of-00101.safetensors", + "model.layers.12.mlp.experts.59.up_proj.weight": "model-00012-of-00101.safetensors", + "model.layers.12.mlp.experts.59.down_proj.weight": "model-00012-of-00101.safetensors", + "model.layers.12.mlp.experts.60.gate_proj.weight": "model-00012-of-00101.safetensors", + "model.layers.12.mlp.experts.60.up_proj.weight": "model-00012-of-00101.safetensors", + "model.layers.12.mlp.experts.60.down_proj.weight": "model-00012-of-00101.safetensors", + "model.layers.12.mlp.experts.61.gate_proj.weight": "model-00012-of-00101.safetensors", + "model.layers.12.mlp.experts.61.up_proj.weight": "model-00012-of-00101.safetensors", + "model.layers.12.mlp.experts.61.down_proj.weight": "model-00012-of-00101.safetensors", + "model.layers.12.mlp.experts.62.gate_proj.weight": "model-00012-of-00101.safetensors", + "model.layers.12.mlp.experts.62.up_proj.weight": "model-00012-of-00101.safetensors", + "model.layers.12.mlp.experts.62.down_proj.weight": "model-00012-of-00101.safetensors", + "model.layers.12.mlp.experts.63.gate_proj.weight": "model-00012-of-00101.safetensors", + "model.layers.12.mlp.experts.63.up_proj.weight": "model-00012-of-00101.safetensors", + "model.layers.12.mlp.experts.63.down_proj.weight": "model-00012-of-00101.safetensors", + "model.layers.12.mlp.experts.64.gate_proj.weight": "model-00012-of-00101.safetensors", + "model.layers.12.mlp.experts.64.up_proj.weight": "model-00012-of-00101.safetensors", + "model.layers.12.mlp.experts.64.down_proj.weight": "model-00012-of-00101.safetensors", + "model.layers.12.mlp.experts.65.gate_proj.weight": "model-00012-of-00101.safetensors", + "model.layers.12.mlp.experts.65.up_proj.weight": "model-00012-of-00101.safetensors", + "model.layers.12.mlp.experts.65.down_proj.weight": "model-00012-of-00101.safetensors", + "model.layers.12.mlp.experts.66.gate_proj.weight": "model-00012-of-00101.safetensors", + "model.layers.12.mlp.experts.66.up_proj.weight": "model-00012-of-00101.safetensors", + "model.layers.12.mlp.experts.66.down_proj.weight": "model-00012-of-00101.safetensors", + "model.layers.12.mlp.experts.67.gate_proj.weight": "model-00012-of-00101.safetensors", + "model.layers.12.mlp.experts.67.up_proj.weight": "model-00012-of-00101.safetensors", + "model.layers.12.mlp.experts.67.down_proj.weight": "model-00012-of-00101.safetensors", + "model.layers.12.mlp.experts.68.gate_proj.weight": "model-00012-of-00101.safetensors", + "model.layers.12.mlp.experts.68.up_proj.weight": "model-00012-of-00101.safetensors", + "model.layers.12.mlp.experts.68.down_proj.weight": "model-00012-of-00101.safetensors", + "model.layers.12.mlp.experts.69.gate_proj.weight": "model-00012-of-00101.safetensors", + "model.layers.12.mlp.experts.69.up_proj.weight": "model-00012-of-00101.safetensors", + "model.layers.12.mlp.experts.69.down_proj.weight": "model-00012-of-00101.safetensors", + "model.layers.12.mlp.experts.70.gate_proj.weight": "model-00012-of-00101.safetensors", + "model.layers.12.mlp.experts.70.up_proj.weight": "model-00012-of-00101.safetensors", + "model.layers.12.mlp.experts.70.down_proj.weight": "model-00012-of-00101.safetensors", + "model.layers.12.mlp.experts.71.gate_proj.weight": "model-00012-of-00101.safetensors", + "model.layers.12.mlp.experts.71.up_proj.weight": "model-00012-of-00101.safetensors", + "model.layers.12.mlp.experts.71.down_proj.weight": "model-00012-of-00101.safetensors", + "model.layers.12.mlp.experts.72.gate_proj.weight": "model-00012-of-00101.safetensors", + "model.layers.12.mlp.experts.72.up_proj.weight": "model-00012-of-00101.safetensors", + "model.layers.12.mlp.experts.72.down_proj.weight": "model-00012-of-00101.safetensors", + "model.layers.12.mlp.experts.73.gate_proj.weight": "model-00012-of-00101.safetensors", + "model.layers.12.mlp.experts.73.up_proj.weight": "model-00012-of-00101.safetensors", + "model.layers.12.mlp.experts.73.down_proj.weight": "model-00012-of-00101.safetensors", + "model.layers.12.mlp.experts.74.gate_proj.weight": "model-00012-of-00101.safetensors", + "model.layers.12.mlp.experts.74.up_proj.weight": "model-00012-of-00101.safetensors", + "model.layers.12.mlp.experts.74.down_proj.weight": "model-00012-of-00101.safetensors", + "model.layers.12.mlp.experts.75.gate_proj.weight": "model-00012-of-00101.safetensors", + "model.layers.12.mlp.experts.75.up_proj.weight": "model-00012-of-00101.safetensors", + "model.layers.12.mlp.experts.75.down_proj.weight": "model-00012-of-00101.safetensors", + "model.layers.12.mlp.experts.76.gate_proj.weight": "model-00012-of-00101.safetensors", + "model.layers.12.mlp.experts.76.up_proj.weight": "model-00012-of-00101.safetensors", + "model.layers.12.mlp.experts.76.down_proj.weight": "model-00012-of-00101.safetensors", + "model.layers.12.mlp.experts.77.gate_proj.weight": "model-00012-of-00101.safetensors", + "model.layers.12.mlp.experts.77.up_proj.weight": "model-00012-of-00101.safetensors", + "model.layers.12.mlp.experts.77.down_proj.weight": "model-00012-of-00101.safetensors", + "model.layers.12.mlp.experts.78.gate_proj.weight": "model-00012-of-00101.safetensors", + "model.layers.12.mlp.experts.78.up_proj.weight": "model-00012-of-00101.safetensors", + "model.layers.12.mlp.experts.78.down_proj.weight": "model-00012-of-00101.safetensors", + "model.layers.12.mlp.experts.79.gate_proj.weight": "model-00012-of-00101.safetensors", + "model.layers.12.mlp.experts.79.up_proj.weight": "model-00012-of-00101.safetensors", + "model.layers.12.mlp.experts.79.down_proj.weight": "model-00012-of-00101.safetensors", + "model.layers.12.mlp.experts.80.gate_proj.weight": "model-00012-of-00101.safetensors", + "model.layers.12.mlp.experts.80.up_proj.weight": "model-00012-of-00101.safetensors", + "model.layers.12.mlp.experts.80.down_proj.weight": "model-00012-of-00101.safetensors", + "model.layers.12.mlp.experts.81.gate_proj.weight": "model-00012-of-00101.safetensors", + "model.layers.12.mlp.experts.81.up_proj.weight": "model-00012-of-00101.safetensors", + "model.layers.12.mlp.experts.81.down_proj.weight": "model-00012-of-00101.safetensors", + "model.layers.12.mlp.experts.82.gate_proj.weight": "model-00012-of-00101.safetensors", + "model.layers.12.mlp.experts.82.up_proj.weight": "model-00012-of-00101.safetensors", + "model.layers.12.mlp.experts.82.down_proj.weight": "model-00012-of-00101.safetensors", + "model.layers.12.mlp.experts.83.gate_proj.weight": "model-00012-of-00101.safetensors", + "model.layers.12.mlp.experts.83.up_proj.weight": "model-00012-of-00101.safetensors", + "model.layers.12.mlp.experts.83.down_proj.weight": "model-00012-of-00101.safetensors", + "model.layers.12.mlp.experts.84.gate_proj.weight": "model-00012-of-00101.safetensors", + "model.layers.12.mlp.experts.84.up_proj.weight": "model-00012-of-00101.safetensors", + "model.layers.12.mlp.experts.84.down_proj.weight": "model-00012-of-00101.safetensors", + "model.layers.12.mlp.experts.85.gate_proj.weight": "model-00012-of-00101.safetensors", + "model.layers.12.mlp.experts.85.up_proj.weight": "model-00012-of-00101.safetensors", + "model.layers.12.mlp.experts.85.down_proj.weight": "model-00012-of-00101.safetensors", + "model.layers.12.mlp.experts.86.gate_proj.weight": "model-00012-of-00101.safetensors", + "model.layers.12.mlp.experts.86.up_proj.weight": "model-00012-of-00101.safetensors", + "model.layers.12.mlp.experts.86.down_proj.weight": "model-00012-of-00101.safetensors", + "model.layers.12.mlp.experts.87.gate_proj.weight": "model-00012-of-00101.safetensors", + "model.layers.12.mlp.experts.87.up_proj.weight": "model-00012-of-00101.safetensors", + "model.layers.12.mlp.experts.87.down_proj.weight": "model-00012-of-00101.safetensors", + "model.layers.12.mlp.experts.88.gate_proj.weight": "model-00012-of-00101.safetensors", + "model.layers.12.mlp.experts.88.up_proj.weight": "model-00012-of-00101.safetensors", + "model.layers.12.mlp.experts.88.down_proj.weight": "model-00012-of-00101.safetensors", + "model.layers.12.mlp.experts.89.gate_proj.weight": "model-00012-of-00101.safetensors", + "model.layers.12.mlp.experts.89.up_proj.weight": "model-00012-of-00101.safetensors", + "model.layers.12.mlp.experts.89.down_proj.weight": "model-00012-of-00101.safetensors", + "model.layers.12.mlp.experts.90.gate_proj.weight": "model-00012-of-00101.safetensors", + "model.layers.12.mlp.experts.90.up_proj.weight": "model-00012-of-00101.safetensors", + "model.layers.12.mlp.experts.90.down_proj.weight": "model-00012-of-00101.safetensors", + "model.layers.12.mlp.experts.91.gate_proj.weight": "model-00012-of-00101.safetensors", + "model.layers.12.mlp.experts.91.up_proj.weight": "model-00012-of-00101.safetensors", + "model.layers.12.mlp.experts.91.down_proj.weight": "model-00012-of-00101.safetensors", + "model.layers.12.mlp.experts.92.gate_proj.weight": "model-00012-of-00101.safetensors", + "model.layers.12.mlp.experts.92.up_proj.weight": "model-00012-of-00101.safetensors", + "model.layers.12.mlp.experts.92.down_proj.weight": "model-00012-of-00101.safetensors", + "model.layers.12.mlp.experts.93.gate_proj.weight": "model-00012-of-00101.safetensors", + "model.layers.12.mlp.experts.93.up_proj.weight": "model-00012-of-00101.safetensors", + "model.layers.12.mlp.experts.93.down_proj.weight": "model-00012-of-00101.safetensors", + "model.layers.12.mlp.experts.94.gate_proj.weight": "model-00012-of-00101.safetensors", + "model.layers.12.mlp.experts.94.up_proj.weight": "model-00012-of-00101.safetensors", + "model.layers.12.mlp.experts.94.down_proj.weight": "model-00012-of-00101.safetensors", + "model.layers.12.mlp.experts.95.gate_proj.weight": "model-00012-of-00101.safetensors", + "model.layers.12.mlp.experts.95.up_proj.weight": "model-00012-of-00101.safetensors", + "model.layers.12.mlp.experts.95.down_proj.weight": "model-00012-of-00101.safetensors", + "model.layers.12.mlp.experts.96.gate_proj.weight": "model-00012-of-00101.safetensors", + "model.layers.12.mlp.experts.96.up_proj.weight": "model-00012-of-00101.safetensors", + "model.layers.12.mlp.experts.96.down_proj.weight": "model-00012-of-00101.safetensors", + "model.layers.12.mlp.experts.97.gate_proj.weight": "model-00012-of-00101.safetensors", + "model.layers.12.mlp.experts.97.up_proj.weight": "model-00012-of-00101.safetensors", + "model.layers.12.mlp.experts.97.down_proj.weight": "model-00012-of-00101.safetensors", + "model.layers.12.mlp.experts.98.gate_proj.weight": "model-00012-of-00101.safetensors", + "model.layers.12.mlp.experts.98.up_proj.weight": "model-00012-of-00101.safetensors", + "model.layers.12.mlp.experts.98.down_proj.weight": "model-00012-of-00101.safetensors", + "model.layers.12.mlp.experts.99.gate_proj.weight": "model-00012-of-00101.safetensors", + "model.layers.12.mlp.experts.99.up_proj.weight": "model-00012-of-00101.safetensors", + "model.layers.12.mlp.experts.99.down_proj.weight": "model-00012-of-00101.safetensors", + "model.layers.12.mlp.experts.100.gate_proj.weight": "model-00012-of-00101.safetensors", + "model.layers.12.mlp.experts.100.up_proj.weight": "model-00012-of-00101.safetensors", + "model.layers.12.mlp.experts.100.down_proj.weight": "model-00012-of-00101.safetensors", + "model.layers.12.mlp.experts.101.gate_proj.weight": "model-00012-of-00101.safetensors", + "model.layers.12.mlp.experts.101.up_proj.weight": "model-00012-of-00101.safetensors", + "model.layers.12.mlp.experts.101.down_proj.weight": "model-00012-of-00101.safetensors", + "model.layers.12.mlp.experts.102.gate_proj.weight": "model-00012-of-00101.safetensors", + "model.layers.12.mlp.experts.102.up_proj.weight": "model-00012-of-00101.safetensors", + "model.layers.12.mlp.experts.102.down_proj.weight": "model-00012-of-00101.safetensors", + "model.layers.12.mlp.experts.103.gate_proj.weight": "model-00012-of-00101.safetensors", + "model.layers.12.mlp.experts.103.up_proj.weight": "model-00012-of-00101.safetensors", + "model.layers.12.mlp.experts.103.down_proj.weight": "model-00012-of-00101.safetensors", + "model.layers.12.mlp.experts.104.gate_proj.weight": "model-00012-of-00101.safetensors", + "model.layers.12.mlp.experts.104.up_proj.weight": "model-00012-of-00101.safetensors", + "model.layers.12.mlp.experts.104.down_proj.weight": "model-00012-of-00101.safetensors", + "model.layers.12.mlp.experts.105.gate_proj.weight": "model-00012-of-00101.safetensors", + "model.layers.12.mlp.experts.105.up_proj.weight": "model-00012-of-00101.safetensors", + "model.layers.12.mlp.experts.105.down_proj.weight": "model-00012-of-00101.safetensors", + "model.layers.12.mlp.experts.106.gate_proj.weight": "model-00012-of-00101.safetensors", + "model.layers.12.mlp.experts.106.up_proj.weight": "model-00012-of-00101.safetensors", + "model.layers.12.mlp.experts.106.down_proj.weight": "model-00012-of-00101.safetensors", + "model.layers.12.mlp.experts.107.gate_proj.weight": "model-00012-of-00101.safetensors", + "model.layers.12.mlp.experts.107.up_proj.weight": "model-00012-of-00101.safetensors", + "model.layers.12.mlp.experts.107.down_proj.weight": "model-00012-of-00101.safetensors", + "model.layers.12.mlp.experts.108.gate_proj.weight": "model-00012-of-00101.safetensors", + "model.layers.12.mlp.experts.108.up_proj.weight": "model-00012-of-00101.safetensors", + "model.layers.12.mlp.experts.108.down_proj.weight": "model-00012-of-00101.safetensors", + "model.layers.12.mlp.experts.109.gate_proj.weight": "model-00012-of-00101.safetensors", + "model.layers.12.mlp.experts.109.up_proj.weight": "model-00012-of-00101.safetensors", + "model.layers.12.mlp.experts.109.down_proj.weight": "model-00012-of-00101.safetensors", + "model.layers.12.mlp.experts.110.gate_proj.weight": "model-00012-of-00101.safetensors", + "model.layers.12.mlp.experts.110.up_proj.weight": "model-00012-of-00101.safetensors", + "model.layers.12.mlp.experts.110.down_proj.weight": "model-00012-of-00101.safetensors", + "model.layers.12.mlp.experts.111.gate_proj.weight": "model-00012-of-00101.safetensors", + "model.layers.12.mlp.experts.111.up_proj.weight": "model-00012-of-00101.safetensors", + "model.layers.12.mlp.experts.111.down_proj.weight": "model-00012-of-00101.safetensors", + "model.layers.12.mlp.experts.112.gate_proj.weight": "model-00012-of-00101.safetensors", + "model.layers.12.mlp.experts.112.up_proj.weight": "model-00012-of-00101.safetensors", + "model.layers.12.mlp.experts.112.down_proj.weight": "model-00012-of-00101.safetensors", + "model.layers.12.mlp.experts.113.gate_proj.weight": "model-00012-of-00101.safetensors", + "model.layers.12.mlp.experts.113.up_proj.weight": "model-00012-of-00101.safetensors", + "model.layers.12.mlp.experts.113.down_proj.weight": "model-00012-of-00101.safetensors", + "model.layers.12.mlp.experts.114.gate_proj.weight": "model-00012-of-00101.safetensors", + "model.layers.12.mlp.experts.114.up_proj.weight": "model-00012-of-00101.safetensors", + "model.layers.12.mlp.experts.114.down_proj.weight": "model-00012-of-00101.safetensors", + "model.layers.12.mlp.experts.115.gate_proj.weight": "model-00012-of-00101.safetensors", + "model.layers.12.mlp.experts.115.up_proj.weight": "model-00012-of-00101.safetensors", + "model.layers.12.mlp.experts.115.down_proj.weight": "model-00012-of-00101.safetensors", + "model.layers.12.mlp.experts.116.gate_proj.weight": "model-00012-of-00101.safetensors", + "model.layers.12.mlp.experts.116.up_proj.weight": "model-00012-of-00101.safetensors", + "model.layers.12.mlp.experts.116.down_proj.weight": "model-00012-of-00101.safetensors", + "model.layers.12.mlp.experts.117.gate_proj.weight": "model-00012-of-00101.safetensors", + "model.layers.12.mlp.experts.117.up_proj.weight": "model-00012-of-00101.safetensors", + "model.layers.12.mlp.experts.117.down_proj.weight": "model-00012-of-00101.safetensors", + "model.layers.12.mlp.experts.118.gate_proj.weight": "model-00012-of-00101.safetensors", + "model.layers.12.mlp.experts.118.up_proj.weight": "model-00012-of-00101.safetensors", + "model.layers.12.mlp.experts.118.down_proj.weight": "model-00012-of-00101.safetensors", + "model.layers.12.mlp.experts.119.gate_proj.weight": "model-00012-of-00101.safetensors", + "model.layers.12.mlp.experts.119.up_proj.weight": "model-00012-of-00101.safetensors", + "model.layers.12.mlp.experts.119.down_proj.weight": "model-00012-of-00101.safetensors", + "model.layers.12.mlp.gate.weight": "model-00012-of-00101.safetensors", + "model.layers.12.mlp.gate.e_score_correction_bias": "model-00012-of-00101.safetensors", + "model.layers.12.mlp.shared_experts.gate_proj.weight": "model-00012-of-00101.safetensors", + "model.layers.12.mlp.shared_experts.up_proj.weight": "model-00012-of-00101.safetensors", + "model.layers.12.mlp.shared_experts.down_proj.weight": "model-00012-of-00101.safetensors", + "model.layers.12.input_layernorm.weight": "model-00012-of-00101.safetensors", + "model.layers.12.post_attention_layernorm.weight": "model-00012-of-00101.safetensors", + "model.layers.13.self_attn.q_proj.weight": "model-00012-of-00101.safetensors", + "model.layers.13.self_attn.q_proj.bias": "model-00012-of-00101.safetensors", + "model.layers.13.self_attn.k_proj.weight": "model-00012-of-00101.safetensors", + "model.layers.13.self_attn.k_proj.bias": "model-00012-of-00101.safetensors", + "model.layers.13.self_attn.v_proj.weight": "model-00012-of-00101.safetensors", + "model.layers.13.self_attn.v_proj.bias": "model-00012-of-00101.safetensors", + "model.layers.13.self_attn.o_proj.weight": "model-00012-of-00101.safetensors", + "model.layers.13.self_attn.q_norm.weight": "model-00012-of-00101.safetensors", + "model.layers.13.self_attn.k_norm.weight": "model-00012-of-00101.safetensors", + "model.layers.13.mlp.experts.0.gate_proj.weight": "model-00012-of-00101.safetensors", + "model.layers.13.mlp.experts.0.up_proj.weight": "model-00012-of-00101.safetensors", + "model.layers.13.mlp.experts.0.down_proj.weight": "model-00012-of-00101.safetensors", + "model.layers.13.mlp.experts.1.gate_proj.weight": "model-00012-of-00101.safetensors", + "model.layers.13.mlp.experts.1.up_proj.weight": "model-00012-of-00101.safetensors", + "model.layers.13.mlp.experts.1.down_proj.weight": "model-00012-of-00101.safetensors", + "model.layers.13.mlp.experts.2.gate_proj.weight": "model-00012-of-00101.safetensors", + "model.layers.13.mlp.experts.2.up_proj.weight": "model-00012-of-00101.safetensors", + "model.layers.13.mlp.experts.2.down_proj.weight": "model-00012-of-00101.safetensors", + "model.layers.13.mlp.experts.3.gate_proj.weight": "model-00012-of-00101.safetensors", + "model.layers.13.mlp.experts.3.up_proj.weight": "model-00012-of-00101.safetensors", + "model.layers.13.mlp.experts.3.down_proj.weight": "model-00012-of-00101.safetensors", + "model.layers.13.mlp.experts.4.gate_proj.weight": "model-00012-of-00101.safetensors", + "model.layers.13.mlp.experts.4.up_proj.weight": "model-00012-of-00101.safetensors", + "model.layers.13.mlp.experts.4.down_proj.weight": "model-00012-of-00101.safetensors", + "model.layers.13.mlp.experts.5.gate_proj.weight": "model-00012-of-00101.safetensors", + "model.layers.13.mlp.experts.5.up_proj.weight": "model-00012-of-00101.safetensors", + "model.layers.13.mlp.experts.5.down_proj.weight": "model-00012-of-00101.safetensors", + "model.layers.13.mlp.experts.6.gate_proj.weight": "model-00012-of-00101.safetensors", + "model.layers.13.mlp.experts.6.up_proj.weight": "model-00012-of-00101.safetensors", + "model.layers.13.mlp.experts.6.down_proj.weight": "model-00012-of-00101.safetensors", + "model.layers.13.mlp.experts.7.gate_proj.weight": "model-00012-of-00101.safetensors", + "model.layers.13.mlp.experts.7.up_proj.weight": "model-00012-of-00101.safetensors", + "model.layers.13.mlp.experts.7.down_proj.weight": "model-00012-of-00101.safetensors", + "model.layers.13.mlp.experts.8.gate_proj.weight": "model-00012-of-00101.safetensors", + "model.layers.13.mlp.experts.8.up_proj.weight": "model-00012-of-00101.safetensors", + "model.layers.13.mlp.experts.8.down_proj.weight": "model-00012-of-00101.safetensors", + "model.layers.13.mlp.experts.9.gate_proj.weight": "model-00012-of-00101.safetensors", + "model.layers.13.mlp.experts.9.up_proj.weight": "model-00012-of-00101.safetensors", + "model.layers.13.mlp.experts.9.down_proj.weight": "model-00012-of-00101.safetensors", + "model.layers.13.mlp.experts.10.gate_proj.weight": "model-00012-of-00101.safetensors", + "model.layers.13.mlp.experts.10.up_proj.weight": "model-00012-of-00101.safetensors", + "model.layers.13.mlp.experts.10.down_proj.weight": "model-00012-of-00101.safetensors", + "model.layers.13.mlp.experts.11.gate_proj.weight": "model-00012-of-00101.safetensors", + "model.layers.13.mlp.experts.11.up_proj.weight": "model-00012-of-00101.safetensors", + "model.layers.13.mlp.experts.11.down_proj.weight": "model-00012-of-00101.safetensors", + "model.layers.13.mlp.experts.12.gate_proj.weight": "model-00012-of-00101.safetensors", + "model.layers.13.mlp.experts.12.up_proj.weight": "model-00012-of-00101.safetensors", + "model.layers.13.mlp.experts.12.down_proj.weight": "model-00012-of-00101.safetensors", + "model.layers.13.mlp.experts.13.gate_proj.weight": "model-00012-of-00101.safetensors", + "model.layers.13.mlp.experts.13.up_proj.weight": "model-00012-of-00101.safetensors", + "model.layers.13.mlp.experts.13.down_proj.weight": "model-00012-of-00101.safetensors", + "model.layers.13.mlp.experts.14.gate_proj.weight": "model-00013-of-00101.safetensors", + "model.layers.13.mlp.experts.14.up_proj.weight": "model-00013-of-00101.safetensors", + "model.layers.13.mlp.experts.14.down_proj.weight": "model-00013-of-00101.safetensors", + "model.layers.13.mlp.experts.15.gate_proj.weight": "model-00013-of-00101.safetensors", + "model.layers.13.mlp.experts.15.up_proj.weight": "model-00013-of-00101.safetensors", + "model.layers.13.mlp.experts.15.down_proj.weight": "model-00013-of-00101.safetensors", + "model.layers.13.mlp.experts.16.gate_proj.weight": "model-00013-of-00101.safetensors", + "model.layers.13.mlp.experts.16.up_proj.weight": "model-00013-of-00101.safetensors", + "model.layers.13.mlp.experts.16.down_proj.weight": "model-00013-of-00101.safetensors", + "model.layers.13.mlp.experts.17.gate_proj.weight": "model-00013-of-00101.safetensors", + "model.layers.13.mlp.experts.17.up_proj.weight": "model-00013-of-00101.safetensors", + "model.layers.13.mlp.experts.17.down_proj.weight": "model-00013-of-00101.safetensors", + "model.layers.13.mlp.experts.18.gate_proj.weight": "model-00013-of-00101.safetensors", + "model.layers.13.mlp.experts.18.up_proj.weight": "model-00013-of-00101.safetensors", + "model.layers.13.mlp.experts.18.down_proj.weight": "model-00013-of-00101.safetensors", + "model.layers.13.mlp.experts.19.gate_proj.weight": "model-00013-of-00101.safetensors", + "model.layers.13.mlp.experts.19.up_proj.weight": "model-00013-of-00101.safetensors", + "model.layers.13.mlp.experts.19.down_proj.weight": "model-00013-of-00101.safetensors", + "model.layers.13.mlp.experts.20.gate_proj.weight": "model-00013-of-00101.safetensors", + "model.layers.13.mlp.experts.20.up_proj.weight": "model-00013-of-00101.safetensors", + "model.layers.13.mlp.experts.20.down_proj.weight": "model-00013-of-00101.safetensors", + "model.layers.13.mlp.experts.21.gate_proj.weight": "model-00013-of-00101.safetensors", + "model.layers.13.mlp.experts.21.up_proj.weight": "model-00013-of-00101.safetensors", + "model.layers.13.mlp.experts.21.down_proj.weight": "model-00013-of-00101.safetensors", + "model.layers.13.mlp.experts.22.gate_proj.weight": "model-00013-of-00101.safetensors", + "model.layers.13.mlp.experts.22.up_proj.weight": "model-00013-of-00101.safetensors", + "model.layers.13.mlp.experts.22.down_proj.weight": "model-00013-of-00101.safetensors", + "model.layers.13.mlp.experts.23.gate_proj.weight": "model-00013-of-00101.safetensors", + "model.layers.13.mlp.experts.23.up_proj.weight": "model-00013-of-00101.safetensors", + "model.layers.13.mlp.experts.23.down_proj.weight": "model-00013-of-00101.safetensors", + "model.layers.13.mlp.experts.24.gate_proj.weight": "model-00013-of-00101.safetensors", + "model.layers.13.mlp.experts.24.up_proj.weight": "model-00013-of-00101.safetensors", + "model.layers.13.mlp.experts.24.down_proj.weight": "model-00013-of-00101.safetensors", + "model.layers.13.mlp.experts.25.gate_proj.weight": "model-00013-of-00101.safetensors", + "model.layers.13.mlp.experts.25.up_proj.weight": "model-00013-of-00101.safetensors", + "model.layers.13.mlp.experts.25.down_proj.weight": "model-00013-of-00101.safetensors", + "model.layers.13.mlp.experts.26.gate_proj.weight": "model-00013-of-00101.safetensors", + "model.layers.13.mlp.experts.26.up_proj.weight": "model-00013-of-00101.safetensors", + "model.layers.13.mlp.experts.26.down_proj.weight": "model-00013-of-00101.safetensors", + "model.layers.13.mlp.experts.27.gate_proj.weight": "model-00013-of-00101.safetensors", + "model.layers.13.mlp.experts.27.up_proj.weight": "model-00013-of-00101.safetensors", + "model.layers.13.mlp.experts.27.down_proj.weight": "model-00013-of-00101.safetensors", + "model.layers.13.mlp.experts.28.gate_proj.weight": "model-00013-of-00101.safetensors", + "model.layers.13.mlp.experts.28.up_proj.weight": "model-00013-of-00101.safetensors", + "model.layers.13.mlp.experts.28.down_proj.weight": "model-00013-of-00101.safetensors", + "model.layers.13.mlp.experts.29.gate_proj.weight": "model-00013-of-00101.safetensors", + "model.layers.13.mlp.experts.29.up_proj.weight": "model-00013-of-00101.safetensors", + "model.layers.13.mlp.experts.29.down_proj.weight": "model-00013-of-00101.safetensors", + "model.layers.13.mlp.experts.30.gate_proj.weight": "model-00013-of-00101.safetensors", + "model.layers.13.mlp.experts.30.up_proj.weight": "model-00013-of-00101.safetensors", + "model.layers.13.mlp.experts.30.down_proj.weight": "model-00013-of-00101.safetensors", + "model.layers.13.mlp.experts.31.gate_proj.weight": "model-00013-of-00101.safetensors", + "model.layers.13.mlp.experts.31.up_proj.weight": "model-00013-of-00101.safetensors", + "model.layers.13.mlp.experts.31.down_proj.weight": "model-00013-of-00101.safetensors", + "model.layers.13.mlp.experts.32.gate_proj.weight": "model-00013-of-00101.safetensors", + "model.layers.13.mlp.experts.32.up_proj.weight": "model-00013-of-00101.safetensors", + "model.layers.13.mlp.experts.32.down_proj.weight": "model-00013-of-00101.safetensors", + "model.layers.13.mlp.experts.33.gate_proj.weight": "model-00013-of-00101.safetensors", + "model.layers.13.mlp.experts.33.up_proj.weight": "model-00013-of-00101.safetensors", + "model.layers.13.mlp.experts.33.down_proj.weight": "model-00013-of-00101.safetensors", + "model.layers.13.mlp.experts.34.gate_proj.weight": "model-00013-of-00101.safetensors", + "model.layers.13.mlp.experts.34.up_proj.weight": "model-00013-of-00101.safetensors", + "model.layers.13.mlp.experts.34.down_proj.weight": "model-00013-of-00101.safetensors", + "model.layers.13.mlp.experts.35.gate_proj.weight": "model-00013-of-00101.safetensors", + "model.layers.13.mlp.experts.35.up_proj.weight": "model-00013-of-00101.safetensors", + "model.layers.13.mlp.experts.35.down_proj.weight": "model-00013-of-00101.safetensors", + "model.layers.13.mlp.experts.36.gate_proj.weight": "model-00013-of-00101.safetensors", + "model.layers.13.mlp.experts.36.up_proj.weight": "model-00013-of-00101.safetensors", + "model.layers.13.mlp.experts.36.down_proj.weight": "model-00013-of-00101.safetensors", + "model.layers.13.mlp.experts.37.gate_proj.weight": "model-00013-of-00101.safetensors", + "model.layers.13.mlp.experts.37.up_proj.weight": "model-00013-of-00101.safetensors", + "model.layers.13.mlp.experts.37.down_proj.weight": "model-00013-of-00101.safetensors", + "model.layers.13.mlp.experts.38.gate_proj.weight": "model-00013-of-00101.safetensors", + "model.layers.13.mlp.experts.38.up_proj.weight": "model-00013-of-00101.safetensors", + "model.layers.13.mlp.experts.38.down_proj.weight": "model-00013-of-00101.safetensors", + "model.layers.13.mlp.experts.39.gate_proj.weight": "model-00013-of-00101.safetensors", + "model.layers.13.mlp.experts.39.up_proj.weight": "model-00013-of-00101.safetensors", + "model.layers.13.mlp.experts.39.down_proj.weight": "model-00013-of-00101.safetensors", + "model.layers.13.mlp.experts.40.gate_proj.weight": "model-00013-of-00101.safetensors", + "model.layers.13.mlp.experts.40.up_proj.weight": "model-00013-of-00101.safetensors", + "model.layers.13.mlp.experts.40.down_proj.weight": "model-00013-of-00101.safetensors", + "model.layers.13.mlp.experts.41.gate_proj.weight": "model-00013-of-00101.safetensors", + "model.layers.13.mlp.experts.41.up_proj.weight": "model-00013-of-00101.safetensors", + "model.layers.13.mlp.experts.41.down_proj.weight": "model-00013-of-00101.safetensors", + "model.layers.13.mlp.experts.42.gate_proj.weight": "model-00013-of-00101.safetensors", + "model.layers.13.mlp.experts.42.up_proj.weight": "model-00013-of-00101.safetensors", + "model.layers.13.mlp.experts.42.down_proj.weight": "model-00013-of-00101.safetensors", + "model.layers.13.mlp.experts.43.gate_proj.weight": "model-00013-of-00101.safetensors", + "model.layers.13.mlp.experts.43.up_proj.weight": "model-00013-of-00101.safetensors", + "model.layers.13.mlp.experts.43.down_proj.weight": "model-00013-of-00101.safetensors", + "model.layers.13.mlp.experts.44.gate_proj.weight": "model-00013-of-00101.safetensors", + "model.layers.13.mlp.experts.44.up_proj.weight": "model-00013-of-00101.safetensors", + "model.layers.13.mlp.experts.44.down_proj.weight": "model-00013-of-00101.safetensors", + "model.layers.13.mlp.experts.45.gate_proj.weight": "model-00013-of-00101.safetensors", + "model.layers.13.mlp.experts.45.up_proj.weight": "model-00013-of-00101.safetensors", + "model.layers.13.mlp.experts.45.down_proj.weight": "model-00013-of-00101.safetensors", + "model.layers.13.mlp.experts.46.gate_proj.weight": "model-00013-of-00101.safetensors", + "model.layers.13.mlp.experts.46.up_proj.weight": "model-00013-of-00101.safetensors", + "model.layers.13.mlp.experts.46.down_proj.weight": "model-00013-of-00101.safetensors", + "model.layers.13.mlp.experts.47.gate_proj.weight": "model-00013-of-00101.safetensors", + "model.layers.13.mlp.experts.47.up_proj.weight": "model-00013-of-00101.safetensors", + "model.layers.13.mlp.experts.47.down_proj.weight": "model-00013-of-00101.safetensors", + "model.layers.13.mlp.experts.48.gate_proj.weight": "model-00013-of-00101.safetensors", + "model.layers.13.mlp.experts.48.up_proj.weight": "model-00013-of-00101.safetensors", + "model.layers.13.mlp.experts.48.down_proj.weight": "model-00013-of-00101.safetensors", + "model.layers.13.mlp.experts.49.gate_proj.weight": "model-00013-of-00101.safetensors", + "model.layers.13.mlp.experts.49.up_proj.weight": "model-00013-of-00101.safetensors", + "model.layers.13.mlp.experts.49.down_proj.weight": "model-00013-of-00101.safetensors", + "model.layers.13.mlp.experts.50.gate_proj.weight": "model-00013-of-00101.safetensors", + "model.layers.13.mlp.experts.50.up_proj.weight": "model-00013-of-00101.safetensors", + "model.layers.13.mlp.experts.50.down_proj.weight": "model-00013-of-00101.safetensors", + "model.layers.13.mlp.experts.51.gate_proj.weight": "model-00013-of-00101.safetensors", + "model.layers.13.mlp.experts.51.up_proj.weight": "model-00013-of-00101.safetensors", + "model.layers.13.mlp.experts.51.down_proj.weight": "model-00013-of-00101.safetensors", + "model.layers.13.mlp.experts.52.gate_proj.weight": "model-00013-of-00101.safetensors", + "model.layers.13.mlp.experts.52.up_proj.weight": "model-00013-of-00101.safetensors", + "model.layers.13.mlp.experts.52.down_proj.weight": "model-00013-of-00101.safetensors", + "model.layers.13.mlp.experts.53.gate_proj.weight": "model-00013-of-00101.safetensors", + "model.layers.13.mlp.experts.53.up_proj.weight": "model-00013-of-00101.safetensors", + "model.layers.13.mlp.experts.53.down_proj.weight": "model-00013-of-00101.safetensors", + "model.layers.13.mlp.experts.54.gate_proj.weight": "model-00013-of-00101.safetensors", + "model.layers.13.mlp.experts.54.up_proj.weight": "model-00013-of-00101.safetensors", + "model.layers.13.mlp.experts.54.down_proj.weight": "model-00013-of-00101.safetensors", + "model.layers.13.mlp.experts.55.gate_proj.weight": "model-00013-of-00101.safetensors", + "model.layers.13.mlp.experts.55.up_proj.weight": "model-00013-of-00101.safetensors", + "model.layers.13.mlp.experts.55.down_proj.weight": "model-00013-of-00101.safetensors", + "model.layers.13.mlp.experts.56.gate_proj.weight": "model-00013-of-00101.safetensors", + "model.layers.13.mlp.experts.56.up_proj.weight": "model-00013-of-00101.safetensors", + "model.layers.13.mlp.experts.56.down_proj.weight": "model-00013-of-00101.safetensors", + "model.layers.13.mlp.experts.57.gate_proj.weight": "model-00013-of-00101.safetensors", + "model.layers.13.mlp.experts.57.up_proj.weight": "model-00013-of-00101.safetensors", + "model.layers.13.mlp.experts.57.down_proj.weight": "model-00013-of-00101.safetensors", + "model.layers.13.mlp.experts.58.gate_proj.weight": "model-00013-of-00101.safetensors", + "model.layers.13.mlp.experts.58.up_proj.weight": "model-00013-of-00101.safetensors", + "model.layers.13.mlp.experts.58.down_proj.weight": "model-00013-of-00101.safetensors", + "model.layers.13.mlp.experts.59.gate_proj.weight": "model-00013-of-00101.safetensors", + "model.layers.13.mlp.experts.59.up_proj.weight": "model-00013-of-00101.safetensors", + "model.layers.13.mlp.experts.59.down_proj.weight": "model-00013-of-00101.safetensors", + "model.layers.13.mlp.experts.60.gate_proj.weight": "model-00013-of-00101.safetensors", + "model.layers.13.mlp.experts.60.up_proj.weight": "model-00013-of-00101.safetensors", + "model.layers.13.mlp.experts.60.down_proj.weight": "model-00013-of-00101.safetensors", + "model.layers.13.mlp.experts.61.gate_proj.weight": "model-00013-of-00101.safetensors", + "model.layers.13.mlp.experts.61.up_proj.weight": "model-00013-of-00101.safetensors", + "model.layers.13.mlp.experts.61.down_proj.weight": "model-00013-of-00101.safetensors", + "model.layers.13.mlp.experts.62.gate_proj.weight": "model-00013-of-00101.safetensors", + "model.layers.13.mlp.experts.62.up_proj.weight": "model-00013-of-00101.safetensors", + "model.layers.13.mlp.experts.62.down_proj.weight": "model-00013-of-00101.safetensors", + "model.layers.13.mlp.experts.63.gate_proj.weight": "model-00013-of-00101.safetensors", + "model.layers.13.mlp.experts.63.up_proj.weight": "model-00013-of-00101.safetensors", + "model.layers.13.mlp.experts.63.down_proj.weight": "model-00013-of-00101.safetensors", + "model.layers.13.mlp.experts.64.gate_proj.weight": "model-00013-of-00101.safetensors", + "model.layers.13.mlp.experts.64.up_proj.weight": "model-00013-of-00101.safetensors", + "model.layers.13.mlp.experts.64.down_proj.weight": "model-00013-of-00101.safetensors", + "model.layers.13.mlp.experts.65.gate_proj.weight": "model-00013-of-00101.safetensors", + "model.layers.13.mlp.experts.65.up_proj.weight": "model-00013-of-00101.safetensors", + "model.layers.13.mlp.experts.65.down_proj.weight": "model-00013-of-00101.safetensors", + "model.layers.13.mlp.experts.66.gate_proj.weight": "model-00013-of-00101.safetensors", + "model.layers.13.mlp.experts.66.up_proj.weight": "model-00013-of-00101.safetensors", + "model.layers.13.mlp.experts.66.down_proj.weight": "model-00013-of-00101.safetensors", + "model.layers.13.mlp.experts.67.gate_proj.weight": "model-00013-of-00101.safetensors", + "model.layers.13.mlp.experts.67.up_proj.weight": "model-00013-of-00101.safetensors", + "model.layers.13.mlp.experts.67.down_proj.weight": "model-00013-of-00101.safetensors", + "model.layers.13.mlp.experts.68.gate_proj.weight": "model-00013-of-00101.safetensors", + "model.layers.13.mlp.experts.68.up_proj.weight": "model-00013-of-00101.safetensors", + "model.layers.13.mlp.experts.68.down_proj.weight": "model-00013-of-00101.safetensors", + "model.layers.13.mlp.experts.69.gate_proj.weight": "model-00013-of-00101.safetensors", + "model.layers.13.mlp.experts.69.up_proj.weight": "model-00013-of-00101.safetensors", + "model.layers.13.mlp.experts.69.down_proj.weight": "model-00013-of-00101.safetensors", + "model.layers.13.mlp.experts.70.gate_proj.weight": "model-00013-of-00101.safetensors", + "model.layers.13.mlp.experts.70.up_proj.weight": "model-00013-of-00101.safetensors", + "model.layers.13.mlp.experts.70.down_proj.weight": "model-00013-of-00101.safetensors", + "model.layers.13.mlp.experts.71.gate_proj.weight": "model-00013-of-00101.safetensors", + "model.layers.13.mlp.experts.71.up_proj.weight": "model-00013-of-00101.safetensors", + "model.layers.13.mlp.experts.71.down_proj.weight": "model-00013-of-00101.safetensors", + "model.layers.13.mlp.experts.72.gate_proj.weight": "model-00013-of-00101.safetensors", + "model.layers.13.mlp.experts.72.up_proj.weight": "model-00013-of-00101.safetensors", + "model.layers.13.mlp.experts.72.down_proj.weight": "model-00013-of-00101.safetensors", + "model.layers.13.mlp.experts.73.gate_proj.weight": "model-00013-of-00101.safetensors", + "model.layers.13.mlp.experts.73.up_proj.weight": "model-00013-of-00101.safetensors", + "model.layers.13.mlp.experts.73.down_proj.weight": "model-00013-of-00101.safetensors", + "model.layers.13.mlp.experts.74.gate_proj.weight": "model-00013-of-00101.safetensors", + "model.layers.13.mlp.experts.74.up_proj.weight": "model-00013-of-00101.safetensors", + "model.layers.13.mlp.experts.74.down_proj.weight": "model-00013-of-00101.safetensors", + "model.layers.13.mlp.experts.75.gate_proj.weight": "model-00013-of-00101.safetensors", + "model.layers.13.mlp.experts.75.up_proj.weight": "model-00013-of-00101.safetensors", + "model.layers.13.mlp.experts.75.down_proj.weight": "model-00013-of-00101.safetensors", + "model.layers.13.mlp.experts.76.gate_proj.weight": "model-00013-of-00101.safetensors", + "model.layers.13.mlp.experts.76.up_proj.weight": "model-00013-of-00101.safetensors", + "model.layers.13.mlp.experts.76.down_proj.weight": "model-00013-of-00101.safetensors", + "model.layers.13.mlp.experts.77.gate_proj.weight": "model-00013-of-00101.safetensors", + "model.layers.13.mlp.experts.77.up_proj.weight": "model-00013-of-00101.safetensors", + "model.layers.13.mlp.experts.77.down_proj.weight": "model-00013-of-00101.safetensors", + "model.layers.13.mlp.experts.78.gate_proj.weight": "model-00013-of-00101.safetensors", + "model.layers.13.mlp.experts.78.up_proj.weight": "model-00013-of-00101.safetensors", + "model.layers.13.mlp.experts.78.down_proj.weight": "model-00013-of-00101.safetensors", + "model.layers.13.mlp.experts.79.gate_proj.weight": "model-00013-of-00101.safetensors", + "model.layers.13.mlp.experts.79.up_proj.weight": "model-00013-of-00101.safetensors", + "model.layers.13.mlp.experts.79.down_proj.weight": "model-00013-of-00101.safetensors", + "model.layers.13.mlp.experts.80.gate_proj.weight": "model-00013-of-00101.safetensors", + "model.layers.13.mlp.experts.80.up_proj.weight": "model-00013-of-00101.safetensors", + "model.layers.13.mlp.experts.80.down_proj.weight": "model-00013-of-00101.safetensors", + "model.layers.13.mlp.experts.81.gate_proj.weight": "model-00013-of-00101.safetensors", + "model.layers.13.mlp.experts.81.up_proj.weight": "model-00013-of-00101.safetensors", + "model.layers.13.mlp.experts.81.down_proj.weight": "model-00013-of-00101.safetensors", + "model.layers.13.mlp.experts.82.gate_proj.weight": "model-00013-of-00101.safetensors", + "model.layers.13.mlp.experts.82.up_proj.weight": "model-00013-of-00101.safetensors", + "model.layers.13.mlp.experts.82.down_proj.weight": "model-00013-of-00101.safetensors", + "model.layers.13.mlp.experts.83.gate_proj.weight": "model-00013-of-00101.safetensors", + "model.layers.13.mlp.experts.83.up_proj.weight": "model-00013-of-00101.safetensors", + "model.layers.13.mlp.experts.83.down_proj.weight": "model-00013-of-00101.safetensors", + "model.layers.13.mlp.experts.84.gate_proj.weight": "model-00013-of-00101.safetensors", + "model.layers.13.mlp.experts.84.up_proj.weight": "model-00013-of-00101.safetensors", + "model.layers.13.mlp.experts.84.down_proj.weight": "model-00013-of-00101.safetensors", + "model.layers.13.mlp.experts.85.gate_proj.weight": "model-00013-of-00101.safetensors", + "model.layers.13.mlp.experts.85.up_proj.weight": "model-00013-of-00101.safetensors", + "model.layers.13.mlp.experts.85.down_proj.weight": "model-00013-of-00101.safetensors", + "model.layers.13.mlp.experts.86.gate_proj.weight": "model-00013-of-00101.safetensors", + "model.layers.13.mlp.experts.86.up_proj.weight": "model-00013-of-00101.safetensors", + "model.layers.13.mlp.experts.86.down_proj.weight": "model-00013-of-00101.safetensors", + "model.layers.13.mlp.experts.87.gate_proj.weight": "model-00013-of-00101.safetensors", + "model.layers.13.mlp.experts.87.up_proj.weight": "model-00013-of-00101.safetensors", + "model.layers.13.mlp.experts.87.down_proj.weight": "model-00013-of-00101.safetensors", + "model.layers.13.mlp.experts.88.gate_proj.weight": "model-00013-of-00101.safetensors", + "model.layers.13.mlp.experts.88.up_proj.weight": "model-00013-of-00101.safetensors", + "model.layers.13.mlp.experts.88.down_proj.weight": "model-00013-of-00101.safetensors", + "model.layers.13.mlp.experts.89.gate_proj.weight": "model-00013-of-00101.safetensors", + "model.layers.13.mlp.experts.89.up_proj.weight": "model-00013-of-00101.safetensors", + "model.layers.13.mlp.experts.89.down_proj.weight": "model-00013-of-00101.safetensors", + "model.layers.13.mlp.experts.90.gate_proj.weight": "model-00013-of-00101.safetensors", + "model.layers.13.mlp.experts.90.up_proj.weight": "model-00013-of-00101.safetensors", + "model.layers.13.mlp.experts.90.down_proj.weight": "model-00013-of-00101.safetensors", + "model.layers.13.mlp.experts.91.gate_proj.weight": "model-00013-of-00101.safetensors", + "model.layers.13.mlp.experts.91.up_proj.weight": "model-00013-of-00101.safetensors", + "model.layers.13.mlp.experts.91.down_proj.weight": "model-00013-of-00101.safetensors", + "model.layers.13.mlp.experts.92.gate_proj.weight": "model-00013-of-00101.safetensors", + "model.layers.13.mlp.experts.92.up_proj.weight": "model-00013-of-00101.safetensors", + "model.layers.13.mlp.experts.92.down_proj.weight": "model-00013-of-00101.safetensors", + "model.layers.13.mlp.experts.93.gate_proj.weight": "model-00013-of-00101.safetensors", + "model.layers.13.mlp.experts.93.up_proj.weight": "model-00013-of-00101.safetensors", + "model.layers.13.mlp.experts.93.down_proj.weight": "model-00013-of-00101.safetensors", + "model.layers.13.mlp.experts.94.gate_proj.weight": "model-00013-of-00101.safetensors", + "model.layers.13.mlp.experts.94.up_proj.weight": "model-00013-of-00101.safetensors", + "model.layers.13.mlp.experts.94.down_proj.weight": "model-00013-of-00101.safetensors", + "model.layers.13.mlp.experts.95.gate_proj.weight": "model-00013-of-00101.safetensors", + "model.layers.13.mlp.experts.95.up_proj.weight": "model-00013-of-00101.safetensors", + "model.layers.13.mlp.experts.95.down_proj.weight": "model-00013-of-00101.safetensors", + "model.layers.13.mlp.experts.96.gate_proj.weight": "model-00013-of-00101.safetensors", + "model.layers.13.mlp.experts.96.up_proj.weight": "model-00013-of-00101.safetensors", + "model.layers.13.mlp.experts.96.down_proj.weight": "model-00013-of-00101.safetensors", + "model.layers.13.mlp.experts.97.gate_proj.weight": "model-00013-of-00101.safetensors", + "model.layers.13.mlp.experts.97.up_proj.weight": "model-00013-of-00101.safetensors", + "model.layers.13.mlp.experts.97.down_proj.weight": "model-00013-of-00101.safetensors", + "model.layers.13.mlp.experts.98.gate_proj.weight": "model-00013-of-00101.safetensors", + "model.layers.13.mlp.experts.98.up_proj.weight": "model-00013-of-00101.safetensors", + "model.layers.13.mlp.experts.98.down_proj.weight": "model-00013-of-00101.safetensors", + "model.layers.13.mlp.experts.99.gate_proj.weight": "model-00013-of-00101.safetensors", + "model.layers.13.mlp.experts.99.up_proj.weight": "model-00013-of-00101.safetensors", + "model.layers.13.mlp.experts.99.down_proj.weight": "model-00013-of-00101.safetensors", + "model.layers.13.mlp.experts.100.gate_proj.weight": "model-00013-of-00101.safetensors", + "model.layers.13.mlp.experts.100.up_proj.weight": "model-00013-of-00101.safetensors", + "model.layers.13.mlp.experts.100.down_proj.weight": "model-00013-of-00101.safetensors", + "model.layers.13.mlp.experts.101.gate_proj.weight": "model-00013-of-00101.safetensors", + "model.layers.13.mlp.experts.101.up_proj.weight": "model-00013-of-00101.safetensors", + "model.layers.13.mlp.experts.101.down_proj.weight": "model-00013-of-00101.safetensors", + "model.layers.13.mlp.experts.102.gate_proj.weight": "model-00013-of-00101.safetensors", + "model.layers.13.mlp.experts.102.up_proj.weight": "model-00013-of-00101.safetensors", + "model.layers.13.mlp.experts.102.down_proj.weight": "model-00013-of-00101.safetensors", + "model.layers.13.mlp.experts.103.gate_proj.weight": "model-00013-of-00101.safetensors", + "model.layers.13.mlp.experts.103.up_proj.weight": "model-00013-of-00101.safetensors", + "model.layers.13.mlp.experts.103.down_proj.weight": "model-00013-of-00101.safetensors", + "model.layers.13.mlp.experts.104.gate_proj.weight": "model-00013-of-00101.safetensors", + "model.layers.13.mlp.experts.104.up_proj.weight": "model-00013-of-00101.safetensors", + "model.layers.13.mlp.experts.104.down_proj.weight": "model-00013-of-00101.safetensors", + "model.layers.13.mlp.experts.105.gate_proj.weight": "model-00013-of-00101.safetensors", + "model.layers.13.mlp.experts.105.up_proj.weight": "model-00013-of-00101.safetensors", + "model.layers.13.mlp.experts.105.down_proj.weight": "model-00013-of-00101.safetensors", + "model.layers.13.mlp.experts.106.gate_proj.weight": "model-00013-of-00101.safetensors", + "model.layers.13.mlp.experts.106.up_proj.weight": "model-00013-of-00101.safetensors", + "model.layers.13.mlp.experts.106.down_proj.weight": "model-00013-of-00101.safetensors", + "model.layers.13.mlp.experts.107.gate_proj.weight": "model-00013-of-00101.safetensors", + "model.layers.13.mlp.experts.107.up_proj.weight": "model-00013-of-00101.safetensors", + "model.layers.13.mlp.experts.107.down_proj.weight": "model-00013-of-00101.safetensors", + "model.layers.13.mlp.experts.108.gate_proj.weight": "model-00013-of-00101.safetensors", + "model.layers.13.mlp.experts.108.up_proj.weight": "model-00013-of-00101.safetensors", + "model.layers.13.mlp.experts.108.down_proj.weight": "model-00013-of-00101.safetensors", + "model.layers.13.mlp.experts.109.gate_proj.weight": "model-00013-of-00101.safetensors", + "model.layers.13.mlp.experts.109.up_proj.weight": "model-00013-of-00101.safetensors", + "model.layers.13.mlp.experts.109.down_proj.weight": "model-00013-of-00101.safetensors", + "model.layers.13.mlp.experts.110.gate_proj.weight": "model-00013-of-00101.safetensors", + "model.layers.13.mlp.experts.110.up_proj.weight": "model-00013-of-00101.safetensors", + "model.layers.13.mlp.experts.110.down_proj.weight": "model-00013-of-00101.safetensors", + "model.layers.13.mlp.experts.111.gate_proj.weight": "model-00013-of-00101.safetensors", + "model.layers.13.mlp.experts.111.up_proj.weight": "model-00013-of-00101.safetensors", + "model.layers.13.mlp.experts.111.down_proj.weight": "model-00013-of-00101.safetensors", + "model.layers.13.mlp.experts.112.gate_proj.weight": "model-00013-of-00101.safetensors", + "model.layers.13.mlp.experts.112.up_proj.weight": "model-00013-of-00101.safetensors", + "model.layers.13.mlp.experts.112.down_proj.weight": "model-00013-of-00101.safetensors", + "model.layers.13.mlp.experts.113.gate_proj.weight": "model-00013-of-00101.safetensors", + "model.layers.13.mlp.experts.113.up_proj.weight": "model-00013-of-00101.safetensors", + "model.layers.13.mlp.experts.113.down_proj.weight": "model-00013-of-00101.safetensors", + "model.layers.13.mlp.experts.114.gate_proj.weight": "model-00013-of-00101.safetensors", + "model.layers.13.mlp.experts.114.up_proj.weight": "model-00013-of-00101.safetensors", + "model.layers.13.mlp.experts.114.down_proj.weight": "model-00013-of-00101.safetensors", + "model.layers.13.mlp.experts.115.gate_proj.weight": "model-00013-of-00101.safetensors", + "model.layers.13.mlp.experts.115.up_proj.weight": "model-00013-of-00101.safetensors", + "model.layers.13.mlp.experts.115.down_proj.weight": "model-00013-of-00101.safetensors", + "model.layers.13.mlp.experts.116.gate_proj.weight": "model-00013-of-00101.safetensors", + "model.layers.13.mlp.experts.116.up_proj.weight": "model-00013-of-00101.safetensors", + "model.layers.13.mlp.experts.116.down_proj.weight": "model-00013-of-00101.safetensors", + "model.layers.13.mlp.experts.117.gate_proj.weight": "model-00013-of-00101.safetensors", + "model.layers.13.mlp.experts.117.up_proj.weight": "model-00013-of-00101.safetensors", + "model.layers.13.mlp.experts.117.down_proj.weight": "model-00013-of-00101.safetensors", + "model.layers.13.mlp.experts.118.gate_proj.weight": "model-00013-of-00101.safetensors", + "model.layers.13.mlp.experts.118.up_proj.weight": "model-00013-of-00101.safetensors", + "model.layers.13.mlp.experts.118.down_proj.weight": "model-00013-of-00101.safetensors", + "model.layers.13.mlp.experts.119.gate_proj.weight": "model-00013-of-00101.safetensors", + "model.layers.13.mlp.experts.119.up_proj.weight": "model-00013-of-00101.safetensors", + "model.layers.13.mlp.experts.119.down_proj.weight": "model-00013-of-00101.safetensors", + "model.layers.13.mlp.gate.weight": "model-00013-of-00101.safetensors", + "model.layers.13.mlp.gate.e_score_correction_bias": "model-00013-of-00101.safetensors", + "model.layers.13.mlp.shared_experts.gate_proj.weight": "model-00013-of-00101.safetensors", + "model.layers.13.mlp.shared_experts.up_proj.weight": "model-00013-of-00101.safetensors", + "model.layers.13.mlp.shared_experts.down_proj.weight": "model-00013-of-00101.safetensors", + "model.layers.13.input_layernorm.weight": "model-00013-of-00101.safetensors", + "model.layers.13.post_attention_layernorm.weight": "model-00013-of-00101.safetensors", + "model.layers.14.self_attn.q_proj.weight": "model-00013-of-00101.safetensors", + "model.layers.14.self_attn.q_proj.bias": "model-00013-of-00101.safetensors", + "model.layers.14.self_attn.k_proj.weight": "model-00013-of-00101.safetensors", + "model.layers.14.self_attn.k_proj.bias": "model-00013-of-00101.safetensors", + "model.layers.14.self_attn.v_proj.weight": "model-00013-of-00101.safetensors", + "model.layers.14.self_attn.v_proj.bias": "model-00013-of-00101.safetensors", + "model.layers.14.self_attn.o_proj.weight": "model-00013-of-00101.safetensors", + "model.layers.14.self_attn.q_norm.weight": "model-00013-of-00101.safetensors", + "model.layers.14.self_attn.k_norm.weight": "model-00013-of-00101.safetensors", + "model.layers.14.mlp.experts.0.gate_proj.weight": "model-00013-of-00101.safetensors", + "model.layers.14.mlp.experts.0.up_proj.weight": "model-00013-of-00101.safetensors", + "model.layers.14.mlp.experts.0.down_proj.weight": "model-00014-of-00101.safetensors", + "model.layers.14.mlp.experts.1.gate_proj.weight": "model-00014-of-00101.safetensors", + "model.layers.14.mlp.experts.1.up_proj.weight": "model-00014-of-00101.safetensors", + "model.layers.14.mlp.experts.1.down_proj.weight": "model-00014-of-00101.safetensors", + "model.layers.14.mlp.experts.2.gate_proj.weight": "model-00014-of-00101.safetensors", + "model.layers.14.mlp.experts.2.up_proj.weight": "model-00014-of-00101.safetensors", + "model.layers.14.mlp.experts.2.down_proj.weight": "model-00014-of-00101.safetensors", + "model.layers.14.mlp.experts.3.gate_proj.weight": "model-00014-of-00101.safetensors", + "model.layers.14.mlp.experts.3.up_proj.weight": "model-00014-of-00101.safetensors", + "model.layers.14.mlp.experts.3.down_proj.weight": "model-00014-of-00101.safetensors", + "model.layers.14.mlp.experts.4.gate_proj.weight": "model-00014-of-00101.safetensors", + "model.layers.14.mlp.experts.4.up_proj.weight": "model-00014-of-00101.safetensors", + "model.layers.14.mlp.experts.4.down_proj.weight": "model-00014-of-00101.safetensors", + "model.layers.14.mlp.experts.5.gate_proj.weight": "model-00014-of-00101.safetensors", + "model.layers.14.mlp.experts.5.up_proj.weight": "model-00014-of-00101.safetensors", + "model.layers.14.mlp.experts.5.down_proj.weight": "model-00014-of-00101.safetensors", + "model.layers.14.mlp.experts.6.gate_proj.weight": "model-00014-of-00101.safetensors", + "model.layers.14.mlp.experts.6.up_proj.weight": "model-00014-of-00101.safetensors", + "model.layers.14.mlp.experts.6.down_proj.weight": "model-00014-of-00101.safetensors", + "model.layers.14.mlp.experts.7.gate_proj.weight": "model-00014-of-00101.safetensors", + "model.layers.14.mlp.experts.7.up_proj.weight": "model-00014-of-00101.safetensors", + "model.layers.14.mlp.experts.7.down_proj.weight": "model-00014-of-00101.safetensors", + "model.layers.14.mlp.experts.8.gate_proj.weight": "model-00014-of-00101.safetensors", + "model.layers.14.mlp.experts.8.up_proj.weight": "model-00014-of-00101.safetensors", + "model.layers.14.mlp.experts.8.down_proj.weight": "model-00014-of-00101.safetensors", + "model.layers.14.mlp.experts.9.gate_proj.weight": "model-00014-of-00101.safetensors", + "model.layers.14.mlp.experts.9.up_proj.weight": "model-00014-of-00101.safetensors", + "model.layers.14.mlp.experts.9.down_proj.weight": "model-00014-of-00101.safetensors", + "model.layers.14.mlp.experts.10.gate_proj.weight": "model-00014-of-00101.safetensors", + "model.layers.14.mlp.experts.10.up_proj.weight": "model-00014-of-00101.safetensors", + "model.layers.14.mlp.experts.10.down_proj.weight": "model-00014-of-00101.safetensors", + "model.layers.14.mlp.experts.11.gate_proj.weight": "model-00014-of-00101.safetensors", + "model.layers.14.mlp.experts.11.up_proj.weight": "model-00014-of-00101.safetensors", + "model.layers.14.mlp.experts.11.down_proj.weight": "model-00014-of-00101.safetensors", + "model.layers.14.mlp.experts.12.gate_proj.weight": "model-00014-of-00101.safetensors", + "model.layers.14.mlp.experts.12.up_proj.weight": "model-00014-of-00101.safetensors", + "model.layers.14.mlp.experts.12.down_proj.weight": "model-00014-of-00101.safetensors", + "model.layers.14.mlp.experts.13.gate_proj.weight": "model-00014-of-00101.safetensors", + "model.layers.14.mlp.experts.13.up_proj.weight": "model-00014-of-00101.safetensors", + "model.layers.14.mlp.experts.13.down_proj.weight": "model-00014-of-00101.safetensors", + "model.layers.14.mlp.experts.14.gate_proj.weight": "model-00014-of-00101.safetensors", + "model.layers.14.mlp.experts.14.up_proj.weight": "model-00014-of-00101.safetensors", + "model.layers.14.mlp.experts.14.down_proj.weight": "model-00014-of-00101.safetensors", + "model.layers.14.mlp.experts.15.gate_proj.weight": "model-00014-of-00101.safetensors", + "model.layers.14.mlp.experts.15.up_proj.weight": "model-00014-of-00101.safetensors", + "model.layers.14.mlp.experts.15.down_proj.weight": "model-00014-of-00101.safetensors", + "model.layers.14.mlp.experts.16.gate_proj.weight": "model-00014-of-00101.safetensors", + "model.layers.14.mlp.experts.16.up_proj.weight": "model-00014-of-00101.safetensors", + "model.layers.14.mlp.experts.16.down_proj.weight": "model-00014-of-00101.safetensors", + "model.layers.14.mlp.experts.17.gate_proj.weight": "model-00014-of-00101.safetensors", + "model.layers.14.mlp.experts.17.up_proj.weight": "model-00014-of-00101.safetensors", + "model.layers.14.mlp.experts.17.down_proj.weight": "model-00014-of-00101.safetensors", + "model.layers.14.mlp.experts.18.gate_proj.weight": "model-00014-of-00101.safetensors", + "model.layers.14.mlp.experts.18.up_proj.weight": "model-00014-of-00101.safetensors", + "model.layers.14.mlp.experts.18.down_proj.weight": "model-00014-of-00101.safetensors", + "model.layers.14.mlp.experts.19.gate_proj.weight": "model-00014-of-00101.safetensors", + "model.layers.14.mlp.experts.19.up_proj.weight": "model-00014-of-00101.safetensors", + "model.layers.14.mlp.experts.19.down_proj.weight": "model-00014-of-00101.safetensors", + "model.layers.14.mlp.experts.20.gate_proj.weight": "model-00014-of-00101.safetensors", + "model.layers.14.mlp.experts.20.up_proj.weight": "model-00014-of-00101.safetensors", + "model.layers.14.mlp.experts.20.down_proj.weight": "model-00014-of-00101.safetensors", + "model.layers.14.mlp.experts.21.gate_proj.weight": "model-00014-of-00101.safetensors", + "model.layers.14.mlp.experts.21.up_proj.weight": "model-00014-of-00101.safetensors", + "model.layers.14.mlp.experts.21.down_proj.weight": "model-00014-of-00101.safetensors", + "model.layers.14.mlp.experts.22.gate_proj.weight": "model-00014-of-00101.safetensors", + "model.layers.14.mlp.experts.22.up_proj.weight": "model-00014-of-00101.safetensors", + "model.layers.14.mlp.experts.22.down_proj.weight": "model-00014-of-00101.safetensors", + "model.layers.14.mlp.experts.23.gate_proj.weight": "model-00014-of-00101.safetensors", + "model.layers.14.mlp.experts.23.up_proj.weight": "model-00014-of-00101.safetensors", + "model.layers.14.mlp.experts.23.down_proj.weight": "model-00014-of-00101.safetensors", + "model.layers.14.mlp.experts.24.gate_proj.weight": "model-00014-of-00101.safetensors", + "model.layers.14.mlp.experts.24.up_proj.weight": "model-00014-of-00101.safetensors", + "model.layers.14.mlp.experts.24.down_proj.weight": "model-00014-of-00101.safetensors", + "model.layers.14.mlp.experts.25.gate_proj.weight": "model-00014-of-00101.safetensors", + "model.layers.14.mlp.experts.25.up_proj.weight": "model-00014-of-00101.safetensors", + "model.layers.14.mlp.experts.25.down_proj.weight": "model-00014-of-00101.safetensors", + "model.layers.14.mlp.experts.26.gate_proj.weight": "model-00014-of-00101.safetensors", + "model.layers.14.mlp.experts.26.up_proj.weight": "model-00014-of-00101.safetensors", + "model.layers.14.mlp.experts.26.down_proj.weight": "model-00014-of-00101.safetensors", + "model.layers.14.mlp.experts.27.gate_proj.weight": "model-00014-of-00101.safetensors", + "model.layers.14.mlp.experts.27.up_proj.weight": "model-00014-of-00101.safetensors", + "model.layers.14.mlp.experts.27.down_proj.weight": "model-00014-of-00101.safetensors", + "model.layers.14.mlp.experts.28.gate_proj.weight": "model-00014-of-00101.safetensors", + "model.layers.14.mlp.experts.28.up_proj.weight": "model-00014-of-00101.safetensors", + "model.layers.14.mlp.experts.28.down_proj.weight": "model-00014-of-00101.safetensors", + "model.layers.14.mlp.experts.29.gate_proj.weight": "model-00014-of-00101.safetensors", + "model.layers.14.mlp.experts.29.up_proj.weight": "model-00014-of-00101.safetensors", + "model.layers.14.mlp.experts.29.down_proj.weight": "model-00014-of-00101.safetensors", + "model.layers.14.mlp.experts.30.gate_proj.weight": "model-00014-of-00101.safetensors", + "model.layers.14.mlp.experts.30.up_proj.weight": "model-00014-of-00101.safetensors", + "model.layers.14.mlp.experts.30.down_proj.weight": "model-00014-of-00101.safetensors", + "model.layers.14.mlp.experts.31.gate_proj.weight": "model-00014-of-00101.safetensors", + "model.layers.14.mlp.experts.31.up_proj.weight": "model-00014-of-00101.safetensors", + "model.layers.14.mlp.experts.31.down_proj.weight": "model-00014-of-00101.safetensors", + "model.layers.14.mlp.experts.32.gate_proj.weight": "model-00014-of-00101.safetensors", + "model.layers.14.mlp.experts.32.up_proj.weight": "model-00014-of-00101.safetensors", + "model.layers.14.mlp.experts.32.down_proj.weight": "model-00014-of-00101.safetensors", + "model.layers.14.mlp.experts.33.gate_proj.weight": "model-00014-of-00101.safetensors", + "model.layers.14.mlp.experts.33.up_proj.weight": "model-00014-of-00101.safetensors", + "model.layers.14.mlp.experts.33.down_proj.weight": "model-00014-of-00101.safetensors", + "model.layers.14.mlp.experts.34.gate_proj.weight": "model-00014-of-00101.safetensors", + "model.layers.14.mlp.experts.34.up_proj.weight": "model-00014-of-00101.safetensors", + "model.layers.14.mlp.experts.34.down_proj.weight": "model-00014-of-00101.safetensors", + "model.layers.14.mlp.experts.35.gate_proj.weight": "model-00014-of-00101.safetensors", + "model.layers.14.mlp.experts.35.up_proj.weight": "model-00014-of-00101.safetensors", + "model.layers.14.mlp.experts.35.down_proj.weight": "model-00014-of-00101.safetensors", + "model.layers.14.mlp.experts.36.gate_proj.weight": "model-00014-of-00101.safetensors", + "model.layers.14.mlp.experts.36.up_proj.weight": "model-00014-of-00101.safetensors", + "model.layers.14.mlp.experts.36.down_proj.weight": "model-00014-of-00101.safetensors", + "model.layers.14.mlp.experts.37.gate_proj.weight": "model-00014-of-00101.safetensors", + "model.layers.14.mlp.experts.37.up_proj.weight": "model-00014-of-00101.safetensors", + "model.layers.14.mlp.experts.37.down_proj.weight": "model-00014-of-00101.safetensors", + "model.layers.14.mlp.experts.38.gate_proj.weight": "model-00014-of-00101.safetensors", + "model.layers.14.mlp.experts.38.up_proj.weight": "model-00014-of-00101.safetensors", + "model.layers.14.mlp.experts.38.down_proj.weight": "model-00014-of-00101.safetensors", + "model.layers.14.mlp.experts.39.gate_proj.weight": "model-00014-of-00101.safetensors", + "model.layers.14.mlp.experts.39.up_proj.weight": "model-00014-of-00101.safetensors", + "model.layers.14.mlp.experts.39.down_proj.weight": "model-00014-of-00101.safetensors", + "model.layers.14.mlp.experts.40.gate_proj.weight": "model-00014-of-00101.safetensors", + "model.layers.14.mlp.experts.40.up_proj.weight": "model-00014-of-00101.safetensors", + "model.layers.14.mlp.experts.40.down_proj.weight": "model-00014-of-00101.safetensors", + "model.layers.14.mlp.experts.41.gate_proj.weight": "model-00014-of-00101.safetensors", + "model.layers.14.mlp.experts.41.up_proj.weight": "model-00014-of-00101.safetensors", + "model.layers.14.mlp.experts.41.down_proj.weight": "model-00014-of-00101.safetensors", + "model.layers.14.mlp.experts.42.gate_proj.weight": "model-00014-of-00101.safetensors", + "model.layers.14.mlp.experts.42.up_proj.weight": "model-00014-of-00101.safetensors", + "model.layers.14.mlp.experts.42.down_proj.weight": "model-00014-of-00101.safetensors", + "model.layers.14.mlp.experts.43.gate_proj.weight": "model-00014-of-00101.safetensors", + "model.layers.14.mlp.experts.43.up_proj.weight": "model-00014-of-00101.safetensors", + "model.layers.14.mlp.experts.43.down_proj.weight": "model-00014-of-00101.safetensors", + "model.layers.14.mlp.experts.44.gate_proj.weight": "model-00014-of-00101.safetensors", + "model.layers.14.mlp.experts.44.up_proj.weight": "model-00014-of-00101.safetensors", + "model.layers.14.mlp.experts.44.down_proj.weight": "model-00014-of-00101.safetensors", + "model.layers.14.mlp.experts.45.gate_proj.weight": "model-00014-of-00101.safetensors", + "model.layers.14.mlp.experts.45.up_proj.weight": "model-00014-of-00101.safetensors", + "model.layers.14.mlp.experts.45.down_proj.weight": "model-00014-of-00101.safetensors", + "model.layers.14.mlp.experts.46.gate_proj.weight": "model-00014-of-00101.safetensors", + "model.layers.14.mlp.experts.46.up_proj.weight": "model-00014-of-00101.safetensors", + "model.layers.14.mlp.experts.46.down_proj.weight": "model-00014-of-00101.safetensors", + "model.layers.14.mlp.experts.47.gate_proj.weight": "model-00014-of-00101.safetensors", + "model.layers.14.mlp.experts.47.up_proj.weight": "model-00014-of-00101.safetensors", + "model.layers.14.mlp.experts.47.down_proj.weight": "model-00014-of-00101.safetensors", + "model.layers.14.mlp.experts.48.gate_proj.weight": "model-00014-of-00101.safetensors", + "model.layers.14.mlp.experts.48.up_proj.weight": "model-00014-of-00101.safetensors", + "model.layers.14.mlp.experts.48.down_proj.weight": "model-00014-of-00101.safetensors", + "model.layers.14.mlp.experts.49.gate_proj.weight": "model-00014-of-00101.safetensors", + "model.layers.14.mlp.experts.49.up_proj.weight": "model-00014-of-00101.safetensors", + "model.layers.14.mlp.experts.49.down_proj.weight": "model-00014-of-00101.safetensors", + "model.layers.14.mlp.experts.50.gate_proj.weight": "model-00014-of-00101.safetensors", + "model.layers.14.mlp.experts.50.up_proj.weight": "model-00014-of-00101.safetensors", + "model.layers.14.mlp.experts.50.down_proj.weight": "model-00014-of-00101.safetensors", + "model.layers.14.mlp.experts.51.gate_proj.weight": "model-00014-of-00101.safetensors", + "model.layers.14.mlp.experts.51.up_proj.weight": "model-00014-of-00101.safetensors", + "model.layers.14.mlp.experts.51.down_proj.weight": "model-00014-of-00101.safetensors", + "model.layers.14.mlp.experts.52.gate_proj.weight": "model-00014-of-00101.safetensors", + "model.layers.14.mlp.experts.52.up_proj.weight": "model-00014-of-00101.safetensors", + "model.layers.14.mlp.experts.52.down_proj.weight": "model-00014-of-00101.safetensors", + "model.layers.14.mlp.experts.53.gate_proj.weight": "model-00014-of-00101.safetensors", + "model.layers.14.mlp.experts.53.up_proj.weight": "model-00014-of-00101.safetensors", + "model.layers.14.mlp.experts.53.down_proj.weight": "model-00014-of-00101.safetensors", + "model.layers.14.mlp.experts.54.gate_proj.weight": "model-00014-of-00101.safetensors", + "model.layers.14.mlp.experts.54.up_proj.weight": "model-00014-of-00101.safetensors", + "model.layers.14.mlp.experts.54.down_proj.weight": "model-00014-of-00101.safetensors", + "model.layers.14.mlp.experts.55.gate_proj.weight": "model-00014-of-00101.safetensors", + "model.layers.14.mlp.experts.55.up_proj.weight": "model-00014-of-00101.safetensors", + "model.layers.14.mlp.experts.55.down_proj.weight": "model-00014-of-00101.safetensors", + "model.layers.14.mlp.experts.56.gate_proj.weight": "model-00014-of-00101.safetensors", + "model.layers.14.mlp.experts.56.up_proj.weight": "model-00014-of-00101.safetensors", + "model.layers.14.mlp.experts.56.down_proj.weight": "model-00014-of-00101.safetensors", + "model.layers.14.mlp.experts.57.gate_proj.weight": "model-00014-of-00101.safetensors", + "model.layers.14.mlp.experts.57.up_proj.weight": "model-00014-of-00101.safetensors", + "model.layers.14.mlp.experts.57.down_proj.weight": "model-00014-of-00101.safetensors", + "model.layers.14.mlp.experts.58.gate_proj.weight": "model-00014-of-00101.safetensors", + "model.layers.14.mlp.experts.58.up_proj.weight": "model-00014-of-00101.safetensors", + "model.layers.14.mlp.experts.58.down_proj.weight": "model-00014-of-00101.safetensors", + "model.layers.14.mlp.experts.59.gate_proj.weight": "model-00014-of-00101.safetensors", + "model.layers.14.mlp.experts.59.up_proj.weight": "model-00014-of-00101.safetensors", + "model.layers.14.mlp.experts.59.down_proj.weight": "model-00014-of-00101.safetensors", + "model.layers.14.mlp.experts.60.gate_proj.weight": "model-00014-of-00101.safetensors", + "model.layers.14.mlp.experts.60.up_proj.weight": "model-00014-of-00101.safetensors", + "model.layers.14.mlp.experts.60.down_proj.weight": "model-00014-of-00101.safetensors", + "model.layers.14.mlp.experts.61.gate_proj.weight": "model-00014-of-00101.safetensors", + "model.layers.14.mlp.experts.61.up_proj.weight": "model-00014-of-00101.safetensors", + "model.layers.14.mlp.experts.61.down_proj.weight": "model-00014-of-00101.safetensors", + "model.layers.14.mlp.experts.62.gate_proj.weight": "model-00014-of-00101.safetensors", + "model.layers.14.mlp.experts.62.up_proj.weight": "model-00014-of-00101.safetensors", + "model.layers.14.mlp.experts.62.down_proj.weight": "model-00014-of-00101.safetensors", + "model.layers.14.mlp.experts.63.gate_proj.weight": "model-00014-of-00101.safetensors", + "model.layers.14.mlp.experts.63.up_proj.weight": "model-00014-of-00101.safetensors", + "model.layers.14.mlp.experts.63.down_proj.weight": "model-00014-of-00101.safetensors", + "model.layers.14.mlp.experts.64.gate_proj.weight": "model-00014-of-00101.safetensors", + "model.layers.14.mlp.experts.64.up_proj.weight": "model-00014-of-00101.safetensors", + "model.layers.14.mlp.experts.64.down_proj.weight": "model-00014-of-00101.safetensors", + "model.layers.14.mlp.experts.65.gate_proj.weight": "model-00014-of-00101.safetensors", + "model.layers.14.mlp.experts.65.up_proj.weight": "model-00014-of-00101.safetensors", + "model.layers.14.mlp.experts.65.down_proj.weight": "model-00014-of-00101.safetensors", + "model.layers.14.mlp.experts.66.gate_proj.weight": "model-00014-of-00101.safetensors", + "model.layers.14.mlp.experts.66.up_proj.weight": "model-00014-of-00101.safetensors", + "model.layers.14.mlp.experts.66.down_proj.weight": "model-00014-of-00101.safetensors", + "model.layers.14.mlp.experts.67.gate_proj.weight": "model-00014-of-00101.safetensors", + "model.layers.14.mlp.experts.67.up_proj.weight": "model-00014-of-00101.safetensors", + "model.layers.14.mlp.experts.67.down_proj.weight": "model-00014-of-00101.safetensors", + "model.layers.14.mlp.experts.68.gate_proj.weight": "model-00014-of-00101.safetensors", + "model.layers.14.mlp.experts.68.up_proj.weight": "model-00014-of-00101.safetensors", + "model.layers.14.mlp.experts.68.down_proj.weight": "model-00014-of-00101.safetensors", + "model.layers.14.mlp.experts.69.gate_proj.weight": "model-00014-of-00101.safetensors", + "model.layers.14.mlp.experts.69.up_proj.weight": "model-00014-of-00101.safetensors", + "model.layers.14.mlp.experts.69.down_proj.weight": "model-00014-of-00101.safetensors", + "model.layers.14.mlp.experts.70.gate_proj.weight": "model-00014-of-00101.safetensors", + "model.layers.14.mlp.experts.70.up_proj.weight": "model-00014-of-00101.safetensors", + "model.layers.14.mlp.experts.70.down_proj.weight": "model-00014-of-00101.safetensors", + "model.layers.14.mlp.experts.71.gate_proj.weight": "model-00014-of-00101.safetensors", + "model.layers.14.mlp.experts.71.up_proj.weight": "model-00014-of-00101.safetensors", + "model.layers.14.mlp.experts.71.down_proj.weight": "model-00014-of-00101.safetensors", + "model.layers.14.mlp.experts.72.gate_proj.weight": "model-00014-of-00101.safetensors", + "model.layers.14.mlp.experts.72.up_proj.weight": "model-00014-of-00101.safetensors", + "model.layers.14.mlp.experts.72.down_proj.weight": "model-00014-of-00101.safetensors", + "model.layers.14.mlp.experts.73.gate_proj.weight": "model-00014-of-00101.safetensors", + "model.layers.14.mlp.experts.73.up_proj.weight": "model-00014-of-00101.safetensors", + "model.layers.14.mlp.experts.73.down_proj.weight": "model-00014-of-00101.safetensors", + "model.layers.14.mlp.experts.74.gate_proj.weight": "model-00014-of-00101.safetensors", + "model.layers.14.mlp.experts.74.up_proj.weight": "model-00014-of-00101.safetensors", + "model.layers.14.mlp.experts.74.down_proj.weight": "model-00014-of-00101.safetensors", + "model.layers.14.mlp.experts.75.gate_proj.weight": "model-00014-of-00101.safetensors", + "model.layers.14.mlp.experts.75.up_proj.weight": "model-00014-of-00101.safetensors", + "model.layers.14.mlp.experts.75.down_proj.weight": "model-00014-of-00101.safetensors", + "model.layers.14.mlp.experts.76.gate_proj.weight": "model-00014-of-00101.safetensors", + "model.layers.14.mlp.experts.76.up_proj.weight": "model-00014-of-00101.safetensors", + "model.layers.14.mlp.experts.76.down_proj.weight": "model-00014-of-00101.safetensors", + "model.layers.14.mlp.experts.77.gate_proj.weight": "model-00014-of-00101.safetensors", + "model.layers.14.mlp.experts.77.up_proj.weight": "model-00014-of-00101.safetensors", + "model.layers.14.mlp.experts.77.down_proj.weight": "model-00014-of-00101.safetensors", + "model.layers.14.mlp.experts.78.gate_proj.weight": "model-00014-of-00101.safetensors", + "model.layers.14.mlp.experts.78.up_proj.weight": "model-00014-of-00101.safetensors", + "model.layers.14.mlp.experts.78.down_proj.weight": "model-00014-of-00101.safetensors", + "model.layers.14.mlp.experts.79.gate_proj.weight": "model-00014-of-00101.safetensors", + "model.layers.14.mlp.experts.79.up_proj.weight": "model-00014-of-00101.safetensors", + "model.layers.14.mlp.experts.79.down_proj.weight": "model-00014-of-00101.safetensors", + "model.layers.14.mlp.experts.80.gate_proj.weight": "model-00014-of-00101.safetensors", + "model.layers.14.mlp.experts.80.up_proj.weight": "model-00014-of-00101.safetensors", + "model.layers.14.mlp.experts.80.down_proj.weight": "model-00014-of-00101.safetensors", + "model.layers.14.mlp.experts.81.gate_proj.weight": "model-00014-of-00101.safetensors", + "model.layers.14.mlp.experts.81.up_proj.weight": "model-00014-of-00101.safetensors", + "model.layers.14.mlp.experts.81.down_proj.weight": "model-00014-of-00101.safetensors", + "model.layers.14.mlp.experts.82.gate_proj.weight": "model-00014-of-00101.safetensors", + "model.layers.14.mlp.experts.82.up_proj.weight": "model-00014-of-00101.safetensors", + "model.layers.14.mlp.experts.82.down_proj.weight": "model-00014-of-00101.safetensors", + "model.layers.14.mlp.experts.83.gate_proj.weight": "model-00014-of-00101.safetensors", + "model.layers.14.mlp.experts.83.up_proj.weight": "model-00014-of-00101.safetensors", + "model.layers.14.mlp.experts.83.down_proj.weight": "model-00014-of-00101.safetensors", + "model.layers.14.mlp.experts.84.gate_proj.weight": "model-00014-of-00101.safetensors", + "model.layers.14.mlp.experts.84.up_proj.weight": "model-00014-of-00101.safetensors", + "model.layers.14.mlp.experts.84.down_proj.weight": "model-00014-of-00101.safetensors", + "model.layers.14.mlp.experts.85.gate_proj.weight": "model-00014-of-00101.safetensors", + "model.layers.14.mlp.experts.85.up_proj.weight": "model-00014-of-00101.safetensors", + "model.layers.14.mlp.experts.85.down_proj.weight": "model-00014-of-00101.safetensors", + "model.layers.14.mlp.experts.86.gate_proj.weight": "model-00014-of-00101.safetensors", + "model.layers.14.mlp.experts.86.up_proj.weight": "model-00014-of-00101.safetensors", + "model.layers.14.mlp.experts.86.down_proj.weight": "model-00014-of-00101.safetensors", + "model.layers.14.mlp.experts.87.gate_proj.weight": "model-00014-of-00101.safetensors", + "model.layers.14.mlp.experts.87.up_proj.weight": "model-00014-of-00101.safetensors", + "model.layers.14.mlp.experts.87.down_proj.weight": "model-00014-of-00101.safetensors", + "model.layers.14.mlp.experts.88.gate_proj.weight": "model-00014-of-00101.safetensors", + "model.layers.14.mlp.experts.88.up_proj.weight": "model-00014-of-00101.safetensors", + "model.layers.14.mlp.experts.88.down_proj.weight": "model-00014-of-00101.safetensors", + "model.layers.14.mlp.experts.89.gate_proj.weight": "model-00014-of-00101.safetensors", + "model.layers.14.mlp.experts.89.up_proj.weight": "model-00014-of-00101.safetensors", + "model.layers.14.mlp.experts.89.down_proj.weight": "model-00014-of-00101.safetensors", + "model.layers.14.mlp.experts.90.gate_proj.weight": "model-00014-of-00101.safetensors", + "model.layers.14.mlp.experts.90.up_proj.weight": "model-00014-of-00101.safetensors", + "model.layers.14.mlp.experts.90.down_proj.weight": "model-00014-of-00101.safetensors", + "model.layers.14.mlp.experts.91.gate_proj.weight": "model-00014-of-00101.safetensors", + "model.layers.14.mlp.experts.91.up_proj.weight": "model-00014-of-00101.safetensors", + "model.layers.14.mlp.experts.91.down_proj.weight": "model-00014-of-00101.safetensors", + "model.layers.14.mlp.experts.92.gate_proj.weight": "model-00014-of-00101.safetensors", + "model.layers.14.mlp.experts.92.up_proj.weight": "model-00014-of-00101.safetensors", + "model.layers.14.mlp.experts.92.down_proj.weight": "model-00014-of-00101.safetensors", + "model.layers.14.mlp.experts.93.gate_proj.weight": "model-00014-of-00101.safetensors", + "model.layers.14.mlp.experts.93.up_proj.weight": "model-00014-of-00101.safetensors", + "model.layers.14.mlp.experts.93.down_proj.weight": "model-00014-of-00101.safetensors", + "model.layers.14.mlp.experts.94.gate_proj.weight": "model-00014-of-00101.safetensors", + "model.layers.14.mlp.experts.94.up_proj.weight": "model-00014-of-00101.safetensors", + "model.layers.14.mlp.experts.94.down_proj.weight": "model-00014-of-00101.safetensors", + "model.layers.14.mlp.experts.95.gate_proj.weight": "model-00014-of-00101.safetensors", + "model.layers.14.mlp.experts.95.up_proj.weight": "model-00014-of-00101.safetensors", + "model.layers.14.mlp.experts.95.down_proj.weight": "model-00014-of-00101.safetensors", + "model.layers.14.mlp.experts.96.gate_proj.weight": "model-00014-of-00101.safetensors", + "model.layers.14.mlp.experts.96.up_proj.weight": "model-00014-of-00101.safetensors", + "model.layers.14.mlp.experts.96.down_proj.weight": "model-00014-of-00101.safetensors", + "model.layers.14.mlp.experts.97.gate_proj.weight": "model-00014-of-00101.safetensors", + "model.layers.14.mlp.experts.97.up_proj.weight": "model-00014-of-00101.safetensors", + "model.layers.14.mlp.experts.97.down_proj.weight": "model-00014-of-00101.safetensors", + "model.layers.14.mlp.experts.98.gate_proj.weight": "model-00014-of-00101.safetensors", + "model.layers.14.mlp.experts.98.up_proj.weight": "model-00014-of-00101.safetensors", + "model.layers.14.mlp.experts.98.down_proj.weight": "model-00014-of-00101.safetensors", + "model.layers.14.mlp.experts.99.gate_proj.weight": "model-00014-of-00101.safetensors", + "model.layers.14.mlp.experts.99.up_proj.weight": "model-00014-of-00101.safetensors", + "model.layers.14.mlp.experts.99.down_proj.weight": "model-00014-of-00101.safetensors", + "model.layers.14.mlp.experts.100.gate_proj.weight": "model-00014-of-00101.safetensors", + "model.layers.14.mlp.experts.100.up_proj.weight": "model-00014-of-00101.safetensors", + "model.layers.14.mlp.experts.100.down_proj.weight": "model-00014-of-00101.safetensors", + "model.layers.14.mlp.experts.101.gate_proj.weight": "model-00014-of-00101.safetensors", + "model.layers.14.mlp.experts.101.up_proj.weight": "model-00014-of-00101.safetensors", + "model.layers.14.mlp.experts.101.down_proj.weight": "model-00014-of-00101.safetensors", + "model.layers.14.mlp.experts.102.gate_proj.weight": "model-00014-of-00101.safetensors", + "model.layers.14.mlp.experts.102.up_proj.weight": "model-00014-of-00101.safetensors", + "model.layers.14.mlp.experts.102.down_proj.weight": "model-00014-of-00101.safetensors", + "model.layers.14.mlp.experts.103.gate_proj.weight": "model-00014-of-00101.safetensors", + "model.layers.14.mlp.experts.103.up_proj.weight": "model-00014-of-00101.safetensors", + "model.layers.14.mlp.experts.103.down_proj.weight": "model-00014-of-00101.safetensors", + "model.layers.14.mlp.experts.104.gate_proj.weight": "model-00014-of-00101.safetensors", + "model.layers.14.mlp.experts.104.up_proj.weight": "model-00014-of-00101.safetensors", + "model.layers.14.mlp.experts.104.down_proj.weight": "model-00014-of-00101.safetensors", + "model.layers.14.mlp.experts.105.gate_proj.weight": "model-00014-of-00101.safetensors", + "model.layers.14.mlp.experts.105.up_proj.weight": "model-00014-of-00101.safetensors", + "model.layers.14.mlp.experts.105.down_proj.weight": "model-00014-of-00101.safetensors", + "model.layers.14.mlp.experts.106.gate_proj.weight": "model-00014-of-00101.safetensors", + "model.layers.14.mlp.experts.106.up_proj.weight": "model-00014-of-00101.safetensors", + "model.layers.14.mlp.experts.106.down_proj.weight": "model-00014-of-00101.safetensors", + "model.layers.14.mlp.experts.107.gate_proj.weight": "model-00014-of-00101.safetensors", + "model.layers.14.mlp.experts.107.up_proj.weight": "model-00014-of-00101.safetensors", + "model.layers.14.mlp.experts.107.down_proj.weight": "model-00014-of-00101.safetensors", + "model.layers.14.mlp.experts.108.gate_proj.weight": "model-00014-of-00101.safetensors", + "model.layers.14.mlp.experts.108.up_proj.weight": "model-00014-of-00101.safetensors", + "model.layers.14.mlp.experts.108.down_proj.weight": "model-00014-of-00101.safetensors", + "model.layers.14.mlp.experts.109.gate_proj.weight": "model-00014-of-00101.safetensors", + "model.layers.14.mlp.experts.109.up_proj.weight": "model-00014-of-00101.safetensors", + "model.layers.14.mlp.experts.109.down_proj.weight": "model-00014-of-00101.safetensors", + "model.layers.14.mlp.experts.110.gate_proj.weight": "model-00014-of-00101.safetensors", + "model.layers.14.mlp.experts.110.up_proj.weight": "model-00014-of-00101.safetensors", + "model.layers.14.mlp.experts.110.down_proj.weight": "model-00014-of-00101.safetensors", + "model.layers.14.mlp.experts.111.gate_proj.weight": "model-00014-of-00101.safetensors", + "model.layers.14.mlp.experts.111.up_proj.weight": "model-00014-of-00101.safetensors", + "model.layers.14.mlp.experts.111.down_proj.weight": "model-00014-of-00101.safetensors", + "model.layers.14.mlp.experts.112.gate_proj.weight": "model-00014-of-00101.safetensors", + "model.layers.14.mlp.experts.112.up_proj.weight": "model-00014-of-00101.safetensors", + "model.layers.14.mlp.experts.112.down_proj.weight": "model-00014-of-00101.safetensors", + "model.layers.14.mlp.experts.113.gate_proj.weight": "model-00014-of-00101.safetensors", + "model.layers.14.mlp.experts.113.up_proj.weight": "model-00014-of-00101.safetensors", + "model.layers.14.mlp.experts.113.down_proj.weight": "model-00014-of-00101.safetensors", + "model.layers.14.mlp.experts.114.gate_proj.weight": "model-00014-of-00101.safetensors", + "model.layers.14.mlp.experts.114.up_proj.weight": "model-00015-of-00101.safetensors", + "model.layers.14.mlp.experts.114.down_proj.weight": "model-00015-of-00101.safetensors", + "model.layers.14.mlp.experts.115.gate_proj.weight": "model-00015-of-00101.safetensors", + "model.layers.14.mlp.experts.115.up_proj.weight": "model-00015-of-00101.safetensors", + "model.layers.14.mlp.experts.115.down_proj.weight": "model-00015-of-00101.safetensors", + "model.layers.14.mlp.experts.116.gate_proj.weight": "model-00015-of-00101.safetensors", + "model.layers.14.mlp.experts.116.up_proj.weight": "model-00015-of-00101.safetensors", + "model.layers.14.mlp.experts.116.down_proj.weight": "model-00015-of-00101.safetensors", + "model.layers.14.mlp.experts.117.gate_proj.weight": "model-00015-of-00101.safetensors", + "model.layers.14.mlp.experts.117.up_proj.weight": "model-00015-of-00101.safetensors", + "model.layers.14.mlp.experts.117.down_proj.weight": "model-00015-of-00101.safetensors", + "model.layers.14.mlp.experts.118.gate_proj.weight": "model-00015-of-00101.safetensors", + "model.layers.14.mlp.experts.118.up_proj.weight": "model-00015-of-00101.safetensors", + "model.layers.14.mlp.experts.118.down_proj.weight": "model-00015-of-00101.safetensors", + "model.layers.14.mlp.experts.119.gate_proj.weight": "model-00015-of-00101.safetensors", + "model.layers.14.mlp.experts.119.up_proj.weight": "model-00015-of-00101.safetensors", + "model.layers.14.mlp.experts.119.down_proj.weight": "model-00015-of-00101.safetensors", + "model.layers.14.mlp.gate.weight": "model-00015-of-00101.safetensors", + "model.layers.14.mlp.gate.e_score_correction_bias": "model-00015-of-00101.safetensors", + "model.layers.14.mlp.shared_experts.gate_proj.weight": "model-00015-of-00101.safetensors", + "model.layers.14.mlp.shared_experts.up_proj.weight": "model-00015-of-00101.safetensors", + "model.layers.14.mlp.shared_experts.down_proj.weight": "model-00015-of-00101.safetensors", + "model.layers.14.input_layernorm.weight": "model-00015-of-00101.safetensors", + "model.layers.14.post_attention_layernorm.weight": "model-00015-of-00101.safetensors", + "model.layers.15.self_attn.q_proj.weight": "model-00015-of-00101.safetensors", + "model.layers.15.self_attn.q_proj.bias": "model-00015-of-00101.safetensors", + "model.layers.15.self_attn.k_proj.weight": "model-00015-of-00101.safetensors", + "model.layers.15.self_attn.k_proj.bias": "model-00015-of-00101.safetensors", + "model.layers.15.self_attn.v_proj.weight": "model-00015-of-00101.safetensors", + "model.layers.15.self_attn.v_proj.bias": "model-00015-of-00101.safetensors", + "model.layers.15.self_attn.o_proj.weight": "model-00015-of-00101.safetensors", + "model.layers.15.self_attn.q_norm.weight": "model-00015-of-00101.safetensors", + "model.layers.15.self_attn.k_norm.weight": "model-00015-of-00101.safetensors", + "model.layers.15.mlp.experts.0.gate_proj.weight": "model-00015-of-00101.safetensors", + "model.layers.15.mlp.experts.0.up_proj.weight": "model-00015-of-00101.safetensors", + "model.layers.15.mlp.experts.0.down_proj.weight": "model-00015-of-00101.safetensors", + "model.layers.15.mlp.experts.1.gate_proj.weight": "model-00015-of-00101.safetensors", + "model.layers.15.mlp.experts.1.up_proj.weight": "model-00015-of-00101.safetensors", + "model.layers.15.mlp.experts.1.down_proj.weight": "model-00015-of-00101.safetensors", + "model.layers.15.mlp.experts.2.gate_proj.weight": "model-00015-of-00101.safetensors", + "model.layers.15.mlp.experts.2.up_proj.weight": "model-00015-of-00101.safetensors", + "model.layers.15.mlp.experts.2.down_proj.weight": "model-00015-of-00101.safetensors", + "model.layers.15.mlp.experts.3.gate_proj.weight": "model-00015-of-00101.safetensors", + "model.layers.15.mlp.experts.3.up_proj.weight": "model-00015-of-00101.safetensors", + "model.layers.15.mlp.experts.3.down_proj.weight": "model-00015-of-00101.safetensors", + "model.layers.15.mlp.experts.4.gate_proj.weight": "model-00015-of-00101.safetensors", + "model.layers.15.mlp.experts.4.up_proj.weight": "model-00015-of-00101.safetensors", + "model.layers.15.mlp.experts.4.down_proj.weight": "model-00015-of-00101.safetensors", + "model.layers.15.mlp.experts.5.gate_proj.weight": "model-00015-of-00101.safetensors", + "model.layers.15.mlp.experts.5.up_proj.weight": "model-00015-of-00101.safetensors", + "model.layers.15.mlp.experts.5.down_proj.weight": "model-00015-of-00101.safetensors", + "model.layers.15.mlp.experts.6.gate_proj.weight": "model-00015-of-00101.safetensors", + "model.layers.15.mlp.experts.6.up_proj.weight": "model-00015-of-00101.safetensors", + "model.layers.15.mlp.experts.6.down_proj.weight": "model-00015-of-00101.safetensors", + "model.layers.15.mlp.experts.7.gate_proj.weight": "model-00015-of-00101.safetensors", + "model.layers.15.mlp.experts.7.up_proj.weight": "model-00015-of-00101.safetensors", + "model.layers.15.mlp.experts.7.down_proj.weight": "model-00015-of-00101.safetensors", + "model.layers.15.mlp.experts.8.gate_proj.weight": "model-00015-of-00101.safetensors", + "model.layers.15.mlp.experts.8.up_proj.weight": "model-00015-of-00101.safetensors", + "model.layers.15.mlp.experts.8.down_proj.weight": "model-00015-of-00101.safetensors", + "model.layers.15.mlp.experts.9.gate_proj.weight": "model-00015-of-00101.safetensors", + "model.layers.15.mlp.experts.9.up_proj.weight": "model-00015-of-00101.safetensors", + "model.layers.15.mlp.experts.9.down_proj.weight": "model-00015-of-00101.safetensors", + "model.layers.15.mlp.experts.10.gate_proj.weight": "model-00015-of-00101.safetensors", + "model.layers.15.mlp.experts.10.up_proj.weight": "model-00015-of-00101.safetensors", + "model.layers.15.mlp.experts.10.down_proj.weight": "model-00015-of-00101.safetensors", + "model.layers.15.mlp.experts.11.gate_proj.weight": "model-00015-of-00101.safetensors", + "model.layers.15.mlp.experts.11.up_proj.weight": "model-00015-of-00101.safetensors", + "model.layers.15.mlp.experts.11.down_proj.weight": "model-00015-of-00101.safetensors", + "model.layers.15.mlp.experts.12.gate_proj.weight": "model-00015-of-00101.safetensors", + "model.layers.15.mlp.experts.12.up_proj.weight": "model-00015-of-00101.safetensors", + "model.layers.15.mlp.experts.12.down_proj.weight": "model-00015-of-00101.safetensors", + "model.layers.15.mlp.experts.13.gate_proj.weight": "model-00015-of-00101.safetensors", + "model.layers.15.mlp.experts.13.up_proj.weight": "model-00015-of-00101.safetensors", + "model.layers.15.mlp.experts.13.down_proj.weight": "model-00015-of-00101.safetensors", + "model.layers.15.mlp.experts.14.gate_proj.weight": "model-00015-of-00101.safetensors", + "model.layers.15.mlp.experts.14.up_proj.weight": "model-00015-of-00101.safetensors", + "model.layers.15.mlp.experts.14.down_proj.weight": "model-00015-of-00101.safetensors", + "model.layers.15.mlp.experts.15.gate_proj.weight": "model-00015-of-00101.safetensors", + "model.layers.15.mlp.experts.15.up_proj.weight": "model-00015-of-00101.safetensors", + "model.layers.15.mlp.experts.15.down_proj.weight": "model-00015-of-00101.safetensors", + "model.layers.15.mlp.experts.16.gate_proj.weight": "model-00015-of-00101.safetensors", + "model.layers.15.mlp.experts.16.up_proj.weight": "model-00015-of-00101.safetensors", + "model.layers.15.mlp.experts.16.down_proj.weight": "model-00015-of-00101.safetensors", + "model.layers.15.mlp.experts.17.gate_proj.weight": "model-00015-of-00101.safetensors", + "model.layers.15.mlp.experts.17.up_proj.weight": "model-00015-of-00101.safetensors", + "model.layers.15.mlp.experts.17.down_proj.weight": "model-00015-of-00101.safetensors", + "model.layers.15.mlp.experts.18.gate_proj.weight": "model-00015-of-00101.safetensors", + "model.layers.15.mlp.experts.18.up_proj.weight": "model-00015-of-00101.safetensors", + "model.layers.15.mlp.experts.18.down_proj.weight": "model-00015-of-00101.safetensors", + "model.layers.15.mlp.experts.19.gate_proj.weight": "model-00015-of-00101.safetensors", + "model.layers.15.mlp.experts.19.up_proj.weight": "model-00015-of-00101.safetensors", + "model.layers.15.mlp.experts.19.down_proj.weight": "model-00015-of-00101.safetensors", + "model.layers.15.mlp.experts.20.gate_proj.weight": "model-00015-of-00101.safetensors", + "model.layers.15.mlp.experts.20.up_proj.weight": "model-00015-of-00101.safetensors", + "model.layers.15.mlp.experts.20.down_proj.weight": "model-00015-of-00101.safetensors", + "model.layers.15.mlp.experts.21.gate_proj.weight": "model-00015-of-00101.safetensors", + "model.layers.15.mlp.experts.21.up_proj.weight": "model-00015-of-00101.safetensors", + "model.layers.15.mlp.experts.21.down_proj.weight": "model-00015-of-00101.safetensors", + "model.layers.15.mlp.experts.22.gate_proj.weight": "model-00015-of-00101.safetensors", + "model.layers.15.mlp.experts.22.up_proj.weight": "model-00015-of-00101.safetensors", + "model.layers.15.mlp.experts.22.down_proj.weight": "model-00015-of-00101.safetensors", + "model.layers.15.mlp.experts.23.gate_proj.weight": "model-00015-of-00101.safetensors", + "model.layers.15.mlp.experts.23.up_proj.weight": "model-00015-of-00101.safetensors", + "model.layers.15.mlp.experts.23.down_proj.weight": "model-00015-of-00101.safetensors", + "model.layers.15.mlp.experts.24.gate_proj.weight": "model-00015-of-00101.safetensors", + "model.layers.15.mlp.experts.24.up_proj.weight": "model-00015-of-00101.safetensors", + "model.layers.15.mlp.experts.24.down_proj.weight": "model-00015-of-00101.safetensors", + "model.layers.15.mlp.experts.25.gate_proj.weight": "model-00015-of-00101.safetensors", + "model.layers.15.mlp.experts.25.up_proj.weight": "model-00015-of-00101.safetensors", + "model.layers.15.mlp.experts.25.down_proj.weight": "model-00015-of-00101.safetensors", + "model.layers.15.mlp.experts.26.gate_proj.weight": "model-00015-of-00101.safetensors", + "model.layers.15.mlp.experts.26.up_proj.weight": "model-00015-of-00101.safetensors", + "model.layers.15.mlp.experts.26.down_proj.weight": "model-00015-of-00101.safetensors", + "model.layers.15.mlp.experts.27.gate_proj.weight": "model-00015-of-00101.safetensors", + "model.layers.15.mlp.experts.27.up_proj.weight": "model-00015-of-00101.safetensors", + "model.layers.15.mlp.experts.27.down_proj.weight": "model-00015-of-00101.safetensors", + "model.layers.15.mlp.experts.28.gate_proj.weight": "model-00015-of-00101.safetensors", + "model.layers.15.mlp.experts.28.up_proj.weight": "model-00015-of-00101.safetensors", + "model.layers.15.mlp.experts.28.down_proj.weight": "model-00015-of-00101.safetensors", + "model.layers.15.mlp.experts.29.gate_proj.weight": "model-00015-of-00101.safetensors", + "model.layers.15.mlp.experts.29.up_proj.weight": "model-00015-of-00101.safetensors", + "model.layers.15.mlp.experts.29.down_proj.weight": "model-00015-of-00101.safetensors", + "model.layers.15.mlp.experts.30.gate_proj.weight": "model-00015-of-00101.safetensors", + "model.layers.15.mlp.experts.30.up_proj.weight": "model-00015-of-00101.safetensors", + "model.layers.15.mlp.experts.30.down_proj.weight": "model-00015-of-00101.safetensors", + "model.layers.15.mlp.experts.31.gate_proj.weight": "model-00015-of-00101.safetensors", + "model.layers.15.mlp.experts.31.up_proj.weight": "model-00015-of-00101.safetensors", + "model.layers.15.mlp.experts.31.down_proj.weight": "model-00015-of-00101.safetensors", + "model.layers.15.mlp.experts.32.gate_proj.weight": "model-00015-of-00101.safetensors", + "model.layers.15.mlp.experts.32.up_proj.weight": "model-00015-of-00101.safetensors", + "model.layers.15.mlp.experts.32.down_proj.weight": "model-00015-of-00101.safetensors", + "model.layers.15.mlp.experts.33.gate_proj.weight": "model-00015-of-00101.safetensors", + "model.layers.15.mlp.experts.33.up_proj.weight": "model-00015-of-00101.safetensors", + "model.layers.15.mlp.experts.33.down_proj.weight": "model-00015-of-00101.safetensors", + "model.layers.15.mlp.experts.34.gate_proj.weight": "model-00015-of-00101.safetensors", + "model.layers.15.mlp.experts.34.up_proj.weight": "model-00015-of-00101.safetensors", + "model.layers.15.mlp.experts.34.down_proj.weight": "model-00015-of-00101.safetensors", + "model.layers.15.mlp.experts.35.gate_proj.weight": "model-00015-of-00101.safetensors", + "model.layers.15.mlp.experts.35.up_proj.weight": "model-00015-of-00101.safetensors", + "model.layers.15.mlp.experts.35.down_proj.weight": "model-00015-of-00101.safetensors", + "model.layers.15.mlp.experts.36.gate_proj.weight": "model-00015-of-00101.safetensors", + "model.layers.15.mlp.experts.36.up_proj.weight": "model-00015-of-00101.safetensors", + "model.layers.15.mlp.experts.36.down_proj.weight": "model-00015-of-00101.safetensors", + "model.layers.15.mlp.experts.37.gate_proj.weight": "model-00015-of-00101.safetensors", + "model.layers.15.mlp.experts.37.up_proj.weight": "model-00015-of-00101.safetensors", + "model.layers.15.mlp.experts.37.down_proj.weight": "model-00015-of-00101.safetensors", + "model.layers.15.mlp.experts.38.gate_proj.weight": "model-00015-of-00101.safetensors", + "model.layers.15.mlp.experts.38.up_proj.weight": "model-00015-of-00101.safetensors", + "model.layers.15.mlp.experts.38.down_proj.weight": "model-00015-of-00101.safetensors", + "model.layers.15.mlp.experts.39.gate_proj.weight": "model-00015-of-00101.safetensors", + "model.layers.15.mlp.experts.39.up_proj.weight": "model-00015-of-00101.safetensors", + "model.layers.15.mlp.experts.39.down_proj.weight": "model-00015-of-00101.safetensors", + "model.layers.15.mlp.experts.40.gate_proj.weight": "model-00015-of-00101.safetensors", + "model.layers.15.mlp.experts.40.up_proj.weight": "model-00015-of-00101.safetensors", + "model.layers.15.mlp.experts.40.down_proj.weight": "model-00015-of-00101.safetensors", + "model.layers.15.mlp.experts.41.gate_proj.weight": "model-00015-of-00101.safetensors", + "model.layers.15.mlp.experts.41.up_proj.weight": "model-00015-of-00101.safetensors", + "model.layers.15.mlp.experts.41.down_proj.weight": "model-00015-of-00101.safetensors", + "model.layers.15.mlp.experts.42.gate_proj.weight": "model-00015-of-00101.safetensors", + "model.layers.15.mlp.experts.42.up_proj.weight": "model-00015-of-00101.safetensors", + "model.layers.15.mlp.experts.42.down_proj.weight": "model-00015-of-00101.safetensors", + "model.layers.15.mlp.experts.43.gate_proj.weight": "model-00015-of-00101.safetensors", + "model.layers.15.mlp.experts.43.up_proj.weight": "model-00015-of-00101.safetensors", + "model.layers.15.mlp.experts.43.down_proj.weight": "model-00015-of-00101.safetensors", + "model.layers.15.mlp.experts.44.gate_proj.weight": "model-00015-of-00101.safetensors", + "model.layers.15.mlp.experts.44.up_proj.weight": "model-00015-of-00101.safetensors", + "model.layers.15.mlp.experts.44.down_proj.weight": "model-00015-of-00101.safetensors", + "model.layers.15.mlp.experts.45.gate_proj.weight": "model-00015-of-00101.safetensors", + "model.layers.15.mlp.experts.45.up_proj.weight": "model-00015-of-00101.safetensors", + "model.layers.15.mlp.experts.45.down_proj.weight": "model-00015-of-00101.safetensors", + "model.layers.15.mlp.experts.46.gate_proj.weight": "model-00015-of-00101.safetensors", + "model.layers.15.mlp.experts.46.up_proj.weight": "model-00015-of-00101.safetensors", + "model.layers.15.mlp.experts.46.down_proj.weight": "model-00015-of-00101.safetensors", + "model.layers.15.mlp.experts.47.gate_proj.weight": "model-00015-of-00101.safetensors", + "model.layers.15.mlp.experts.47.up_proj.weight": "model-00015-of-00101.safetensors", + "model.layers.15.mlp.experts.47.down_proj.weight": "model-00015-of-00101.safetensors", + "model.layers.15.mlp.experts.48.gate_proj.weight": "model-00015-of-00101.safetensors", + "model.layers.15.mlp.experts.48.up_proj.weight": "model-00015-of-00101.safetensors", + "model.layers.15.mlp.experts.48.down_proj.weight": "model-00015-of-00101.safetensors", + "model.layers.15.mlp.experts.49.gate_proj.weight": "model-00015-of-00101.safetensors", + "model.layers.15.mlp.experts.49.up_proj.weight": "model-00015-of-00101.safetensors", + "model.layers.15.mlp.experts.49.down_proj.weight": "model-00015-of-00101.safetensors", + "model.layers.15.mlp.experts.50.gate_proj.weight": "model-00015-of-00101.safetensors", + "model.layers.15.mlp.experts.50.up_proj.weight": "model-00015-of-00101.safetensors", + "model.layers.15.mlp.experts.50.down_proj.weight": "model-00015-of-00101.safetensors", + "model.layers.15.mlp.experts.51.gate_proj.weight": "model-00015-of-00101.safetensors", + "model.layers.15.mlp.experts.51.up_proj.weight": "model-00015-of-00101.safetensors", + "model.layers.15.mlp.experts.51.down_proj.weight": "model-00015-of-00101.safetensors", + "model.layers.15.mlp.experts.52.gate_proj.weight": "model-00015-of-00101.safetensors", + "model.layers.15.mlp.experts.52.up_proj.weight": "model-00015-of-00101.safetensors", + "model.layers.15.mlp.experts.52.down_proj.weight": "model-00015-of-00101.safetensors", + "model.layers.15.mlp.experts.53.gate_proj.weight": "model-00015-of-00101.safetensors", + "model.layers.15.mlp.experts.53.up_proj.weight": "model-00015-of-00101.safetensors", + "model.layers.15.mlp.experts.53.down_proj.weight": "model-00015-of-00101.safetensors", + "model.layers.15.mlp.experts.54.gate_proj.weight": "model-00015-of-00101.safetensors", + "model.layers.15.mlp.experts.54.up_proj.weight": "model-00015-of-00101.safetensors", + "model.layers.15.mlp.experts.54.down_proj.weight": "model-00015-of-00101.safetensors", + "model.layers.15.mlp.experts.55.gate_proj.weight": "model-00015-of-00101.safetensors", + "model.layers.15.mlp.experts.55.up_proj.weight": "model-00015-of-00101.safetensors", + "model.layers.15.mlp.experts.55.down_proj.weight": "model-00015-of-00101.safetensors", + "model.layers.15.mlp.experts.56.gate_proj.weight": "model-00015-of-00101.safetensors", + "model.layers.15.mlp.experts.56.up_proj.weight": "model-00015-of-00101.safetensors", + "model.layers.15.mlp.experts.56.down_proj.weight": "model-00015-of-00101.safetensors", + "model.layers.15.mlp.experts.57.gate_proj.weight": "model-00015-of-00101.safetensors", + "model.layers.15.mlp.experts.57.up_proj.weight": "model-00015-of-00101.safetensors", + "model.layers.15.mlp.experts.57.down_proj.weight": "model-00015-of-00101.safetensors", + "model.layers.15.mlp.experts.58.gate_proj.weight": "model-00015-of-00101.safetensors", + "model.layers.15.mlp.experts.58.up_proj.weight": "model-00015-of-00101.safetensors", + "model.layers.15.mlp.experts.58.down_proj.weight": "model-00015-of-00101.safetensors", + "model.layers.15.mlp.experts.59.gate_proj.weight": "model-00015-of-00101.safetensors", + "model.layers.15.mlp.experts.59.up_proj.weight": "model-00015-of-00101.safetensors", + "model.layers.15.mlp.experts.59.down_proj.weight": "model-00015-of-00101.safetensors", + "model.layers.15.mlp.experts.60.gate_proj.weight": "model-00015-of-00101.safetensors", + "model.layers.15.mlp.experts.60.up_proj.weight": "model-00015-of-00101.safetensors", + "model.layers.15.mlp.experts.60.down_proj.weight": "model-00015-of-00101.safetensors", + "model.layers.15.mlp.experts.61.gate_proj.weight": "model-00015-of-00101.safetensors", + "model.layers.15.mlp.experts.61.up_proj.weight": "model-00015-of-00101.safetensors", + "model.layers.15.mlp.experts.61.down_proj.weight": "model-00015-of-00101.safetensors", + "model.layers.15.mlp.experts.62.gate_proj.weight": "model-00015-of-00101.safetensors", + "model.layers.15.mlp.experts.62.up_proj.weight": "model-00015-of-00101.safetensors", + "model.layers.15.mlp.experts.62.down_proj.weight": "model-00015-of-00101.safetensors", + "model.layers.15.mlp.experts.63.gate_proj.weight": "model-00015-of-00101.safetensors", + "model.layers.15.mlp.experts.63.up_proj.weight": "model-00015-of-00101.safetensors", + "model.layers.15.mlp.experts.63.down_proj.weight": "model-00015-of-00101.safetensors", + "model.layers.15.mlp.experts.64.gate_proj.weight": "model-00015-of-00101.safetensors", + "model.layers.15.mlp.experts.64.up_proj.weight": "model-00015-of-00101.safetensors", + "model.layers.15.mlp.experts.64.down_proj.weight": "model-00015-of-00101.safetensors", + "model.layers.15.mlp.experts.65.gate_proj.weight": "model-00015-of-00101.safetensors", + "model.layers.15.mlp.experts.65.up_proj.weight": "model-00015-of-00101.safetensors", + "model.layers.15.mlp.experts.65.down_proj.weight": "model-00015-of-00101.safetensors", + "model.layers.15.mlp.experts.66.gate_proj.weight": "model-00015-of-00101.safetensors", + "model.layers.15.mlp.experts.66.up_proj.weight": "model-00015-of-00101.safetensors", + "model.layers.15.mlp.experts.66.down_proj.weight": "model-00015-of-00101.safetensors", + "model.layers.15.mlp.experts.67.gate_proj.weight": "model-00015-of-00101.safetensors", + "model.layers.15.mlp.experts.67.up_proj.weight": "model-00015-of-00101.safetensors", + "model.layers.15.mlp.experts.67.down_proj.weight": "model-00015-of-00101.safetensors", + "model.layers.15.mlp.experts.68.gate_proj.weight": "model-00015-of-00101.safetensors", + "model.layers.15.mlp.experts.68.up_proj.weight": "model-00015-of-00101.safetensors", + "model.layers.15.mlp.experts.68.down_proj.weight": "model-00015-of-00101.safetensors", + "model.layers.15.mlp.experts.69.gate_proj.weight": "model-00015-of-00101.safetensors", + "model.layers.15.mlp.experts.69.up_proj.weight": "model-00015-of-00101.safetensors", + "model.layers.15.mlp.experts.69.down_proj.weight": "model-00015-of-00101.safetensors", + "model.layers.15.mlp.experts.70.gate_proj.weight": "model-00015-of-00101.safetensors", + "model.layers.15.mlp.experts.70.up_proj.weight": "model-00015-of-00101.safetensors", + "model.layers.15.mlp.experts.70.down_proj.weight": "model-00015-of-00101.safetensors", + "model.layers.15.mlp.experts.71.gate_proj.weight": "model-00015-of-00101.safetensors", + "model.layers.15.mlp.experts.71.up_proj.weight": "model-00015-of-00101.safetensors", + "model.layers.15.mlp.experts.71.down_proj.weight": "model-00015-of-00101.safetensors", + "model.layers.15.mlp.experts.72.gate_proj.weight": "model-00015-of-00101.safetensors", + "model.layers.15.mlp.experts.72.up_proj.weight": "model-00015-of-00101.safetensors", + "model.layers.15.mlp.experts.72.down_proj.weight": "model-00015-of-00101.safetensors", + "model.layers.15.mlp.experts.73.gate_proj.weight": "model-00015-of-00101.safetensors", + "model.layers.15.mlp.experts.73.up_proj.weight": "model-00015-of-00101.safetensors", + "model.layers.15.mlp.experts.73.down_proj.weight": "model-00015-of-00101.safetensors", + "model.layers.15.mlp.experts.74.gate_proj.weight": "model-00015-of-00101.safetensors", + "model.layers.15.mlp.experts.74.up_proj.weight": "model-00015-of-00101.safetensors", + "model.layers.15.mlp.experts.74.down_proj.weight": "model-00015-of-00101.safetensors", + "model.layers.15.mlp.experts.75.gate_proj.weight": "model-00015-of-00101.safetensors", + "model.layers.15.mlp.experts.75.up_proj.weight": "model-00015-of-00101.safetensors", + "model.layers.15.mlp.experts.75.down_proj.weight": "model-00015-of-00101.safetensors", + "model.layers.15.mlp.experts.76.gate_proj.weight": "model-00015-of-00101.safetensors", + "model.layers.15.mlp.experts.76.up_proj.weight": "model-00015-of-00101.safetensors", + "model.layers.15.mlp.experts.76.down_proj.weight": "model-00015-of-00101.safetensors", + "model.layers.15.mlp.experts.77.gate_proj.weight": "model-00015-of-00101.safetensors", + "model.layers.15.mlp.experts.77.up_proj.weight": "model-00015-of-00101.safetensors", + "model.layers.15.mlp.experts.77.down_proj.weight": "model-00015-of-00101.safetensors", + "model.layers.15.mlp.experts.78.gate_proj.weight": "model-00015-of-00101.safetensors", + "model.layers.15.mlp.experts.78.up_proj.weight": "model-00015-of-00101.safetensors", + "model.layers.15.mlp.experts.78.down_proj.weight": "model-00015-of-00101.safetensors", + "model.layers.15.mlp.experts.79.gate_proj.weight": "model-00015-of-00101.safetensors", + "model.layers.15.mlp.experts.79.up_proj.weight": "model-00015-of-00101.safetensors", + "model.layers.15.mlp.experts.79.down_proj.weight": "model-00015-of-00101.safetensors", + "model.layers.15.mlp.experts.80.gate_proj.weight": "model-00015-of-00101.safetensors", + "model.layers.15.mlp.experts.80.up_proj.weight": "model-00015-of-00101.safetensors", + "model.layers.15.mlp.experts.80.down_proj.weight": "model-00015-of-00101.safetensors", + "model.layers.15.mlp.experts.81.gate_proj.weight": "model-00015-of-00101.safetensors", + "model.layers.15.mlp.experts.81.up_proj.weight": "model-00015-of-00101.safetensors", + "model.layers.15.mlp.experts.81.down_proj.weight": "model-00015-of-00101.safetensors", + "model.layers.15.mlp.experts.82.gate_proj.weight": "model-00015-of-00101.safetensors", + "model.layers.15.mlp.experts.82.up_proj.weight": "model-00015-of-00101.safetensors", + "model.layers.15.mlp.experts.82.down_proj.weight": "model-00015-of-00101.safetensors", + "model.layers.15.mlp.experts.83.gate_proj.weight": "model-00015-of-00101.safetensors", + "model.layers.15.mlp.experts.83.up_proj.weight": "model-00015-of-00101.safetensors", + "model.layers.15.mlp.experts.83.down_proj.weight": "model-00015-of-00101.safetensors", + "model.layers.15.mlp.experts.84.gate_proj.weight": "model-00015-of-00101.safetensors", + "model.layers.15.mlp.experts.84.up_proj.weight": "model-00015-of-00101.safetensors", + "model.layers.15.mlp.experts.84.down_proj.weight": "model-00015-of-00101.safetensors", + "model.layers.15.mlp.experts.85.gate_proj.weight": "model-00015-of-00101.safetensors", + "model.layers.15.mlp.experts.85.up_proj.weight": "model-00015-of-00101.safetensors", + "model.layers.15.mlp.experts.85.down_proj.weight": "model-00015-of-00101.safetensors", + "model.layers.15.mlp.experts.86.gate_proj.weight": "model-00015-of-00101.safetensors", + "model.layers.15.mlp.experts.86.up_proj.weight": "model-00015-of-00101.safetensors", + "model.layers.15.mlp.experts.86.down_proj.weight": "model-00015-of-00101.safetensors", + "model.layers.15.mlp.experts.87.gate_proj.weight": "model-00015-of-00101.safetensors", + "model.layers.15.mlp.experts.87.up_proj.weight": "model-00015-of-00101.safetensors", + "model.layers.15.mlp.experts.87.down_proj.weight": "model-00015-of-00101.safetensors", + "model.layers.15.mlp.experts.88.gate_proj.weight": "model-00015-of-00101.safetensors", + "model.layers.15.mlp.experts.88.up_proj.weight": "model-00015-of-00101.safetensors", + "model.layers.15.mlp.experts.88.down_proj.weight": "model-00015-of-00101.safetensors", + "model.layers.15.mlp.experts.89.gate_proj.weight": "model-00015-of-00101.safetensors", + "model.layers.15.mlp.experts.89.up_proj.weight": "model-00015-of-00101.safetensors", + "model.layers.15.mlp.experts.89.down_proj.weight": "model-00015-of-00101.safetensors", + "model.layers.15.mlp.experts.90.gate_proj.weight": "model-00015-of-00101.safetensors", + "model.layers.15.mlp.experts.90.up_proj.weight": "model-00015-of-00101.safetensors", + "model.layers.15.mlp.experts.90.down_proj.weight": "model-00015-of-00101.safetensors", + "model.layers.15.mlp.experts.91.gate_proj.weight": "model-00015-of-00101.safetensors", + "model.layers.15.mlp.experts.91.up_proj.weight": "model-00015-of-00101.safetensors", + "model.layers.15.mlp.experts.91.down_proj.weight": "model-00015-of-00101.safetensors", + "model.layers.15.mlp.experts.92.gate_proj.weight": "model-00015-of-00101.safetensors", + "model.layers.15.mlp.experts.92.up_proj.weight": "model-00015-of-00101.safetensors", + "model.layers.15.mlp.experts.92.down_proj.weight": "model-00015-of-00101.safetensors", + "model.layers.15.mlp.experts.93.gate_proj.weight": "model-00015-of-00101.safetensors", + "model.layers.15.mlp.experts.93.up_proj.weight": "model-00015-of-00101.safetensors", + "model.layers.15.mlp.experts.93.down_proj.weight": "model-00015-of-00101.safetensors", + "model.layers.15.mlp.experts.94.gate_proj.weight": "model-00015-of-00101.safetensors", + "model.layers.15.mlp.experts.94.up_proj.weight": "model-00015-of-00101.safetensors", + "model.layers.15.mlp.experts.94.down_proj.weight": "model-00015-of-00101.safetensors", + "model.layers.15.mlp.experts.95.gate_proj.weight": "model-00015-of-00101.safetensors", + "model.layers.15.mlp.experts.95.up_proj.weight": "model-00015-of-00101.safetensors", + "model.layers.15.mlp.experts.95.down_proj.weight": "model-00015-of-00101.safetensors", + "model.layers.15.mlp.experts.96.gate_proj.weight": "model-00015-of-00101.safetensors", + "model.layers.15.mlp.experts.96.up_proj.weight": "model-00015-of-00101.safetensors", + "model.layers.15.mlp.experts.96.down_proj.weight": "model-00015-of-00101.safetensors", + "model.layers.15.mlp.experts.97.gate_proj.weight": "model-00015-of-00101.safetensors", + "model.layers.15.mlp.experts.97.up_proj.weight": "model-00015-of-00101.safetensors", + "model.layers.15.mlp.experts.97.down_proj.weight": "model-00015-of-00101.safetensors", + "model.layers.15.mlp.experts.98.gate_proj.weight": "model-00015-of-00101.safetensors", + "model.layers.15.mlp.experts.98.up_proj.weight": "model-00015-of-00101.safetensors", + "model.layers.15.mlp.experts.98.down_proj.weight": "model-00015-of-00101.safetensors", + "model.layers.15.mlp.experts.99.gate_proj.weight": "model-00015-of-00101.safetensors", + "model.layers.15.mlp.experts.99.up_proj.weight": "model-00015-of-00101.safetensors", + "model.layers.15.mlp.experts.99.down_proj.weight": "model-00015-of-00101.safetensors", + "model.layers.15.mlp.experts.100.gate_proj.weight": "model-00015-of-00101.safetensors", + "model.layers.15.mlp.experts.100.up_proj.weight": "model-00015-of-00101.safetensors", + "model.layers.15.mlp.experts.100.down_proj.weight": "model-00015-of-00101.safetensors", + "model.layers.15.mlp.experts.101.gate_proj.weight": "model-00016-of-00101.safetensors", + "model.layers.15.mlp.experts.101.up_proj.weight": "model-00016-of-00101.safetensors", + "model.layers.15.mlp.experts.101.down_proj.weight": "model-00016-of-00101.safetensors", + "model.layers.15.mlp.experts.102.gate_proj.weight": "model-00016-of-00101.safetensors", + "model.layers.15.mlp.experts.102.up_proj.weight": "model-00016-of-00101.safetensors", + "model.layers.15.mlp.experts.102.down_proj.weight": "model-00016-of-00101.safetensors", + "model.layers.15.mlp.experts.103.gate_proj.weight": "model-00016-of-00101.safetensors", + "model.layers.15.mlp.experts.103.up_proj.weight": "model-00016-of-00101.safetensors", + "model.layers.15.mlp.experts.103.down_proj.weight": "model-00016-of-00101.safetensors", + "model.layers.15.mlp.experts.104.gate_proj.weight": "model-00016-of-00101.safetensors", + "model.layers.15.mlp.experts.104.up_proj.weight": "model-00016-of-00101.safetensors", + "model.layers.15.mlp.experts.104.down_proj.weight": "model-00016-of-00101.safetensors", + "model.layers.15.mlp.experts.105.gate_proj.weight": "model-00016-of-00101.safetensors", + "model.layers.15.mlp.experts.105.up_proj.weight": "model-00016-of-00101.safetensors", + "model.layers.15.mlp.experts.105.down_proj.weight": "model-00016-of-00101.safetensors", + "model.layers.15.mlp.experts.106.gate_proj.weight": "model-00016-of-00101.safetensors", + "model.layers.15.mlp.experts.106.up_proj.weight": "model-00016-of-00101.safetensors", + "model.layers.15.mlp.experts.106.down_proj.weight": "model-00016-of-00101.safetensors", + "model.layers.15.mlp.experts.107.gate_proj.weight": "model-00016-of-00101.safetensors", + "model.layers.15.mlp.experts.107.up_proj.weight": "model-00016-of-00101.safetensors", + "model.layers.15.mlp.experts.107.down_proj.weight": "model-00016-of-00101.safetensors", + "model.layers.15.mlp.experts.108.gate_proj.weight": "model-00016-of-00101.safetensors", + "model.layers.15.mlp.experts.108.up_proj.weight": "model-00016-of-00101.safetensors", + "model.layers.15.mlp.experts.108.down_proj.weight": "model-00016-of-00101.safetensors", + "model.layers.15.mlp.experts.109.gate_proj.weight": "model-00016-of-00101.safetensors", + "model.layers.15.mlp.experts.109.up_proj.weight": "model-00016-of-00101.safetensors", + "model.layers.15.mlp.experts.109.down_proj.weight": "model-00016-of-00101.safetensors", + "model.layers.15.mlp.experts.110.gate_proj.weight": "model-00016-of-00101.safetensors", + "model.layers.15.mlp.experts.110.up_proj.weight": "model-00016-of-00101.safetensors", + "model.layers.15.mlp.experts.110.down_proj.weight": "model-00016-of-00101.safetensors", + "model.layers.15.mlp.experts.111.gate_proj.weight": "model-00016-of-00101.safetensors", + "model.layers.15.mlp.experts.111.up_proj.weight": "model-00016-of-00101.safetensors", + "model.layers.15.mlp.experts.111.down_proj.weight": "model-00016-of-00101.safetensors", + "model.layers.15.mlp.experts.112.gate_proj.weight": "model-00016-of-00101.safetensors", + "model.layers.15.mlp.experts.112.up_proj.weight": "model-00016-of-00101.safetensors", + "model.layers.15.mlp.experts.112.down_proj.weight": "model-00016-of-00101.safetensors", + "model.layers.15.mlp.experts.113.gate_proj.weight": "model-00016-of-00101.safetensors", + "model.layers.15.mlp.experts.113.up_proj.weight": "model-00016-of-00101.safetensors", + "model.layers.15.mlp.experts.113.down_proj.weight": "model-00016-of-00101.safetensors", + "model.layers.15.mlp.experts.114.gate_proj.weight": "model-00016-of-00101.safetensors", + "model.layers.15.mlp.experts.114.up_proj.weight": "model-00016-of-00101.safetensors", + "model.layers.15.mlp.experts.114.down_proj.weight": "model-00016-of-00101.safetensors", + "model.layers.15.mlp.experts.115.gate_proj.weight": "model-00016-of-00101.safetensors", + "model.layers.15.mlp.experts.115.up_proj.weight": "model-00016-of-00101.safetensors", + "model.layers.15.mlp.experts.115.down_proj.weight": "model-00016-of-00101.safetensors", + "model.layers.15.mlp.experts.116.gate_proj.weight": "model-00016-of-00101.safetensors", + "model.layers.15.mlp.experts.116.up_proj.weight": "model-00016-of-00101.safetensors", + "model.layers.15.mlp.experts.116.down_proj.weight": "model-00016-of-00101.safetensors", + "model.layers.15.mlp.experts.117.gate_proj.weight": "model-00016-of-00101.safetensors", + "model.layers.15.mlp.experts.117.up_proj.weight": "model-00016-of-00101.safetensors", + "model.layers.15.mlp.experts.117.down_proj.weight": "model-00016-of-00101.safetensors", + "model.layers.15.mlp.experts.118.gate_proj.weight": "model-00016-of-00101.safetensors", + "model.layers.15.mlp.experts.118.up_proj.weight": "model-00016-of-00101.safetensors", + "model.layers.15.mlp.experts.118.down_proj.weight": "model-00016-of-00101.safetensors", + "model.layers.15.mlp.experts.119.gate_proj.weight": "model-00016-of-00101.safetensors", + "model.layers.15.mlp.experts.119.up_proj.weight": "model-00016-of-00101.safetensors", + "model.layers.15.mlp.experts.119.down_proj.weight": "model-00016-of-00101.safetensors", + "model.layers.15.mlp.gate.weight": "model-00016-of-00101.safetensors", + "model.layers.15.mlp.gate.e_score_correction_bias": "model-00016-of-00101.safetensors", + "model.layers.15.mlp.shared_experts.gate_proj.weight": "model-00016-of-00101.safetensors", + "model.layers.15.mlp.shared_experts.up_proj.weight": "model-00016-of-00101.safetensors", + "model.layers.15.mlp.shared_experts.down_proj.weight": "model-00016-of-00101.safetensors", + "model.layers.15.input_layernorm.weight": "model-00016-of-00101.safetensors", + "model.layers.15.post_attention_layernorm.weight": "model-00016-of-00101.safetensors", + "model.layers.16.self_attn.q_proj.weight": "model-00016-of-00101.safetensors", + "model.layers.16.self_attn.q_proj.bias": "model-00016-of-00101.safetensors", + "model.layers.16.self_attn.k_proj.weight": "model-00016-of-00101.safetensors", + "model.layers.16.self_attn.k_proj.bias": "model-00016-of-00101.safetensors", + "model.layers.16.self_attn.v_proj.weight": "model-00016-of-00101.safetensors", + "model.layers.16.self_attn.v_proj.bias": "model-00016-of-00101.safetensors", + "model.layers.16.self_attn.o_proj.weight": "model-00016-of-00101.safetensors", + "model.layers.16.self_attn.q_norm.weight": "model-00016-of-00101.safetensors", + "model.layers.16.self_attn.k_norm.weight": "model-00016-of-00101.safetensors", + "model.layers.16.mlp.experts.0.gate_proj.weight": "model-00016-of-00101.safetensors", + "model.layers.16.mlp.experts.0.up_proj.weight": "model-00016-of-00101.safetensors", + "model.layers.16.mlp.experts.0.down_proj.weight": "model-00016-of-00101.safetensors", + "model.layers.16.mlp.experts.1.gate_proj.weight": "model-00016-of-00101.safetensors", + "model.layers.16.mlp.experts.1.up_proj.weight": "model-00016-of-00101.safetensors", + "model.layers.16.mlp.experts.1.down_proj.weight": "model-00016-of-00101.safetensors", + "model.layers.16.mlp.experts.2.gate_proj.weight": "model-00016-of-00101.safetensors", + "model.layers.16.mlp.experts.2.up_proj.weight": "model-00016-of-00101.safetensors", + "model.layers.16.mlp.experts.2.down_proj.weight": "model-00016-of-00101.safetensors", + "model.layers.16.mlp.experts.3.gate_proj.weight": "model-00016-of-00101.safetensors", + "model.layers.16.mlp.experts.3.up_proj.weight": "model-00016-of-00101.safetensors", + "model.layers.16.mlp.experts.3.down_proj.weight": "model-00016-of-00101.safetensors", + "model.layers.16.mlp.experts.4.gate_proj.weight": "model-00016-of-00101.safetensors", + "model.layers.16.mlp.experts.4.up_proj.weight": "model-00016-of-00101.safetensors", + "model.layers.16.mlp.experts.4.down_proj.weight": "model-00016-of-00101.safetensors", + "model.layers.16.mlp.experts.5.gate_proj.weight": "model-00016-of-00101.safetensors", + "model.layers.16.mlp.experts.5.up_proj.weight": "model-00016-of-00101.safetensors", + "model.layers.16.mlp.experts.5.down_proj.weight": "model-00016-of-00101.safetensors", + "model.layers.16.mlp.experts.6.gate_proj.weight": "model-00016-of-00101.safetensors", + "model.layers.16.mlp.experts.6.up_proj.weight": "model-00016-of-00101.safetensors", + "model.layers.16.mlp.experts.6.down_proj.weight": "model-00016-of-00101.safetensors", + "model.layers.16.mlp.experts.7.gate_proj.weight": "model-00016-of-00101.safetensors", + "model.layers.16.mlp.experts.7.up_proj.weight": "model-00016-of-00101.safetensors", + "model.layers.16.mlp.experts.7.down_proj.weight": "model-00016-of-00101.safetensors", + "model.layers.16.mlp.experts.8.gate_proj.weight": "model-00016-of-00101.safetensors", + "model.layers.16.mlp.experts.8.up_proj.weight": "model-00016-of-00101.safetensors", + "model.layers.16.mlp.experts.8.down_proj.weight": "model-00016-of-00101.safetensors", + "model.layers.16.mlp.experts.9.gate_proj.weight": "model-00016-of-00101.safetensors", + "model.layers.16.mlp.experts.9.up_proj.weight": "model-00016-of-00101.safetensors", + "model.layers.16.mlp.experts.9.down_proj.weight": "model-00016-of-00101.safetensors", + "model.layers.16.mlp.experts.10.gate_proj.weight": "model-00016-of-00101.safetensors", + "model.layers.16.mlp.experts.10.up_proj.weight": "model-00016-of-00101.safetensors", + "model.layers.16.mlp.experts.10.down_proj.weight": "model-00016-of-00101.safetensors", + "model.layers.16.mlp.experts.11.gate_proj.weight": "model-00016-of-00101.safetensors", + "model.layers.16.mlp.experts.11.up_proj.weight": "model-00016-of-00101.safetensors", + "model.layers.16.mlp.experts.11.down_proj.weight": "model-00016-of-00101.safetensors", + "model.layers.16.mlp.experts.12.gate_proj.weight": "model-00016-of-00101.safetensors", + "model.layers.16.mlp.experts.12.up_proj.weight": "model-00016-of-00101.safetensors", + "model.layers.16.mlp.experts.12.down_proj.weight": "model-00016-of-00101.safetensors", + "model.layers.16.mlp.experts.13.gate_proj.weight": "model-00016-of-00101.safetensors", + "model.layers.16.mlp.experts.13.up_proj.weight": "model-00016-of-00101.safetensors", + "model.layers.16.mlp.experts.13.down_proj.weight": "model-00016-of-00101.safetensors", + "model.layers.16.mlp.experts.14.gate_proj.weight": "model-00016-of-00101.safetensors", + "model.layers.16.mlp.experts.14.up_proj.weight": "model-00016-of-00101.safetensors", + "model.layers.16.mlp.experts.14.down_proj.weight": "model-00016-of-00101.safetensors", + "model.layers.16.mlp.experts.15.gate_proj.weight": "model-00016-of-00101.safetensors", + "model.layers.16.mlp.experts.15.up_proj.weight": "model-00016-of-00101.safetensors", + "model.layers.16.mlp.experts.15.down_proj.weight": "model-00016-of-00101.safetensors", + "model.layers.16.mlp.experts.16.gate_proj.weight": "model-00016-of-00101.safetensors", + "model.layers.16.mlp.experts.16.up_proj.weight": "model-00016-of-00101.safetensors", + "model.layers.16.mlp.experts.16.down_proj.weight": "model-00016-of-00101.safetensors", + "model.layers.16.mlp.experts.17.gate_proj.weight": "model-00016-of-00101.safetensors", + "model.layers.16.mlp.experts.17.up_proj.weight": "model-00016-of-00101.safetensors", + "model.layers.16.mlp.experts.17.down_proj.weight": "model-00016-of-00101.safetensors", + "model.layers.16.mlp.experts.18.gate_proj.weight": "model-00016-of-00101.safetensors", + "model.layers.16.mlp.experts.18.up_proj.weight": "model-00016-of-00101.safetensors", + "model.layers.16.mlp.experts.18.down_proj.weight": "model-00016-of-00101.safetensors", + "model.layers.16.mlp.experts.19.gate_proj.weight": "model-00016-of-00101.safetensors", + "model.layers.16.mlp.experts.19.up_proj.weight": "model-00016-of-00101.safetensors", + "model.layers.16.mlp.experts.19.down_proj.weight": "model-00016-of-00101.safetensors", + "model.layers.16.mlp.experts.20.gate_proj.weight": "model-00016-of-00101.safetensors", + "model.layers.16.mlp.experts.20.up_proj.weight": "model-00016-of-00101.safetensors", + "model.layers.16.mlp.experts.20.down_proj.weight": "model-00016-of-00101.safetensors", + "model.layers.16.mlp.experts.21.gate_proj.weight": "model-00016-of-00101.safetensors", + "model.layers.16.mlp.experts.21.up_proj.weight": "model-00016-of-00101.safetensors", + "model.layers.16.mlp.experts.21.down_proj.weight": "model-00016-of-00101.safetensors", + "model.layers.16.mlp.experts.22.gate_proj.weight": "model-00016-of-00101.safetensors", + "model.layers.16.mlp.experts.22.up_proj.weight": "model-00016-of-00101.safetensors", + "model.layers.16.mlp.experts.22.down_proj.weight": "model-00016-of-00101.safetensors", + "model.layers.16.mlp.experts.23.gate_proj.weight": "model-00016-of-00101.safetensors", + "model.layers.16.mlp.experts.23.up_proj.weight": "model-00016-of-00101.safetensors", + "model.layers.16.mlp.experts.23.down_proj.weight": "model-00016-of-00101.safetensors", + "model.layers.16.mlp.experts.24.gate_proj.weight": "model-00016-of-00101.safetensors", + "model.layers.16.mlp.experts.24.up_proj.weight": "model-00016-of-00101.safetensors", + "model.layers.16.mlp.experts.24.down_proj.weight": "model-00016-of-00101.safetensors", + "model.layers.16.mlp.experts.25.gate_proj.weight": "model-00016-of-00101.safetensors", + "model.layers.16.mlp.experts.25.up_proj.weight": "model-00016-of-00101.safetensors", + "model.layers.16.mlp.experts.25.down_proj.weight": "model-00016-of-00101.safetensors", + "model.layers.16.mlp.experts.26.gate_proj.weight": "model-00016-of-00101.safetensors", + "model.layers.16.mlp.experts.26.up_proj.weight": "model-00016-of-00101.safetensors", + "model.layers.16.mlp.experts.26.down_proj.weight": "model-00016-of-00101.safetensors", + "model.layers.16.mlp.experts.27.gate_proj.weight": "model-00016-of-00101.safetensors", + "model.layers.16.mlp.experts.27.up_proj.weight": "model-00016-of-00101.safetensors", + "model.layers.16.mlp.experts.27.down_proj.weight": "model-00016-of-00101.safetensors", + "model.layers.16.mlp.experts.28.gate_proj.weight": "model-00016-of-00101.safetensors", + "model.layers.16.mlp.experts.28.up_proj.weight": "model-00016-of-00101.safetensors", + "model.layers.16.mlp.experts.28.down_proj.weight": "model-00016-of-00101.safetensors", + "model.layers.16.mlp.experts.29.gate_proj.weight": "model-00016-of-00101.safetensors", + "model.layers.16.mlp.experts.29.up_proj.weight": "model-00016-of-00101.safetensors", + "model.layers.16.mlp.experts.29.down_proj.weight": "model-00016-of-00101.safetensors", + "model.layers.16.mlp.experts.30.gate_proj.weight": "model-00016-of-00101.safetensors", + "model.layers.16.mlp.experts.30.up_proj.weight": "model-00016-of-00101.safetensors", + "model.layers.16.mlp.experts.30.down_proj.weight": "model-00016-of-00101.safetensors", + "model.layers.16.mlp.experts.31.gate_proj.weight": "model-00016-of-00101.safetensors", + "model.layers.16.mlp.experts.31.up_proj.weight": "model-00016-of-00101.safetensors", + "model.layers.16.mlp.experts.31.down_proj.weight": "model-00016-of-00101.safetensors", + "model.layers.16.mlp.experts.32.gate_proj.weight": "model-00016-of-00101.safetensors", + "model.layers.16.mlp.experts.32.up_proj.weight": "model-00016-of-00101.safetensors", + "model.layers.16.mlp.experts.32.down_proj.weight": "model-00016-of-00101.safetensors", + "model.layers.16.mlp.experts.33.gate_proj.weight": "model-00016-of-00101.safetensors", + "model.layers.16.mlp.experts.33.up_proj.weight": "model-00016-of-00101.safetensors", + "model.layers.16.mlp.experts.33.down_proj.weight": "model-00016-of-00101.safetensors", + "model.layers.16.mlp.experts.34.gate_proj.weight": "model-00016-of-00101.safetensors", + "model.layers.16.mlp.experts.34.up_proj.weight": "model-00016-of-00101.safetensors", + "model.layers.16.mlp.experts.34.down_proj.weight": "model-00016-of-00101.safetensors", + "model.layers.16.mlp.experts.35.gate_proj.weight": "model-00016-of-00101.safetensors", + "model.layers.16.mlp.experts.35.up_proj.weight": "model-00016-of-00101.safetensors", + "model.layers.16.mlp.experts.35.down_proj.weight": "model-00016-of-00101.safetensors", + "model.layers.16.mlp.experts.36.gate_proj.weight": "model-00016-of-00101.safetensors", + "model.layers.16.mlp.experts.36.up_proj.weight": "model-00016-of-00101.safetensors", + "model.layers.16.mlp.experts.36.down_proj.weight": "model-00016-of-00101.safetensors", + "model.layers.16.mlp.experts.37.gate_proj.weight": "model-00016-of-00101.safetensors", + "model.layers.16.mlp.experts.37.up_proj.weight": "model-00016-of-00101.safetensors", + "model.layers.16.mlp.experts.37.down_proj.weight": "model-00016-of-00101.safetensors", + "model.layers.16.mlp.experts.38.gate_proj.weight": "model-00016-of-00101.safetensors", + "model.layers.16.mlp.experts.38.up_proj.weight": "model-00016-of-00101.safetensors", + "model.layers.16.mlp.experts.38.down_proj.weight": "model-00016-of-00101.safetensors", + "model.layers.16.mlp.experts.39.gate_proj.weight": "model-00016-of-00101.safetensors", + "model.layers.16.mlp.experts.39.up_proj.weight": "model-00016-of-00101.safetensors", + "model.layers.16.mlp.experts.39.down_proj.weight": "model-00016-of-00101.safetensors", + "model.layers.16.mlp.experts.40.gate_proj.weight": "model-00016-of-00101.safetensors", + "model.layers.16.mlp.experts.40.up_proj.weight": "model-00016-of-00101.safetensors", + "model.layers.16.mlp.experts.40.down_proj.weight": "model-00016-of-00101.safetensors", + "model.layers.16.mlp.experts.41.gate_proj.weight": "model-00016-of-00101.safetensors", + "model.layers.16.mlp.experts.41.up_proj.weight": "model-00016-of-00101.safetensors", + "model.layers.16.mlp.experts.41.down_proj.weight": "model-00016-of-00101.safetensors", + "model.layers.16.mlp.experts.42.gate_proj.weight": "model-00016-of-00101.safetensors", + "model.layers.16.mlp.experts.42.up_proj.weight": "model-00016-of-00101.safetensors", + "model.layers.16.mlp.experts.42.down_proj.weight": "model-00016-of-00101.safetensors", + "model.layers.16.mlp.experts.43.gate_proj.weight": "model-00016-of-00101.safetensors", + "model.layers.16.mlp.experts.43.up_proj.weight": "model-00016-of-00101.safetensors", + "model.layers.16.mlp.experts.43.down_proj.weight": "model-00016-of-00101.safetensors", + "model.layers.16.mlp.experts.44.gate_proj.weight": "model-00016-of-00101.safetensors", + "model.layers.16.mlp.experts.44.up_proj.weight": "model-00016-of-00101.safetensors", + "model.layers.16.mlp.experts.44.down_proj.weight": "model-00016-of-00101.safetensors", + "model.layers.16.mlp.experts.45.gate_proj.weight": "model-00016-of-00101.safetensors", + "model.layers.16.mlp.experts.45.up_proj.weight": "model-00016-of-00101.safetensors", + "model.layers.16.mlp.experts.45.down_proj.weight": "model-00016-of-00101.safetensors", + "model.layers.16.mlp.experts.46.gate_proj.weight": "model-00016-of-00101.safetensors", + "model.layers.16.mlp.experts.46.up_proj.weight": "model-00016-of-00101.safetensors", + "model.layers.16.mlp.experts.46.down_proj.weight": "model-00016-of-00101.safetensors", + "model.layers.16.mlp.experts.47.gate_proj.weight": "model-00016-of-00101.safetensors", + "model.layers.16.mlp.experts.47.up_proj.weight": "model-00016-of-00101.safetensors", + "model.layers.16.mlp.experts.47.down_proj.weight": "model-00016-of-00101.safetensors", + "model.layers.16.mlp.experts.48.gate_proj.weight": "model-00016-of-00101.safetensors", + "model.layers.16.mlp.experts.48.up_proj.weight": "model-00016-of-00101.safetensors", + "model.layers.16.mlp.experts.48.down_proj.weight": "model-00016-of-00101.safetensors", + "model.layers.16.mlp.experts.49.gate_proj.weight": "model-00016-of-00101.safetensors", + "model.layers.16.mlp.experts.49.up_proj.weight": "model-00016-of-00101.safetensors", + "model.layers.16.mlp.experts.49.down_proj.weight": "model-00016-of-00101.safetensors", + "model.layers.16.mlp.experts.50.gate_proj.weight": "model-00016-of-00101.safetensors", + "model.layers.16.mlp.experts.50.up_proj.weight": "model-00016-of-00101.safetensors", + "model.layers.16.mlp.experts.50.down_proj.weight": "model-00016-of-00101.safetensors", + "model.layers.16.mlp.experts.51.gate_proj.weight": "model-00016-of-00101.safetensors", + "model.layers.16.mlp.experts.51.up_proj.weight": "model-00016-of-00101.safetensors", + "model.layers.16.mlp.experts.51.down_proj.weight": "model-00016-of-00101.safetensors", + "model.layers.16.mlp.experts.52.gate_proj.weight": "model-00016-of-00101.safetensors", + "model.layers.16.mlp.experts.52.up_proj.weight": "model-00016-of-00101.safetensors", + "model.layers.16.mlp.experts.52.down_proj.weight": "model-00016-of-00101.safetensors", + "model.layers.16.mlp.experts.53.gate_proj.weight": "model-00016-of-00101.safetensors", + "model.layers.16.mlp.experts.53.up_proj.weight": "model-00016-of-00101.safetensors", + "model.layers.16.mlp.experts.53.down_proj.weight": "model-00016-of-00101.safetensors", + "model.layers.16.mlp.experts.54.gate_proj.weight": "model-00016-of-00101.safetensors", + "model.layers.16.mlp.experts.54.up_proj.weight": "model-00016-of-00101.safetensors", + "model.layers.16.mlp.experts.54.down_proj.weight": "model-00016-of-00101.safetensors", + "model.layers.16.mlp.experts.55.gate_proj.weight": "model-00016-of-00101.safetensors", + "model.layers.16.mlp.experts.55.up_proj.weight": "model-00016-of-00101.safetensors", + "model.layers.16.mlp.experts.55.down_proj.weight": "model-00016-of-00101.safetensors", + "model.layers.16.mlp.experts.56.gate_proj.weight": "model-00016-of-00101.safetensors", + "model.layers.16.mlp.experts.56.up_proj.weight": "model-00016-of-00101.safetensors", + "model.layers.16.mlp.experts.56.down_proj.weight": "model-00016-of-00101.safetensors", + "model.layers.16.mlp.experts.57.gate_proj.weight": "model-00016-of-00101.safetensors", + "model.layers.16.mlp.experts.57.up_proj.weight": "model-00016-of-00101.safetensors", + "model.layers.16.mlp.experts.57.down_proj.weight": "model-00016-of-00101.safetensors", + "model.layers.16.mlp.experts.58.gate_proj.weight": "model-00016-of-00101.safetensors", + "model.layers.16.mlp.experts.58.up_proj.weight": "model-00016-of-00101.safetensors", + "model.layers.16.mlp.experts.58.down_proj.weight": "model-00016-of-00101.safetensors", + "model.layers.16.mlp.experts.59.gate_proj.weight": "model-00016-of-00101.safetensors", + "model.layers.16.mlp.experts.59.up_proj.weight": "model-00016-of-00101.safetensors", + "model.layers.16.mlp.experts.59.down_proj.weight": "model-00016-of-00101.safetensors", + "model.layers.16.mlp.experts.60.gate_proj.weight": "model-00016-of-00101.safetensors", + "model.layers.16.mlp.experts.60.up_proj.weight": "model-00016-of-00101.safetensors", + "model.layers.16.mlp.experts.60.down_proj.weight": "model-00016-of-00101.safetensors", + "model.layers.16.mlp.experts.61.gate_proj.weight": "model-00016-of-00101.safetensors", + "model.layers.16.mlp.experts.61.up_proj.weight": "model-00016-of-00101.safetensors", + "model.layers.16.mlp.experts.61.down_proj.weight": "model-00016-of-00101.safetensors", + "model.layers.16.mlp.experts.62.gate_proj.weight": "model-00016-of-00101.safetensors", + "model.layers.16.mlp.experts.62.up_proj.weight": "model-00016-of-00101.safetensors", + "model.layers.16.mlp.experts.62.down_proj.weight": "model-00016-of-00101.safetensors", + "model.layers.16.mlp.experts.63.gate_proj.weight": "model-00016-of-00101.safetensors", + "model.layers.16.mlp.experts.63.up_proj.weight": "model-00016-of-00101.safetensors", + "model.layers.16.mlp.experts.63.down_proj.weight": "model-00016-of-00101.safetensors", + "model.layers.16.mlp.experts.64.gate_proj.weight": "model-00016-of-00101.safetensors", + "model.layers.16.mlp.experts.64.up_proj.weight": "model-00016-of-00101.safetensors", + "model.layers.16.mlp.experts.64.down_proj.weight": "model-00016-of-00101.safetensors", + "model.layers.16.mlp.experts.65.gate_proj.weight": "model-00016-of-00101.safetensors", + "model.layers.16.mlp.experts.65.up_proj.weight": "model-00016-of-00101.safetensors", + "model.layers.16.mlp.experts.65.down_proj.weight": "model-00016-of-00101.safetensors", + "model.layers.16.mlp.experts.66.gate_proj.weight": "model-00016-of-00101.safetensors", + "model.layers.16.mlp.experts.66.up_proj.weight": "model-00016-of-00101.safetensors", + "model.layers.16.mlp.experts.66.down_proj.weight": "model-00016-of-00101.safetensors", + "model.layers.16.mlp.experts.67.gate_proj.weight": "model-00016-of-00101.safetensors", + "model.layers.16.mlp.experts.67.up_proj.weight": "model-00016-of-00101.safetensors", + "model.layers.16.mlp.experts.67.down_proj.weight": "model-00016-of-00101.safetensors", + "model.layers.16.mlp.experts.68.gate_proj.weight": "model-00016-of-00101.safetensors", + "model.layers.16.mlp.experts.68.up_proj.weight": "model-00016-of-00101.safetensors", + "model.layers.16.mlp.experts.68.down_proj.weight": "model-00016-of-00101.safetensors", + "model.layers.16.mlp.experts.69.gate_proj.weight": "model-00016-of-00101.safetensors", + "model.layers.16.mlp.experts.69.up_proj.weight": "model-00016-of-00101.safetensors", + "model.layers.16.mlp.experts.69.down_proj.weight": "model-00016-of-00101.safetensors", + "model.layers.16.mlp.experts.70.gate_proj.weight": "model-00016-of-00101.safetensors", + "model.layers.16.mlp.experts.70.up_proj.weight": "model-00016-of-00101.safetensors", + "model.layers.16.mlp.experts.70.down_proj.weight": "model-00016-of-00101.safetensors", + "model.layers.16.mlp.experts.71.gate_proj.weight": "model-00016-of-00101.safetensors", + "model.layers.16.mlp.experts.71.up_proj.weight": "model-00016-of-00101.safetensors", + "model.layers.16.mlp.experts.71.down_proj.weight": "model-00016-of-00101.safetensors", + "model.layers.16.mlp.experts.72.gate_proj.weight": "model-00016-of-00101.safetensors", + "model.layers.16.mlp.experts.72.up_proj.weight": "model-00016-of-00101.safetensors", + "model.layers.16.mlp.experts.72.down_proj.weight": "model-00016-of-00101.safetensors", + "model.layers.16.mlp.experts.73.gate_proj.weight": "model-00016-of-00101.safetensors", + "model.layers.16.mlp.experts.73.up_proj.weight": "model-00016-of-00101.safetensors", + "model.layers.16.mlp.experts.73.down_proj.weight": "model-00016-of-00101.safetensors", + "model.layers.16.mlp.experts.74.gate_proj.weight": "model-00016-of-00101.safetensors", + "model.layers.16.mlp.experts.74.up_proj.weight": "model-00016-of-00101.safetensors", + "model.layers.16.mlp.experts.74.down_proj.weight": "model-00016-of-00101.safetensors", + "model.layers.16.mlp.experts.75.gate_proj.weight": "model-00016-of-00101.safetensors", + "model.layers.16.mlp.experts.75.up_proj.weight": "model-00016-of-00101.safetensors", + "model.layers.16.mlp.experts.75.down_proj.weight": "model-00016-of-00101.safetensors", + "model.layers.16.mlp.experts.76.gate_proj.weight": "model-00016-of-00101.safetensors", + "model.layers.16.mlp.experts.76.up_proj.weight": "model-00016-of-00101.safetensors", + "model.layers.16.mlp.experts.76.down_proj.weight": "model-00016-of-00101.safetensors", + "model.layers.16.mlp.experts.77.gate_proj.weight": "model-00016-of-00101.safetensors", + "model.layers.16.mlp.experts.77.up_proj.weight": "model-00016-of-00101.safetensors", + "model.layers.16.mlp.experts.77.down_proj.weight": "model-00016-of-00101.safetensors", + "model.layers.16.mlp.experts.78.gate_proj.weight": "model-00016-of-00101.safetensors", + "model.layers.16.mlp.experts.78.up_proj.weight": "model-00016-of-00101.safetensors", + "model.layers.16.mlp.experts.78.down_proj.weight": "model-00016-of-00101.safetensors", + "model.layers.16.mlp.experts.79.gate_proj.weight": "model-00016-of-00101.safetensors", + "model.layers.16.mlp.experts.79.up_proj.weight": "model-00016-of-00101.safetensors", + "model.layers.16.mlp.experts.79.down_proj.weight": "model-00016-of-00101.safetensors", + "model.layers.16.mlp.experts.80.gate_proj.weight": "model-00016-of-00101.safetensors", + "model.layers.16.mlp.experts.80.up_proj.weight": "model-00016-of-00101.safetensors", + "model.layers.16.mlp.experts.80.down_proj.weight": "model-00016-of-00101.safetensors", + "model.layers.16.mlp.experts.81.gate_proj.weight": "model-00016-of-00101.safetensors", + "model.layers.16.mlp.experts.81.up_proj.weight": "model-00016-of-00101.safetensors", + "model.layers.16.mlp.experts.81.down_proj.weight": "model-00016-of-00101.safetensors", + "model.layers.16.mlp.experts.82.gate_proj.weight": "model-00016-of-00101.safetensors", + "model.layers.16.mlp.experts.82.up_proj.weight": "model-00016-of-00101.safetensors", + "model.layers.16.mlp.experts.82.down_proj.weight": "model-00016-of-00101.safetensors", + "model.layers.16.mlp.experts.83.gate_proj.weight": "model-00016-of-00101.safetensors", + "model.layers.16.mlp.experts.83.up_proj.weight": "model-00016-of-00101.safetensors", + "model.layers.16.mlp.experts.83.down_proj.weight": "model-00016-of-00101.safetensors", + "model.layers.16.mlp.experts.84.gate_proj.weight": "model-00016-of-00101.safetensors", + "model.layers.16.mlp.experts.84.up_proj.weight": "model-00016-of-00101.safetensors", + "model.layers.16.mlp.experts.84.down_proj.weight": "model-00016-of-00101.safetensors", + "model.layers.16.mlp.experts.85.gate_proj.weight": "model-00016-of-00101.safetensors", + "model.layers.16.mlp.experts.85.up_proj.weight": "model-00016-of-00101.safetensors", + "model.layers.16.mlp.experts.85.down_proj.weight": "model-00016-of-00101.safetensors", + "model.layers.16.mlp.experts.86.gate_proj.weight": "model-00016-of-00101.safetensors", + "model.layers.16.mlp.experts.86.up_proj.weight": "model-00016-of-00101.safetensors", + "model.layers.16.mlp.experts.86.down_proj.weight": "model-00016-of-00101.safetensors", + "model.layers.16.mlp.experts.87.gate_proj.weight": "model-00016-of-00101.safetensors", + "model.layers.16.mlp.experts.87.up_proj.weight": "model-00016-of-00101.safetensors", + "model.layers.16.mlp.experts.87.down_proj.weight": "model-00017-of-00101.safetensors", + "model.layers.16.mlp.experts.88.gate_proj.weight": "model-00017-of-00101.safetensors", + "model.layers.16.mlp.experts.88.up_proj.weight": "model-00017-of-00101.safetensors", + "model.layers.16.mlp.experts.88.down_proj.weight": "model-00017-of-00101.safetensors", + "model.layers.16.mlp.experts.89.gate_proj.weight": "model-00017-of-00101.safetensors", + "model.layers.16.mlp.experts.89.up_proj.weight": "model-00017-of-00101.safetensors", + "model.layers.16.mlp.experts.89.down_proj.weight": "model-00017-of-00101.safetensors", + "model.layers.16.mlp.experts.90.gate_proj.weight": "model-00017-of-00101.safetensors", + "model.layers.16.mlp.experts.90.up_proj.weight": "model-00017-of-00101.safetensors", + "model.layers.16.mlp.experts.90.down_proj.weight": "model-00017-of-00101.safetensors", + "model.layers.16.mlp.experts.91.gate_proj.weight": "model-00017-of-00101.safetensors", + "model.layers.16.mlp.experts.91.up_proj.weight": "model-00017-of-00101.safetensors", + "model.layers.16.mlp.experts.91.down_proj.weight": "model-00017-of-00101.safetensors", + "model.layers.16.mlp.experts.92.gate_proj.weight": "model-00017-of-00101.safetensors", + "model.layers.16.mlp.experts.92.up_proj.weight": "model-00017-of-00101.safetensors", + "model.layers.16.mlp.experts.92.down_proj.weight": "model-00017-of-00101.safetensors", + "model.layers.16.mlp.experts.93.gate_proj.weight": "model-00017-of-00101.safetensors", + "model.layers.16.mlp.experts.93.up_proj.weight": "model-00017-of-00101.safetensors", + "model.layers.16.mlp.experts.93.down_proj.weight": "model-00017-of-00101.safetensors", + "model.layers.16.mlp.experts.94.gate_proj.weight": "model-00017-of-00101.safetensors", + "model.layers.16.mlp.experts.94.up_proj.weight": "model-00017-of-00101.safetensors", + "model.layers.16.mlp.experts.94.down_proj.weight": "model-00017-of-00101.safetensors", + "model.layers.16.mlp.experts.95.gate_proj.weight": "model-00017-of-00101.safetensors", + "model.layers.16.mlp.experts.95.up_proj.weight": "model-00017-of-00101.safetensors", + "model.layers.16.mlp.experts.95.down_proj.weight": "model-00017-of-00101.safetensors", + "model.layers.16.mlp.experts.96.gate_proj.weight": "model-00017-of-00101.safetensors", + "model.layers.16.mlp.experts.96.up_proj.weight": "model-00017-of-00101.safetensors", + "model.layers.16.mlp.experts.96.down_proj.weight": "model-00017-of-00101.safetensors", + "model.layers.16.mlp.experts.97.gate_proj.weight": "model-00017-of-00101.safetensors", + "model.layers.16.mlp.experts.97.up_proj.weight": "model-00017-of-00101.safetensors", + "model.layers.16.mlp.experts.97.down_proj.weight": "model-00017-of-00101.safetensors", + "model.layers.16.mlp.experts.98.gate_proj.weight": "model-00017-of-00101.safetensors", + "model.layers.16.mlp.experts.98.up_proj.weight": "model-00017-of-00101.safetensors", + "model.layers.16.mlp.experts.98.down_proj.weight": "model-00017-of-00101.safetensors", + "model.layers.16.mlp.experts.99.gate_proj.weight": "model-00017-of-00101.safetensors", + "model.layers.16.mlp.experts.99.up_proj.weight": "model-00017-of-00101.safetensors", + "model.layers.16.mlp.experts.99.down_proj.weight": "model-00017-of-00101.safetensors", + "model.layers.16.mlp.experts.100.gate_proj.weight": "model-00017-of-00101.safetensors", + "model.layers.16.mlp.experts.100.up_proj.weight": "model-00017-of-00101.safetensors", + "model.layers.16.mlp.experts.100.down_proj.weight": "model-00017-of-00101.safetensors", + "model.layers.16.mlp.experts.101.gate_proj.weight": "model-00017-of-00101.safetensors", + "model.layers.16.mlp.experts.101.up_proj.weight": "model-00017-of-00101.safetensors", + "model.layers.16.mlp.experts.101.down_proj.weight": "model-00017-of-00101.safetensors", + "model.layers.16.mlp.experts.102.gate_proj.weight": "model-00017-of-00101.safetensors", + "model.layers.16.mlp.experts.102.up_proj.weight": "model-00017-of-00101.safetensors", + "model.layers.16.mlp.experts.102.down_proj.weight": "model-00017-of-00101.safetensors", + "model.layers.16.mlp.experts.103.gate_proj.weight": "model-00017-of-00101.safetensors", + "model.layers.16.mlp.experts.103.up_proj.weight": "model-00017-of-00101.safetensors", + "model.layers.16.mlp.experts.103.down_proj.weight": "model-00017-of-00101.safetensors", + "model.layers.16.mlp.experts.104.gate_proj.weight": "model-00017-of-00101.safetensors", + "model.layers.16.mlp.experts.104.up_proj.weight": "model-00017-of-00101.safetensors", + "model.layers.16.mlp.experts.104.down_proj.weight": "model-00017-of-00101.safetensors", + "model.layers.16.mlp.experts.105.gate_proj.weight": "model-00017-of-00101.safetensors", + "model.layers.16.mlp.experts.105.up_proj.weight": "model-00017-of-00101.safetensors", + "model.layers.16.mlp.experts.105.down_proj.weight": "model-00017-of-00101.safetensors", + "model.layers.16.mlp.experts.106.gate_proj.weight": "model-00017-of-00101.safetensors", + "model.layers.16.mlp.experts.106.up_proj.weight": "model-00017-of-00101.safetensors", + "model.layers.16.mlp.experts.106.down_proj.weight": "model-00017-of-00101.safetensors", + "model.layers.16.mlp.experts.107.gate_proj.weight": "model-00017-of-00101.safetensors", + "model.layers.16.mlp.experts.107.up_proj.weight": "model-00017-of-00101.safetensors", + "model.layers.16.mlp.experts.107.down_proj.weight": "model-00017-of-00101.safetensors", + "model.layers.16.mlp.experts.108.gate_proj.weight": "model-00017-of-00101.safetensors", + "model.layers.16.mlp.experts.108.up_proj.weight": "model-00017-of-00101.safetensors", + "model.layers.16.mlp.experts.108.down_proj.weight": "model-00017-of-00101.safetensors", + "model.layers.16.mlp.experts.109.gate_proj.weight": "model-00017-of-00101.safetensors", + "model.layers.16.mlp.experts.109.up_proj.weight": "model-00017-of-00101.safetensors", + "model.layers.16.mlp.experts.109.down_proj.weight": "model-00017-of-00101.safetensors", + "model.layers.16.mlp.experts.110.gate_proj.weight": "model-00017-of-00101.safetensors", + "model.layers.16.mlp.experts.110.up_proj.weight": "model-00017-of-00101.safetensors", + "model.layers.16.mlp.experts.110.down_proj.weight": "model-00017-of-00101.safetensors", + "model.layers.16.mlp.experts.111.gate_proj.weight": "model-00017-of-00101.safetensors", + "model.layers.16.mlp.experts.111.up_proj.weight": "model-00017-of-00101.safetensors", + "model.layers.16.mlp.experts.111.down_proj.weight": "model-00017-of-00101.safetensors", + "model.layers.16.mlp.experts.112.gate_proj.weight": "model-00017-of-00101.safetensors", + "model.layers.16.mlp.experts.112.up_proj.weight": "model-00017-of-00101.safetensors", + "model.layers.16.mlp.experts.112.down_proj.weight": "model-00017-of-00101.safetensors", + "model.layers.16.mlp.experts.113.gate_proj.weight": "model-00017-of-00101.safetensors", + "model.layers.16.mlp.experts.113.up_proj.weight": "model-00017-of-00101.safetensors", + "model.layers.16.mlp.experts.113.down_proj.weight": "model-00017-of-00101.safetensors", + "model.layers.16.mlp.experts.114.gate_proj.weight": "model-00017-of-00101.safetensors", + "model.layers.16.mlp.experts.114.up_proj.weight": "model-00017-of-00101.safetensors", + "model.layers.16.mlp.experts.114.down_proj.weight": "model-00017-of-00101.safetensors", + "model.layers.16.mlp.experts.115.gate_proj.weight": "model-00017-of-00101.safetensors", + "model.layers.16.mlp.experts.115.up_proj.weight": "model-00017-of-00101.safetensors", + "model.layers.16.mlp.experts.115.down_proj.weight": "model-00017-of-00101.safetensors", + "model.layers.16.mlp.experts.116.gate_proj.weight": "model-00017-of-00101.safetensors", + "model.layers.16.mlp.experts.116.up_proj.weight": "model-00017-of-00101.safetensors", + "model.layers.16.mlp.experts.116.down_proj.weight": "model-00017-of-00101.safetensors", + "model.layers.16.mlp.experts.117.gate_proj.weight": "model-00017-of-00101.safetensors", + "model.layers.16.mlp.experts.117.up_proj.weight": "model-00017-of-00101.safetensors", + "model.layers.16.mlp.experts.117.down_proj.weight": "model-00017-of-00101.safetensors", + "model.layers.16.mlp.experts.118.gate_proj.weight": "model-00017-of-00101.safetensors", + "model.layers.16.mlp.experts.118.up_proj.weight": "model-00017-of-00101.safetensors", + "model.layers.16.mlp.experts.118.down_proj.weight": "model-00017-of-00101.safetensors", + "model.layers.16.mlp.experts.119.gate_proj.weight": "model-00017-of-00101.safetensors", + "model.layers.16.mlp.experts.119.up_proj.weight": "model-00017-of-00101.safetensors", + "model.layers.16.mlp.experts.119.down_proj.weight": "model-00017-of-00101.safetensors", + "model.layers.16.mlp.gate.weight": "model-00017-of-00101.safetensors", + "model.layers.16.mlp.gate.e_score_correction_bias": "model-00017-of-00101.safetensors", + "model.layers.16.mlp.shared_experts.gate_proj.weight": "model-00017-of-00101.safetensors", + "model.layers.16.mlp.shared_experts.up_proj.weight": "model-00017-of-00101.safetensors", + "model.layers.16.mlp.shared_experts.down_proj.weight": "model-00017-of-00101.safetensors", + "model.layers.16.input_layernorm.weight": "model-00017-of-00101.safetensors", + "model.layers.16.post_attention_layernorm.weight": "model-00017-of-00101.safetensors", + "model.layers.17.self_attn.q_proj.weight": "model-00017-of-00101.safetensors", + "model.layers.17.self_attn.q_proj.bias": "model-00017-of-00101.safetensors", + "model.layers.17.self_attn.k_proj.weight": "model-00017-of-00101.safetensors", + "model.layers.17.self_attn.k_proj.bias": "model-00017-of-00101.safetensors", + "model.layers.17.self_attn.v_proj.weight": "model-00017-of-00101.safetensors", + "model.layers.17.self_attn.v_proj.bias": "model-00017-of-00101.safetensors", + "model.layers.17.self_attn.o_proj.weight": "model-00017-of-00101.safetensors", + "model.layers.17.self_attn.q_norm.weight": "model-00017-of-00101.safetensors", + "model.layers.17.self_attn.k_norm.weight": "model-00017-of-00101.safetensors", + "model.layers.17.mlp.experts.0.gate_proj.weight": "model-00017-of-00101.safetensors", + "model.layers.17.mlp.experts.0.up_proj.weight": "model-00017-of-00101.safetensors", + "model.layers.17.mlp.experts.0.down_proj.weight": "model-00017-of-00101.safetensors", + "model.layers.17.mlp.experts.1.gate_proj.weight": "model-00017-of-00101.safetensors", + "model.layers.17.mlp.experts.1.up_proj.weight": "model-00017-of-00101.safetensors", + "model.layers.17.mlp.experts.1.down_proj.weight": "model-00017-of-00101.safetensors", + "model.layers.17.mlp.experts.2.gate_proj.weight": "model-00017-of-00101.safetensors", + "model.layers.17.mlp.experts.2.up_proj.weight": "model-00017-of-00101.safetensors", + "model.layers.17.mlp.experts.2.down_proj.weight": "model-00017-of-00101.safetensors", + "model.layers.17.mlp.experts.3.gate_proj.weight": "model-00017-of-00101.safetensors", + "model.layers.17.mlp.experts.3.up_proj.weight": "model-00017-of-00101.safetensors", + "model.layers.17.mlp.experts.3.down_proj.weight": "model-00017-of-00101.safetensors", + "model.layers.17.mlp.experts.4.gate_proj.weight": "model-00017-of-00101.safetensors", + "model.layers.17.mlp.experts.4.up_proj.weight": "model-00017-of-00101.safetensors", + "model.layers.17.mlp.experts.4.down_proj.weight": "model-00017-of-00101.safetensors", + "model.layers.17.mlp.experts.5.gate_proj.weight": "model-00017-of-00101.safetensors", + "model.layers.17.mlp.experts.5.up_proj.weight": "model-00017-of-00101.safetensors", + "model.layers.17.mlp.experts.5.down_proj.weight": "model-00017-of-00101.safetensors", + "model.layers.17.mlp.experts.6.gate_proj.weight": "model-00017-of-00101.safetensors", + "model.layers.17.mlp.experts.6.up_proj.weight": "model-00017-of-00101.safetensors", + "model.layers.17.mlp.experts.6.down_proj.weight": "model-00017-of-00101.safetensors", + "model.layers.17.mlp.experts.7.gate_proj.weight": "model-00017-of-00101.safetensors", + "model.layers.17.mlp.experts.7.up_proj.weight": "model-00017-of-00101.safetensors", + "model.layers.17.mlp.experts.7.down_proj.weight": "model-00017-of-00101.safetensors", + "model.layers.17.mlp.experts.8.gate_proj.weight": "model-00017-of-00101.safetensors", + "model.layers.17.mlp.experts.8.up_proj.weight": "model-00017-of-00101.safetensors", + "model.layers.17.mlp.experts.8.down_proj.weight": "model-00017-of-00101.safetensors", + "model.layers.17.mlp.experts.9.gate_proj.weight": "model-00017-of-00101.safetensors", + "model.layers.17.mlp.experts.9.up_proj.weight": "model-00017-of-00101.safetensors", + "model.layers.17.mlp.experts.9.down_proj.weight": "model-00017-of-00101.safetensors", + "model.layers.17.mlp.experts.10.gate_proj.weight": "model-00017-of-00101.safetensors", + "model.layers.17.mlp.experts.10.up_proj.weight": "model-00017-of-00101.safetensors", + "model.layers.17.mlp.experts.10.down_proj.weight": "model-00017-of-00101.safetensors", + "model.layers.17.mlp.experts.11.gate_proj.weight": "model-00017-of-00101.safetensors", + "model.layers.17.mlp.experts.11.up_proj.weight": "model-00017-of-00101.safetensors", + "model.layers.17.mlp.experts.11.down_proj.weight": "model-00017-of-00101.safetensors", + "model.layers.17.mlp.experts.12.gate_proj.weight": "model-00017-of-00101.safetensors", + "model.layers.17.mlp.experts.12.up_proj.weight": "model-00017-of-00101.safetensors", + "model.layers.17.mlp.experts.12.down_proj.weight": "model-00017-of-00101.safetensors", + "model.layers.17.mlp.experts.13.gate_proj.weight": "model-00017-of-00101.safetensors", + "model.layers.17.mlp.experts.13.up_proj.weight": "model-00017-of-00101.safetensors", + "model.layers.17.mlp.experts.13.down_proj.weight": "model-00017-of-00101.safetensors", + "model.layers.17.mlp.experts.14.gate_proj.weight": "model-00017-of-00101.safetensors", + "model.layers.17.mlp.experts.14.up_proj.weight": "model-00017-of-00101.safetensors", + "model.layers.17.mlp.experts.14.down_proj.weight": "model-00017-of-00101.safetensors", + "model.layers.17.mlp.experts.15.gate_proj.weight": "model-00017-of-00101.safetensors", + "model.layers.17.mlp.experts.15.up_proj.weight": "model-00017-of-00101.safetensors", + "model.layers.17.mlp.experts.15.down_proj.weight": "model-00017-of-00101.safetensors", + "model.layers.17.mlp.experts.16.gate_proj.weight": "model-00017-of-00101.safetensors", + "model.layers.17.mlp.experts.16.up_proj.weight": "model-00017-of-00101.safetensors", + "model.layers.17.mlp.experts.16.down_proj.weight": "model-00017-of-00101.safetensors", + "model.layers.17.mlp.experts.17.gate_proj.weight": "model-00017-of-00101.safetensors", + "model.layers.17.mlp.experts.17.up_proj.weight": "model-00017-of-00101.safetensors", + "model.layers.17.mlp.experts.17.down_proj.weight": "model-00017-of-00101.safetensors", + "model.layers.17.mlp.experts.18.gate_proj.weight": "model-00017-of-00101.safetensors", + "model.layers.17.mlp.experts.18.up_proj.weight": "model-00017-of-00101.safetensors", + "model.layers.17.mlp.experts.18.down_proj.weight": "model-00017-of-00101.safetensors", + "model.layers.17.mlp.experts.19.gate_proj.weight": "model-00017-of-00101.safetensors", + "model.layers.17.mlp.experts.19.up_proj.weight": "model-00017-of-00101.safetensors", + "model.layers.17.mlp.experts.19.down_proj.weight": "model-00017-of-00101.safetensors", + "model.layers.17.mlp.experts.20.gate_proj.weight": "model-00017-of-00101.safetensors", + "model.layers.17.mlp.experts.20.up_proj.weight": "model-00017-of-00101.safetensors", + "model.layers.17.mlp.experts.20.down_proj.weight": "model-00017-of-00101.safetensors", + "model.layers.17.mlp.experts.21.gate_proj.weight": "model-00017-of-00101.safetensors", + "model.layers.17.mlp.experts.21.up_proj.weight": "model-00017-of-00101.safetensors", + "model.layers.17.mlp.experts.21.down_proj.weight": "model-00017-of-00101.safetensors", + "model.layers.17.mlp.experts.22.gate_proj.weight": "model-00017-of-00101.safetensors", + "model.layers.17.mlp.experts.22.up_proj.weight": "model-00017-of-00101.safetensors", + "model.layers.17.mlp.experts.22.down_proj.weight": "model-00017-of-00101.safetensors", + "model.layers.17.mlp.experts.23.gate_proj.weight": "model-00017-of-00101.safetensors", + "model.layers.17.mlp.experts.23.up_proj.weight": "model-00017-of-00101.safetensors", + "model.layers.17.mlp.experts.23.down_proj.weight": "model-00017-of-00101.safetensors", + "model.layers.17.mlp.experts.24.gate_proj.weight": "model-00017-of-00101.safetensors", + "model.layers.17.mlp.experts.24.up_proj.weight": "model-00017-of-00101.safetensors", + "model.layers.17.mlp.experts.24.down_proj.weight": "model-00017-of-00101.safetensors", + "model.layers.17.mlp.experts.25.gate_proj.weight": "model-00017-of-00101.safetensors", + "model.layers.17.mlp.experts.25.up_proj.weight": "model-00017-of-00101.safetensors", + "model.layers.17.mlp.experts.25.down_proj.weight": "model-00017-of-00101.safetensors", + "model.layers.17.mlp.experts.26.gate_proj.weight": "model-00017-of-00101.safetensors", + "model.layers.17.mlp.experts.26.up_proj.weight": "model-00017-of-00101.safetensors", + "model.layers.17.mlp.experts.26.down_proj.weight": "model-00017-of-00101.safetensors", + "model.layers.17.mlp.experts.27.gate_proj.weight": "model-00017-of-00101.safetensors", + "model.layers.17.mlp.experts.27.up_proj.weight": "model-00017-of-00101.safetensors", + "model.layers.17.mlp.experts.27.down_proj.weight": "model-00017-of-00101.safetensors", + "model.layers.17.mlp.experts.28.gate_proj.weight": "model-00017-of-00101.safetensors", + "model.layers.17.mlp.experts.28.up_proj.weight": "model-00017-of-00101.safetensors", + "model.layers.17.mlp.experts.28.down_proj.weight": "model-00017-of-00101.safetensors", + "model.layers.17.mlp.experts.29.gate_proj.weight": "model-00017-of-00101.safetensors", + "model.layers.17.mlp.experts.29.up_proj.weight": "model-00017-of-00101.safetensors", + "model.layers.17.mlp.experts.29.down_proj.weight": "model-00017-of-00101.safetensors", + "model.layers.17.mlp.experts.30.gate_proj.weight": "model-00017-of-00101.safetensors", + "model.layers.17.mlp.experts.30.up_proj.weight": "model-00017-of-00101.safetensors", + "model.layers.17.mlp.experts.30.down_proj.weight": "model-00017-of-00101.safetensors", + "model.layers.17.mlp.experts.31.gate_proj.weight": "model-00017-of-00101.safetensors", + "model.layers.17.mlp.experts.31.up_proj.weight": "model-00017-of-00101.safetensors", + "model.layers.17.mlp.experts.31.down_proj.weight": "model-00017-of-00101.safetensors", + "model.layers.17.mlp.experts.32.gate_proj.weight": "model-00017-of-00101.safetensors", + "model.layers.17.mlp.experts.32.up_proj.weight": "model-00017-of-00101.safetensors", + "model.layers.17.mlp.experts.32.down_proj.weight": "model-00017-of-00101.safetensors", + "model.layers.17.mlp.experts.33.gate_proj.weight": "model-00017-of-00101.safetensors", + "model.layers.17.mlp.experts.33.up_proj.weight": "model-00017-of-00101.safetensors", + "model.layers.17.mlp.experts.33.down_proj.weight": "model-00017-of-00101.safetensors", + "model.layers.17.mlp.experts.34.gate_proj.weight": "model-00017-of-00101.safetensors", + "model.layers.17.mlp.experts.34.up_proj.weight": "model-00017-of-00101.safetensors", + "model.layers.17.mlp.experts.34.down_proj.weight": "model-00017-of-00101.safetensors", + "model.layers.17.mlp.experts.35.gate_proj.weight": "model-00017-of-00101.safetensors", + "model.layers.17.mlp.experts.35.up_proj.weight": "model-00017-of-00101.safetensors", + "model.layers.17.mlp.experts.35.down_proj.weight": "model-00017-of-00101.safetensors", + "model.layers.17.mlp.experts.36.gate_proj.weight": "model-00017-of-00101.safetensors", + "model.layers.17.mlp.experts.36.up_proj.weight": "model-00017-of-00101.safetensors", + "model.layers.17.mlp.experts.36.down_proj.weight": "model-00017-of-00101.safetensors", + "model.layers.17.mlp.experts.37.gate_proj.weight": "model-00017-of-00101.safetensors", + "model.layers.17.mlp.experts.37.up_proj.weight": "model-00017-of-00101.safetensors", + "model.layers.17.mlp.experts.37.down_proj.weight": "model-00017-of-00101.safetensors", + "model.layers.17.mlp.experts.38.gate_proj.weight": "model-00017-of-00101.safetensors", + "model.layers.17.mlp.experts.38.up_proj.weight": "model-00017-of-00101.safetensors", + "model.layers.17.mlp.experts.38.down_proj.weight": "model-00017-of-00101.safetensors", + "model.layers.17.mlp.experts.39.gate_proj.weight": "model-00017-of-00101.safetensors", + "model.layers.17.mlp.experts.39.up_proj.weight": "model-00017-of-00101.safetensors", + "model.layers.17.mlp.experts.39.down_proj.weight": "model-00017-of-00101.safetensors", + "model.layers.17.mlp.experts.40.gate_proj.weight": "model-00017-of-00101.safetensors", + "model.layers.17.mlp.experts.40.up_proj.weight": "model-00017-of-00101.safetensors", + "model.layers.17.mlp.experts.40.down_proj.weight": "model-00017-of-00101.safetensors", + "model.layers.17.mlp.experts.41.gate_proj.weight": "model-00017-of-00101.safetensors", + "model.layers.17.mlp.experts.41.up_proj.weight": "model-00017-of-00101.safetensors", + "model.layers.17.mlp.experts.41.down_proj.weight": "model-00017-of-00101.safetensors", + "model.layers.17.mlp.experts.42.gate_proj.weight": "model-00017-of-00101.safetensors", + "model.layers.17.mlp.experts.42.up_proj.weight": "model-00017-of-00101.safetensors", + "model.layers.17.mlp.experts.42.down_proj.weight": "model-00017-of-00101.safetensors", + "model.layers.17.mlp.experts.43.gate_proj.weight": "model-00017-of-00101.safetensors", + "model.layers.17.mlp.experts.43.up_proj.weight": "model-00017-of-00101.safetensors", + "model.layers.17.mlp.experts.43.down_proj.weight": "model-00017-of-00101.safetensors", + "model.layers.17.mlp.experts.44.gate_proj.weight": "model-00017-of-00101.safetensors", + "model.layers.17.mlp.experts.44.up_proj.weight": "model-00017-of-00101.safetensors", + "model.layers.17.mlp.experts.44.down_proj.weight": "model-00017-of-00101.safetensors", + "model.layers.17.mlp.experts.45.gate_proj.weight": "model-00017-of-00101.safetensors", + "model.layers.17.mlp.experts.45.up_proj.weight": "model-00017-of-00101.safetensors", + "model.layers.17.mlp.experts.45.down_proj.weight": "model-00017-of-00101.safetensors", + "model.layers.17.mlp.experts.46.gate_proj.weight": "model-00017-of-00101.safetensors", + "model.layers.17.mlp.experts.46.up_proj.weight": "model-00017-of-00101.safetensors", + "model.layers.17.mlp.experts.46.down_proj.weight": "model-00017-of-00101.safetensors", + "model.layers.17.mlp.experts.47.gate_proj.weight": "model-00017-of-00101.safetensors", + "model.layers.17.mlp.experts.47.up_proj.weight": "model-00017-of-00101.safetensors", + "model.layers.17.mlp.experts.47.down_proj.weight": "model-00017-of-00101.safetensors", + "model.layers.17.mlp.experts.48.gate_proj.weight": "model-00017-of-00101.safetensors", + "model.layers.17.mlp.experts.48.up_proj.weight": "model-00017-of-00101.safetensors", + "model.layers.17.mlp.experts.48.down_proj.weight": "model-00017-of-00101.safetensors", + "model.layers.17.mlp.experts.49.gate_proj.weight": "model-00017-of-00101.safetensors", + "model.layers.17.mlp.experts.49.up_proj.weight": "model-00017-of-00101.safetensors", + "model.layers.17.mlp.experts.49.down_proj.weight": "model-00017-of-00101.safetensors", + "model.layers.17.mlp.experts.50.gate_proj.weight": "model-00017-of-00101.safetensors", + "model.layers.17.mlp.experts.50.up_proj.weight": "model-00017-of-00101.safetensors", + "model.layers.17.mlp.experts.50.down_proj.weight": "model-00017-of-00101.safetensors", + "model.layers.17.mlp.experts.51.gate_proj.weight": "model-00017-of-00101.safetensors", + "model.layers.17.mlp.experts.51.up_proj.weight": "model-00017-of-00101.safetensors", + "model.layers.17.mlp.experts.51.down_proj.weight": "model-00017-of-00101.safetensors", + "model.layers.17.mlp.experts.52.gate_proj.weight": "model-00017-of-00101.safetensors", + "model.layers.17.mlp.experts.52.up_proj.weight": "model-00017-of-00101.safetensors", + "model.layers.17.mlp.experts.52.down_proj.weight": "model-00017-of-00101.safetensors", + "model.layers.17.mlp.experts.53.gate_proj.weight": "model-00017-of-00101.safetensors", + "model.layers.17.mlp.experts.53.up_proj.weight": "model-00017-of-00101.safetensors", + "model.layers.17.mlp.experts.53.down_proj.weight": "model-00017-of-00101.safetensors", + "model.layers.17.mlp.experts.54.gate_proj.weight": "model-00017-of-00101.safetensors", + "model.layers.17.mlp.experts.54.up_proj.weight": "model-00017-of-00101.safetensors", + "model.layers.17.mlp.experts.54.down_proj.weight": "model-00017-of-00101.safetensors", + "model.layers.17.mlp.experts.55.gate_proj.weight": "model-00017-of-00101.safetensors", + "model.layers.17.mlp.experts.55.up_proj.weight": "model-00017-of-00101.safetensors", + "model.layers.17.mlp.experts.55.down_proj.weight": "model-00017-of-00101.safetensors", + "model.layers.17.mlp.experts.56.gate_proj.weight": "model-00017-of-00101.safetensors", + "model.layers.17.mlp.experts.56.up_proj.weight": "model-00017-of-00101.safetensors", + "model.layers.17.mlp.experts.56.down_proj.weight": "model-00017-of-00101.safetensors", + "model.layers.17.mlp.experts.57.gate_proj.weight": "model-00017-of-00101.safetensors", + "model.layers.17.mlp.experts.57.up_proj.weight": "model-00017-of-00101.safetensors", + "model.layers.17.mlp.experts.57.down_proj.weight": "model-00017-of-00101.safetensors", + "model.layers.17.mlp.experts.58.gate_proj.weight": "model-00017-of-00101.safetensors", + "model.layers.17.mlp.experts.58.up_proj.weight": "model-00017-of-00101.safetensors", + "model.layers.17.mlp.experts.58.down_proj.weight": "model-00017-of-00101.safetensors", + "model.layers.17.mlp.experts.59.gate_proj.weight": "model-00017-of-00101.safetensors", + "model.layers.17.mlp.experts.59.up_proj.weight": "model-00017-of-00101.safetensors", + "model.layers.17.mlp.experts.59.down_proj.weight": "model-00017-of-00101.safetensors", + "model.layers.17.mlp.experts.60.gate_proj.weight": "model-00017-of-00101.safetensors", + "model.layers.17.mlp.experts.60.up_proj.weight": "model-00017-of-00101.safetensors", + "model.layers.17.mlp.experts.60.down_proj.weight": "model-00017-of-00101.safetensors", + "model.layers.17.mlp.experts.61.gate_proj.weight": "model-00017-of-00101.safetensors", + "model.layers.17.mlp.experts.61.up_proj.weight": "model-00017-of-00101.safetensors", + "model.layers.17.mlp.experts.61.down_proj.weight": "model-00017-of-00101.safetensors", + "model.layers.17.mlp.experts.62.gate_proj.weight": "model-00017-of-00101.safetensors", + "model.layers.17.mlp.experts.62.up_proj.weight": "model-00017-of-00101.safetensors", + "model.layers.17.mlp.experts.62.down_proj.weight": "model-00017-of-00101.safetensors", + "model.layers.17.mlp.experts.63.gate_proj.weight": "model-00017-of-00101.safetensors", + "model.layers.17.mlp.experts.63.up_proj.weight": "model-00017-of-00101.safetensors", + "model.layers.17.mlp.experts.63.down_proj.weight": "model-00017-of-00101.safetensors", + "model.layers.17.mlp.experts.64.gate_proj.weight": "model-00017-of-00101.safetensors", + "model.layers.17.mlp.experts.64.up_proj.weight": "model-00017-of-00101.safetensors", + "model.layers.17.mlp.experts.64.down_proj.weight": "model-00017-of-00101.safetensors", + "model.layers.17.mlp.experts.65.gate_proj.weight": "model-00017-of-00101.safetensors", + "model.layers.17.mlp.experts.65.up_proj.weight": "model-00017-of-00101.safetensors", + "model.layers.17.mlp.experts.65.down_proj.weight": "model-00017-of-00101.safetensors", + "model.layers.17.mlp.experts.66.gate_proj.weight": "model-00017-of-00101.safetensors", + "model.layers.17.mlp.experts.66.up_proj.weight": "model-00017-of-00101.safetensors", + "model.layers.17.mlp.experts.66.down_proj.weight": "model-00017-of-00101.safetensors", + "model.layers.17.mlp.experts.67.gate_proj.weight": "model-00017-of-00101.safetensors", + "model.layers.17.mlp.experts.67.up_proj.weight": "model-00017-of-00101.safetensors", + "model.layers.17.mlp.experts.67.down_proj.weight": "model-00017-of-00101.safetensors", + "model.layers.17.mlp.experts.68.gate_proj.weight": "model-00017-of-00101.safetensors", + "model.layers.17.mlp.experts.68.up_proj.weight": "model-00017-of-00101.safetensors", + "model.layers.17.mlp.experts.68.down_proj.weight": "model-00017-of-00101.safetensors", + "model.layers.17.mlp.experts.69.gate_proj.weight": "model-00017-of-00101.safetensors", + "model.layers.17.mlp.experts.69.up_proj.weight": "model-00017-of-00101.safetensors", + "model.layers.17.mlp.experts.69.down_proj.weight": "model-00017-of-00101.safetensors", + "model.layers.17.mlp.experts.70.gate_proj.weight": "model-00017-of-00101.safetensors", + "model.layers.17.mlp.experts.70.up_proj.weight": "model-00017-of-00101.safetensors", + "model.layers.17.mlp.experts.70.down_proj.weight": "model-00017-of-00101.safetensors", + "model.layers.17.mlp.experts.71.gate_proj.weight": "model-00017-of-00101.safetensors", + "model.layers.17.mlp.experts.71.up_proj.weight": "model-00017-of-00101.safetensors", + "model.layers.17.mlp.experts.71.down_proj.weight": "model-00017-of-00101.safetensors", + "model.layers.17.mlp.experts.72.gate_proj.weight": "model-00017-of-00101.safetensors", + "model.layers.17.mlp.experts.72.up_proj.weight": "model-00017-of-00101.safetensors", + "model.layers.17.mlp.experts.72.down_proj.weight": "model-00017-of-00101.safetensors", + "model.layers.17.mlp.experts.73.gate_proj.weight": "model-00017-of-00101.safetensors", + "model.layers.17.mlp.experts.73.up_proj.weight": "model-00017-of-00101.safetensors", + "model.layers.17.mlp.experts.73.down_proj.weight": "model-00017-of-00101.safetensors", + "model.layers.17.mlp.experts.74.gate_proj.weight": "model-00017-of-00101.safetensors", + "model.layers.17.mlp.experts.74.up_proj.weight": "model-00018-of-00101.safetensors", + "model.layers.17.mlp.experts.74.down_proj.weight": "model-00018-of-00101.safetensors", + "model.layers.17.mlp.experts.75.gate_proj.weight": "model-00018-of-00101.safetensors", + "model.layers.17.mlp.experts.75.up_proj.weight": "model-00018-of-00101.safetensors", + "model.layers.17.mlp.experts.75.down_proj.weight": "model-00018-of-00101.safetensors", + "model.layers.17.mlp.experts.76.gate_proj.weight": "model-00018-of-00101.safetensors", + "model.layers.17.mlp.experts.76.up_proj.weight": "model-00018-of-00101.safetensors", + "model.layers.17.mlp.experts.76.down_proj.weight": "model-00018-of-00101.safetensors", + "model.layers.17.mlp.experts.77.gate_proj.weight": "model-00018-of-00101.safetensors", + "model.layers.17.mlp.experts.77.up_proj.weight": "model-00018-of-00101.safetensors", + "model.layers.17.mlp.experts.77.down_proj.weight": "model-00018-of-00101.safetensors", + "model.layers.17.mlp.experts.78.gate_proj.weight": "model-00018-of-00101.safetensors", + "model.layers.17.mlp.experts.78.up_proj.weight": "model-00018-of-00101.safetensors", + "model.layers.17.mlp.experts.78.down_proj.weight": "model-00018-of-00101.safetensors", + "model.layers.17.mlp.experts.79.gate_proj.weight": "model-00018-of-00101.safetensors", + "model.layers.17.mlp.experts.79.up_proj.weight": "model-00018-of-00101.safetensors", + "model.layers.17.mlp.experts.79.down_proj.weight": "model-00018-of-00101.safetensors", + "model.layers.17.mlp.experts.80.gate_proj.weight": "model-00018-of-00101.safetensors", + "model.layers.17.mlp.experts.80.up_proj.weight": "model-00018-of-00101.safetensors", + "model.layers.17.mlp.experts.80.down_proj.weight": "model-00018-of-00101.safetensors", + "model.layers.17.mlp.experts.81.gate_proj.weight": "model-00018-of-00101.safetensors", + "model.layers.17.mlp.experts.81.up_proj.weight": "model-00018-of-00101.safetensors", + "model.layers.17.mlp.experts.81.down_proj.weight": "model-00018-of-00101.safetensors", + "model.layers.17.mlp.experts.82.gate_proj.weight": "model-00018-of-00101.safetensors", + "model.layers.17.mlp.experts.82.up_proj.weight": "model-00018-of-00101.safetensors", + "model.layers.17.mlp.experts.82.down_proj.weight": "model-00018-of-00101.safetensors", + "model.layers.17.mlp.experts.83.gate_proj.weight": "model-00018-of-00101.safetensors", + "model.layers.17.mlp.experts.83.up_proj.weight": "model-00018-of-00101.safetensors", + "model.layers.17.mlp.experts.83.down_proj.weight": "model-00018-of-00101.safetensors", + "model.layers.17.mlp.experts.84.gate_proj.weight": "model-00018-of-00101.safetensors", + "model.layers.17.mlp.experts.84.up_proj.weight": "model-00018-of-00101.safetensors", + "model.layers.17.mlp.experts.84.down_proj.weight": "model-00018-of-00101.safetensors", + "model.layers.17.mlp.experts.85.gate_proj.weight": "model-00018-of-00101.safetensors", + "model.layers.17.mlp.experts.85.up_proj.weight": "model-00018-of-00101.safetensors", + "model.layers.17.mlp.experts.85.down_proj.weight": "model-00018-of-00101.safetensors", + "model.layers.17.mlp.experts.86.gate_proj.weight": "model-00018-of-00101.safetensors", + "model.layers.17.mlp.experts.86.up_proj.weight": "model-00018-of-00101.safetensors", + "model.layers.17.mlp.experts.86.down_proj.weight": "model-00018-of-00101.safetensors", + "model.layers.17.mlp.experts.87.gate_proj.weight": "model-00018-of-00101.safetensors", + "model.layers.17.mlp.experts.87.up_proj.weight": "model-00018-of-00101.safetensors", + "model.layers.17.mlp.experts.87.down_proj.weight": "model-00018-of-00101.safetensors", + "model.layers.17.mlp.experts.88.gate_proj.weight": "model-00018-of-00101.safetensors", + "model.layers.17.mlp.experts.88.up_proj.weight": "model-00018-of-00101.safetensors", + "model.layers.17.mlp.experts.88.down_proj.weight": "model-00018-of-00101.safetensors", + "model.layers.17.mlp.experts.89.gate_proj.weight": "model-00018-of-00101.safetensors", + "model.layers.17.mlp.experts.89.up_proj.weight": "model-00018-of-00101.safetensors", + "model.layers.17.mlp.experts.89.down_proj.weight": "model-00018-of-00101.safetensors", + "model.layers.17.mlp.experts.90.gate_proj.weight": "model-00018-of-00101.safetensors", + "model.layers.17.mlp.experts.90.up_proj.weight": "model-00018-of-00101.safetensors", + "model.layers.17.mlp.experts.90.down_proj.weight": "model-00018-of-00101.safetensors", + "model.layers.17.mlp.experts.91.gate_proj.weight": "model-00018-of-00101.safetensors", + "model.layers.17.mlp.experts.91.up_proj.weight": "model-00018-of-00101.safetensors", + "model.layers.17.mlp.experts.91.down_proj.weight": "model-00018-of-00101.safetensors", + "model.layers.17.mlp.experts.92.gate_proj.weight": "model-00018-of-00101.safetensors", + "model.layers.17.mlp.experts.92.up_proj.weight": "model-00018-of-00101.safetensors", + "model.layers.17.mlp.experts.92.down_proj.weight": "model-00018-of-00101.safetensors", + "model.layers.17.mlp.experts.93.gate_proj.weight": "model-00018-of-00101.safetensors", + "model.layers.17.mlp.experts.93.up_proj.weight": "model-00018-of-00101.safetensors", + "model.layers.17.mlp.experts.93.down_proj.weight": "model-00018-of-00101.safetensors", + "model.layers.17.mlp.experts.94.gate_proj.weight": "model-00018-of-00101.safetensors", + "model.layers.17.mlp.experts.94.up_proj.weight": "model-00018-of-00101.safetensors", + "model.layers.17.mlp.experts.94.down_proj.weight": "model-00018-of-00101.safetensors", + "model.layers.17.mlp.experts.95.gate_proj.weight": "model-00018-of-00101.safetensors", + "model.layers.17.mlp.experts.95.up_proj.weight": "model-00018-of-00101.safetensors", + "model.layers.17.mlp.experts.95.down_proj.weight": "model-00018-of-00101.safetensors", + "model.layers.17.mlp.experts.96.gate_proj.weight": "model-00018-of-00101.safetensors", + "model.layers.17.mlp.experts.96.up_proj.weight": "model-00018-of-00101.safetensors", + "model.layers.17.mlp.experts.96.down_proj.weight": "model-00018-of-00101.safetensors", + "model.layers.17.mlp.experts.97.gate_proj.weight": "model-00018-of-00101.safetensors", + "model.layers.17.mlp.experts.97.up_proj.weight": "model-00018-of-00101.safetensors", + "model.layers.17.mlp.experts.97.down_proj.weight": "model-00018-of-00101.safetensors", + "model.layers.17.mlp.experts.98.gate_proj.weight": "model-00018-of-00101.safetensors", + "model.layers.17.mlp.experts.98.up_proj.weight": "model-00018-of-00101.safetensors", + "model.layers.17.mlp.experts.98.down_proj.weight": "model-00018-of-00101.safetensors", + "model.layers.17.mlp.experts.99.gate_proj.weight": "model-00018-of-00101.safetensors", + "model.layers.17.mlp.experts.99.up_proj.weight": "model-00018-of-00101.safetensors", + "model.layers.17.mlp.experts.99.down_proj.weight": "model-00018-of-00101.safetensors", + "model.layers.17.mlp.experts.100.gate_proj.weight": "model-00018-of-00101.safetensors", + "model.layers.17.mlp.experts.100.up_proj.weight": "model-00018-of-00101.safetensors", + "model.layers.17.mlp.experts.100.down_proj.weight": "model-00018-of-00101.safetensors", + "model.layers.17.mlp.experts.101.gate_proj.weight": "model-00018-of-00101.safetensors", + "model.layers.17.mlp.experts.101.up_proj.weight": "model-00018-of-00101.safetensors", + "model.layers.17.mlp.experts.101.down_proj.weight": "model-00018-of-00101.safetensors", + "model.layers.17.mlp.experts.102.gate_proj.weight": "model-00018-of-00101.safetensors", + "model.layers.17.mlp.experts.102.up_proj.weight": "model-00018-of-00101.safetensors", + "model.layers.17.mlp.experts.102.down_proj.weight": "model-00018-of-00101.safetensors", + "model.layers.17.mlp.experts.103.gate_proj.weight": "model-00018-of-00101.safetensors", + "model.layers.17.mlp.experts.103.up_proj.weight": "model-00018-of-00101.safetensors", + "model.layers.17.mlp.experts.103.down_proj.weight": "model-00018-of-00101.safetensors", + "model.layers.17.mlp.experts.104.gate_proj.weight": "model-00018-of-00101.safetensors", + "model.layers.17.mlp.experts.104.up_proj.weight": "model-00018-of-00101.safetensors", + "model.layers.17.mlp.experts.104.down_proj.weight": "model-00018-of-00101.safetensors", + "model.layers.17.mlp.experts.105.gate_proj.weight": "model-00018-of-00101.safetensors", + "model.layers.17.mlp.experts.105.up_proj.weight": "model-00018-of-00101.safetensors", + "model.layers.17.mlp.experts.105.down_proj.weight": "model-00018-of-00101.safetensors", + "model.layers.17.mlp.experts.106.gate_proj.weight": "model-00018-of-00101.safetensors", + "model.layers.17.mlp.experts.106.up_proj.weight": "model-00018-of-00101.safetensors", + "model.layers.17.mlp.experts.106.down_proj.weight": "model-00018-of-00101.safetensors", + "model.layers.17.mlp.experts.107.gate_proj.weight": "model-00018-of-00101.safetensors", + "model.layers.17.mlp.experts.107.up_proj.weight": "model-00018-of-00101.safetensors", + "model.layers.17.mlp.experts.107.down_proj.weight": "model-00018-of-00101.safetensors", + "model.layers.17.mlp.experts.108.gate_proj.weight": "model-00018-of-00101.safetensors", + "model.layers.17.mlp.experts.108.up_proj.weight": "model-00018-of-00101.safetensors", + "model.layers.17.mlp.experts.108.down_proj.weight": "model-00018-of-00101.safetensors", + "model.layers.17.mlp.experts.109.gate_proj.weight": "model-00018-of-00101.safetensors", + "model.layers.17.mlp.experts.109.up_proj.weight": "model-00018-of-00101.safetensors", + "model.layers.17.mlp.experts.109.down_proj.weight": "model-00018-of-00101.safetensors", + "model.layers.17.mlp.experts.110.gate_proj.weight": "model-00018-of-00101.safetensors", + "model.layers.17.mlp.experts.110.up_proj.weight": "model-00018-of-00101.safetensors", + "model.layers.17.mlp.experts.110.down_proj.weight": "model-00018-of-00101.safetensors", + "model.layers.17.mlp.experts.111.gate_proj.weight": "model-00018-of-00101.safetensors", + "model.layers.17.mlp.experts.111.up_proj.weight": "model-00018-of-00101.safetensors", + "model.layers.17.mlp.experts.111.down_proj.weight": "model-00018-of-00101.safetensors", + "model.layers.17.mlp.experts.112.gate_proj.weight": "model-00018-of-00101.safetensors", + "model.layers.17.mlp.experts.112.up_proj.weight": "model-00018-of-00101.safetensors", + "model.layers.17.mlp.experts.112.down_proj.weight": "model-00018-of-00101.safetensors", + "model.layers.17.mlp.experts.113.gate_proj.weight": "model-00018-of-00101.safetensors", + "model.layers.17.mlp.experts.113.up_proj.weight": "model-00018-of-00101.safetensors", + "model.layers.17.mlp.experts.113.down_proj.weight": "model-00018-of-00101.safetensors", + "model.layers.17.mlp.experts.114.gate_proj.weight": "model-00018-of-00101.safetensors", + "model.layers.17.mlp.experts.114.up_proj.weight": "model-00018-of-00101.safetensors", + "model.layers.17.mlp.experts.114.down_proj.weight": "model-00018-of-00101.safetensors", + "model.layers.17.mlp.experts.115.gate_proj.weight": "model-00018-of-00101.safetensors", + "model.layers.17.mlp.experts.115.up_proj.weight": "model-00018-of-00101.safetensors", + "model.layers.17.mlp.experts.115.down_proj.weight": "model-00018-of-00101.safetensors", + "model.layers.17.mlp.experts.116.gate_proj.weight": "model-00018-of-00101.safetensors", + "model.layers.17.mlp.experts.116.up_proj.weight": "model-00018-of-00101.safetensors", + "model.layers.17.mlp.experts.116.down_proj.weight": "model-00018-of-00101.safetensors", + "model.layers.17.mlp.experts.117.gate_proj.weight": "model-00018-of-00101.safetensors", + "model.layers.17.mlp.experts.117.up_proj.weight": "model-00018-of-00101.safetensors", + "model.layers.17.mlp.experts.117.down_proj.weight": "model-00018-of-00101.safetensors", + "model.layers.17.mlp.experts.118.gate_proj.weight": "model-00018-of-00101.safetensors", + "model.layers.17.mlp.experts.118.up_proj.weight": "model-00018-of-00101.safetensors", + "model.layers.17.mlp.experts.118.down_proj.weight": "model-00018-of-00101.safetensors", + "model.layers.17.mlp.experts.119.gate_proj.weight": "model-00018-of-00101.safetensors", + "model.layers.17.mlp.experts.119.up_proj.weight": "model-00018-of-00101.safetensors", + "model.layers.17.mlp.experts.119.down_proj.weight": "model-00018-of-00101.safetensors", + "model.layers.17.mlp.gate.weight": "model-00018-of-00101.safetensors", + "model.layers.17.mlp.gate.e_score_correction_bias": "model-00018-of-00101.safetensors", + "model.layers.17.mlp.shared_experts.gate_proj.weight": "model-00018-of-00101.safetensors", + "model.layers.17.mlp.shared_experts.up_proj.weight": "model-00018-of-00101.safetensors", + "model.layers.17.mlp.shared_experts.down_proj.weight": "model-00018-of-00101.safetensors", + "model.layers.17.input_layernorm.weight": "model-00018-of-00101.safetensors", + "model.layers.17.post_attention_layernorm.weight": "model-00018-of-00101.safetensors", + "model.layers.18.self_attn.q_proj.weight": "model-00018-of-00101.safetensors", + "model.layers.18.self_attn.q_proj.bias": "model-00018-of-00101.safetensors", + "model.layers.18.self_attn.k_proj.weight": "model-00018-of-00101.safetensors", + "model.layers.18.self_attn.k_proj.bias": "model-00018-of-00101.safetensors", + "model.layers.18.self_attn.v_proj.weight": "model-00018-of-00101.safetensors", + "model.layers.18.self_attn.v_proj.bias": "model-00018-of-00101.safetensors", + "model.layers.18.self_attn.o_proj.weight": "model-00018-of-00101.safetensors", + "model.layers.18.self_attn.q_norm.weight": "model-00018-of-00101.safetensors", + "model.layers.18.self_attn.k_norm.weight": "model-00018-of-00101.safetensors", + "model.layers.18.mlp.experts.0.gate_proj.weight": "model-00018-of-00101.safetensors", + "model.layers.18.mlp.experts.0.up_proj.weight": "model-00018-of-00101.safetensors", + "model.layers.18.mlp.experts.0.down_proj.weight": "model-00018-of-00101.safetensors", + "model.layers.18.mlp.experts.1.gate_proj.weight": "model-00018-of-00101.safetensors", + "model.layers.18.mlp.experts.1.up_proj.weight": "model-00018-of-00101.safetensors", + "model.layers.18.mlp.experts.1.down_proj.weight": "model-00018-of-00101.safetensors", + "model.layers.18.mlp.experts.2.gate_proj.weight": "model-00018-of-00101.safetensors", + "model.layers.18.mlp.experts.2.up_proj.weight": "model-00018-of-00101.safetensors", + "model.layers.18.mlp.experts.2.down_proj.weight": "model-00018-of-00101.safetensors", + "model.layers.18.mlp.experts.3.gate_proj.weight": "model-00018-of-00101.safetensors", + "model.layers.18.mlp.experts.3.up_proj.weight": "model-00018-of-00101.safetensors", + "model.layers.18.mlp.experts.3.down_proj.weight": "model-00018-of-00101.safetensors", + "model.layers.18.mlp.experts.4.gate_proj.weight": "model-00018-of-00101.safetensors", + "model.layers.18.mlp.experts.4.up_proj.weight": "model-00018-of-00101.safetensors", + "model.layers.18.mlp.experts.4.down_proj.weight": "model-00018-of-00101.safetensors", + "model.layers.18.mlp.experts.5.gate_proj.weight": "model-00018-of-00101.safetensors", + "model.layers.18.mlp.experts.5.up_proj.weight": "model-00018-of-00101.safetensors", + "model.layers.18.mlp.experts.5.down_proj.weight": "model-00018-of-00101.safetensors", + "model.layers.18.mlp.experts.6.gate_proj.weight": "model-00018-of-00101.safetensors", + "model.layers.18.mlp.experts.6.up_proj.weight": "model-00018-of-00101.safetensors", + "model.layers.18.mlp.experts.6.down_proj.weight": "model-00018-of-00101.safetensors", + "model.layers.18.mlp.experts.7.gate_proj.weight": "model-00018-of-00101.safetensors", + "model.layers.18.mlp.experts.7.up_proj.weight": "model-00018-of-00101.safetensors", + "model.layers.18.mlp.experts.7.down_proj.weight": "model-00018-of-00101.safetensors", + "model.layers.18.mlp.experts.8.gate_proj.weight": "model-00018-of-00101.safetensors", + "model.layers.18.mlp.experts.8.up_proj.weight": "model-00018-of-00101.safetensors", + "model.layers.18.mlp.experts.8.down_proj.weight": "model-00018-of-00101.safetensors", + "model.layers.18.mlp.experts.9.gate_proj.weight": "model-00018-of-00101.safetensors", + "model.layers.18.mlp.experts.9.up_proj.weight": "model-00018-of-00101.safetensors", + "model.layers.18.mlp.experts.9.down_proj.weight": "model-00018-of-00101.safetensors", + "model.layers.18.mlp.experts.10.gate_proj.weight": "model-00018-of-00101.safetensors", + "model.layers.18.mlp.experts.10.up_proj.weight": "model-00018-of-00101.safetensors", + "model.layers.18.mlp.experts.10.down_proj.weight": "model-00018-of-00101.safetensors", + "model.layers.18.mlp.experts.11.gate_proj.weight": "model-00018-of-00101.safetensors", + "model.layers.18.mlp.experts.11.up_proj.weight": "model-00018-of-00101.safetensors", + "model.layers.18.mlp.experts.11.down_proj.weight": "model-00018-of-00101.safetensors", + "model.layers.18.mlp.experts.12.gate_proj.weight": "model-00018-of-00101.safetensors", + "model.layers.18.mlp.experts.12.up_proj.weight": "model-00018-of-00101.safetensors", + "model.layers.18.mlp.experts.12.down_proj.weight": "model-00018-of-00101.safetensors", + "model.layers.18.mlp.experts.13.gate_proj.weight": "model-00018-of-00101.safetensors", + "model.layers.18.mlp.experts.13.up_proj.weight": "model-00018-of-00101.safetensors", + "model.layers.18.mlp.experts.13.down_proj.weight": "model-00018-of-00101.safetensors", + "model.layers.18.mlp.experts.14.gate_proj.weight": "model-00018-of-00101.safetensors", + "model.layers.18.mlp.experts.14.up_proj.weight": "model-00018-of-00101.safetensors", + "model.layers.18.mlp.experts.14.down_proj.weight": "model-00018-of-00101.safetensors", + "model.layers.18.mlp.experts.15.gate_proj.weight": "model-00018-of-00101.safetensors", + "model.layers.18.mlp.experts.15.up_proj.weight": "model-00018-of-00101.safetensors", + "model.layers.18.mlp.experts.15.down_proj.weight": "model-00018-of-00101.safetensors", + "model.layers.18.mlp.experts.16.gate_proj.weight": "model-00018-of-00101.safetensors", + "model.layers.18.mlp.experts.16.up_proj.weight": "model-00018-of-00101.safetensors", + "model.layers.18.mlp.experts.16.down_proj.weight": "model-00018-of-00101.safetensors", + "model.layers.18.mlp.experts.17.gate_proj.weight": "model-00018-of-00101.safetensors", + "model.layers.18.mlp.experts.17.up_proj.weight": "model-00018-of-00101.safetensors", + "model.layers.18.mlp.experts.17.down_proj.weight": "model-00018-of-00101.safetensors", + "model.layers.18.mlp.experts.18.gate_proj.weight": "model-00018-of-00101.safetensors", + "model.layers.18.mlp.experts.18.up_proj.weight": "model-00018-of-00101.safetensors", + "model.layers.18.mlp.experts.18.down_proj.weight": "model-00018-of-00101.safetensors", + "model.layers.18.mlp.experts.19.gate_proj.weight": "model-00018-of-00101.safetensors", + "model.layers.18.mlp.experts.19.up_proj.weight": "model-00018-of-00101.safetensors", + "model.layers.18.mlp.experts.19.down_proj.weight": "model-00018-of-00101.safetensors", + "model.layers.18.mlp.experts.20.gate_proj.weight": "model-00018-of-00101.safetensors", + "model.layers.18.mlp.experts.20.up_proj.weight": "model-00018-of-00101.safetensors", + "model.layers.18.mlp.experts.20.down_proj.weight": "model-00018-of-00101.safetensors", + "model.layers.18.mlp.experts.21.gate_proj.weight": "model-00018-of-00101.safetensors", + "model.layers.18.mlp.experts.21.up_proj.weight": "model-00018-of-00101.safetensors", + "model.layers.18.mlp.experts.21.down_proj.weight": "model-00018-of-00101.safetensors", + "model.layers.18.mlp.experts.22.gate_proj.weight": "model-00018-of-00101.safetensors", + "model.layers.18.mlp.experts.22.up_proj.weight": "model-00018-of-00101.safetensors", + "model.layers.18.mlp.experts.22.down_proj.weight": "model-00018-of-00101.safetensors", + "model.layers.18.mlp.experts.23.gate_proj.weight": "model-00018-of-00101.safetensors", + "model.layers.18.mlp.experts.23.up_proj.weight": "model-00018-of-00101.safetensors", + "model.layers.18.mlp.experts.23.down_proj.weight": "model-00018-of-00101.safetensors", + "model.layers.18.mlp.experts.24.gate_proj.weight": "model-00018-of-00101.safetensors", + "model.layers.18.mlp.experts.24.up_proj.weight": "model-00018-of-00101.safetensors", + "model.layers.18.mlp.experts.24.down_proj.weight": "model-00018-of-00101.safetensors", + "model.layers.18.mlp.experts.25.gate_proj.weight": "model-00018-of-00101.safetensors", + "model.layers.18.mlp.experts.25.up_proj.weight": "model-00018-of-00101.safetensors", + "model.layers.18.mlp.experts.25.down_proj.weight": "model-00018-of-00101.safetensors", + "model.layers.18.mlp.experts.26.gate_proj.weight": "model-00018-of-00101.safetensors", + "model.layers.18.mlp.experts.26.up_proj.weight": "model-00018-of-00101.safetensors", + "model.layers.18.mlp.experts.26.down_proj.weight": "model-00018-of-00101.safetensors", + "model.layers.18.mlp.experts.27.gate_proj.weight": "model-00018-of-00101.safetensors", + "model.layers.18.mlp.experts.27.up_proj.weight": "model-00018-of-00101.safetensors", + "model.layers.18.mlp.experts.27.down_proj.weight": "model-00018-of-00101.safetensors", + "model.layers.18.mlp.experts.28.gate_proj.weight": "model-00018-of-00101.safetensors", + "model.layers.18.mlp.experts.28.up_proj.weight": "model-00018-of-00101.safetensors", + "model.layers.18.mlp.experts.28.down_proj.weight": "model-00018-of-00101.safetensors", + "model.layers.18.mlp.experts.29.gate_proj.weight": "model-00018-of-00101.safetensors", + "model.layers.18.mlp.experts.29.up_proj.weight": "model-00018-of-00101.safetensors", + "model.layers.18.mlp.experts.29.down_proj.weight": "model-00018-of-00101.safetensors", + "model.layers.18.mlp.experts.30.gate_proj.weight": "model-00018-of-00101.safetensors", + "model.layers.18.mlp.experts.30.up_proj.weight": "model-00018-of-00101.safetensors", + "model.layers.18.mlp.experts.30.down_proj.weight": "model-00018-of-00101.safetensors", + "model.layers.18.mlp.experts.31.gate_proj.weight": "model-00018-of-00101.safetensors", + "model.layers.18.mlp.experts.31.up_proj.weight": "model-00018-of-00101.safetensors", + "model.layers.18.mlp.experts.31.down_proj.weight": "model-00018-of-00101.safetensors", + "model.layers.18.mlp.experts.32.gate_proj.weight": "model-00018-of-00101.safetensors", + "model.layers.18.mlp.experts.32.up_proj.weight": "model-00018-of-00101.safetensors", + "model.layers.18.mlp.experts.32.down_proj.weight": "model-00018-of-00101.safetensors", + "model.layers.18.mlp.experts.33.gate_proj.weight": "model-00018-of-00101.safetensors", + "model.layers.18.mlp.experts.33.up_proj.weight": "model-00018-of-00101.safetensors", + "model.layers.18.mlp.experts.33.down_proj.weight": "model-00018-of-00101.safetensors", + "model.layers.18.mlp.experts.34.gate_proj.weight": "model-00018-of-00101.safetensors", + "model.layers.18.mlp.experts.34.up_proj.weight": "model-00018-of-00101.safetensors", + "model.layers.18.mlp.experts.34.down_proj.weight": "model-00018-of-00101.safetensors", + "model.layers.18.mlp.experts.35.gate_proj.weight": "model-00018-of-00101.safetensors", + "model.layers.18.mlp.experts.35.up_proj.weight": "model-00018-of-00101.safetensors", + "model.layers.18.mlp.experts.35.down_proj.weight": "model-00018-of-00101.safetensors", + "model.layers.18.mlp.experts.36.gate_proj.weight": "model-00018-of-00101.safetensors", + "model.layers.18.mlp.experts.36.up_proj.weight": "model-00018-of-00101.safetensors", + "model.layers.18.mlp.experts.36.down_proj.weight": "model-00018-of-00101.safetensors", + "model.layers.18.mlp.experts.37.gate_proj.weight": "model-00018-of-00101.safetensors", + "model.layers.18.mlp.experts.37.up_proj.weight": "model-00018-of-00101.safetensors", + "model.layers.18.mlp.experts.37.down_proj.weight": "model-00018-of-00101.safetensors", + "model.layers.18.mlp.experts.38.gate_proj.weight": "model-00018-of-00101.safetensors", + "model.layers.18.mlp.experts.38.up_proj.weight": "model-00018-of-00101.safetensors", + "model.layers.18.mlp.experts.38.down_proj.weight": "model-00018-of-00101.safetensors", + "model.layers.18.mlp.experts.39.gate_proj.weight": "model-00018-of-00101.safetensors", + "model.layers.18.mlp.experts.39.up_proj.weight": "model-00018-of-00101.safetensors", + "model.layers.18.mlp.experts.39.down_proj.weight": "model-00018-of-00101.safetensors", + "model.layers.18.mlp.experts.40.gate_proj.weight": "model-00018-of-00101.safetensors", + "model.layers.18.mlp.experts.40.up_proj.weight": "model-00018-of-00101.safetensors", + "model.layers.18.mlp.experts.40.down_proj.weight": "model-00018-of-00101.safetensors", + "model.layers.18.mlp.experts.41.gate_proj.weight": "model-00018-of-00101.safetensors", + "model.layers.18.mlp.experts.41.up_proj.weight": "model-00018-of-00101.safetensors", + "model.layers.18.mlp.experts.41.down_proj.weight": "model-00018-of-00101.safetensors", + "model.layers.18.mlp.experts.42.gate_proj.weight": "model-00018-of-00101.safetensors", + "model.layers.18.mlp.experts.42.up_proj.weight": "model-00018-of-00101.safetensors", + "model.layers.18.mlp.experts.42.down_proj.weight": "model-00018-of-00101.safetensors", + "model.layers.18.mlp.experts.43.gate_proj.weight": "model-00018-of-00101.safetensors", + "model.layers.18.mlp.experts.43.up_proj.weight": "model-00018-of-00101.safetensors", + "model.layers.18.mlp.experts.43.down_proj.weight": "model-00018-of-00101.safetensors", + "model.layers.18.mlp.experts.44.gate_proj.weight": "model-00018-of-00101.safetensors", + "model.layers.18.mlp.experts.44.up_proj.weight": "model-00018-of-00101.safetensors", + "model.layers.18.mlp.experts.44.down_proj.weight": "model-00018-of-00101.safetensors", + "model.layers.18.mlp.experts.45.gate_proj.weight": "model-00018-of-00101.safetensors", + "model.layers.18.mlp.experts.45.up_proj.weight": "model-00018-of-00101.safetensors", + "model.layers.18.mlp.experts.45.down_proj.weight": "model-00018-of-00101.safetensors", + "model.layers.18.mlp.experts.46.gate_proj.weight": "model-00018-of-00101.safetensors", + "model.layers.18.mlp.experts.46.up_proj.weight": "model-00018-of-00101.safetensors", + "model.layers.18.mlp.experts.46.down_proj.weight": "model-00018-of-00101.safetensors", + "model.layers.18.mlp.experts.47.gate_proj.weight": "model-00018-of-00101.safetensors", + "model.layers.18.mlp.experts.47.up_proj.weight": "model-00018-of-00101.safetensors", + "model.layers.18.mlp.experts.47.down_proj.weight": "model-00018-of-00101.safetensors", + "model.layers.18.mlp.experts.48.gate_proj.weight": "model-00018-of-00101.safetensors", + "model.layers.18.mlp.experts.48.up_proj.weight": "model-00018-of-00101.safetensors", + "model.layers.18.mlp.experts.48.down_proj.weight": "model-00018-of-00101.safetensors", + "model.layers.18.mlp.experts.49.gate_proj.weight": "model-00018-of-00101.safetensors", + "model.layers.18.mlp.experts.49.up_proj.weight": "model-00018-of-00101.safetensors", + "model.layers.18.mlp.experts.49.down_proj.weight": "model-00018-of-00101.safetensors", + "model.layers.18.mlp.experts.50.gate_proj.weight": "model-00018-of-00101.safetensors", + "model.layers.18.mlp.experts.50.up_proj.weight": "model-00018-of-00101.safetensors", + "model.layers.18.mlp.experts.50.down_proj.weight": "model-00018-of-00101.safetensors", + "model.layers.18.mlp.experts.51.gate_proj.weight": "model-00018-of-00101.safetensors", + "model.layers.18.mlp.experts.51.up_proj.weight": "model-00018-of-00101.safetensors", + "model.layers.18.mlp.experts.51.down_proj.weight": "model-00018-of-00101.safetensors", + "model.layers.18.mlp.experts.52.gate_proj.weight": "model-00018-of-00101.safetensors", + "model.layers.18.mlp.experts.52.up_proj.weight": "model-00018-of-00101.safetensors", + "model.layers.18.mlp.experts.52.down_proj.weight": "model-00018-of-00101.safetensors", + "model.layers.18.mlp.experts.53.gate_proj.weight": "model-00018-of-00101.safetensors", + "model.layers.18.mlp.experts.53.up_proj.weight": "model-00018-of-00101.safetensors", + "model.layers.18.mlp.experts.53.down_proj.weight": "model-00018-of-00101.safetensors", + "model.layers.18.mlp.experts.54.gate_proj.weight": "model-00018-of-00101.safetensors", + "model.layers.18.mlp.experts.54.up_proj.weight": "model-00018-of-00101.safetensors", + "model.layers.18.mlp.experts.54.down_proj.weight": "model-00018-of-00101.safetensors", + "model.layers.18.mlp.experts.55.gate_proj.weight": "model-00018-of-00101.safetensors", + "model.layers.18.mlp.experts.55.up_proj.weight": "model-00018-of-00101.safetensors", + "model.layers.18.mlp.experts.55.down_proj.weight": "model-00018-of-00101.safetensors", + "model.layers.18.mlp.experts.56.gate_proj.weight": "model-00018-of-00101.safetensors", + "model.layers.18.mlp.experts.56.up_proj.weight": "model-00018-of-00101.safetensors", + "model.layers.18.mlp.experts.56.down_proj.weight": "model-00018-of-00101.safetensors", + "model.layers.18.mlp.experts.57.gate_proj.weight": "model-00018-of-00101.safetensors", + "model.layers.18.mlp.experts.57.up_proj.weight": "model-00018-of-00101.safetensors", + "model.layers.18.mlp.experts.57.down_proj.weight": "model-00018-of-00101.safetensors", + "model.layers.18.mlp.experts.58.gate_proj.weight": "model-00018-of-00101.safetensors", + "model.layers.18.mlp.experts.58.up_proj.weight": "model-00018-of-00101.safetensors", + "model.layers.18.mlp.experts.58.down_proj.weight": "model-00018-of-00101.safetensors", + "model.layers.18.mlp.experts.59.gate_proj.weight": "model-00018-of-00101.safetensors", + "model.layers.18.mlp.experts.59.up_proj.weight": "model-00018-of-00101.safetensors", + "model.layers.18.mlp.experts.59.down_proj.weight": "model-00018-of-00101.safetensors", + "model.layers.18.mlp.experts.60.gate_proj.weight": "model-00018-of-00101.safetensors", + "model.layers.18.mlp.experts.60.up_proj.weight": "model-00018-of-00101.safetensors", + "model.layers.18.mlp.experts.60.down_proj.weight": "model-00018-of-00101.safetensors", + "model.layers.18.mlp.experts.61.gate_proj.weight": "model-00019-of-00101.safetensors", + "model.layers.18.mlp.experts.61.up_proj.weight": "model-00019-of-00101.safetensors", + "model.layers.18.mlp.experts.61.down_proj.weight": "model-00019-of-00101.safetensors", + "model.layers.18.mlp.experts.62.gate_proj.weight": "model-00019-of-00101.safetensors", + "model.layers.18.mlp.experts.62.up_proj.weight": "model-00019-of-00101.safetensors", + "model.layers.18.mlp.experts.62.down_proj.weight": "model-00019-of-00101.safetensors", + "model.layers.18.mlp.experts.63.gate_proj.weight": "model-00019-of-00101.safetensors", + "model.layers.18.mlp.experts.63.up_proj.weight": "model-00019-of-00101.safetensors", + "model.layers.18.mlp.experts.63.down_proj.weight": "model-00019-of-00101.safetensors", + "model.layers.18.mlp.experts.64.gate_proj.weight": "model-00019-of-00101.safetensors", + "model.layers.18.mlp.experts.64.up_proj.weight": "model-00019-of-00101.safetensors", + "model.layers.18.mlp.experts.64.down_proj.weight": "model-00019-of-00101.safetensors", + "model.layers.18.mlp.experts.65.gate_proj.weight": "model-00019-of-00101.safetensors", + "model.layers.18.mlp.experts.65.up_proj.weight": "model-00019-of-00101.safetensors", + "model.layers.18.mlp.experts.65.down_proj.weight": "model-00019-of-00101.safetensors", + "model.layers.18.mlp.experts.66.gate_proj.weight": "model-00019-of-00101.safetensors", + "model.layers.18.mlp.experts.66.up_proj.weight": "model-00019-of-00101.safetensors", + "model.layers.18.mlp.experts.66.down_proj.weight": "model-00019-of-00101.safetensors", + "model.layers.18.mlp.experts.67.gate_proj.weight": "model-00019-of-00101.safetensors", + "model.layers.18.mlp.experts.67.up_proj.weight": "model-00019-of-00101.safetensors", + "model.layers.18.mlp.experts.67.down_proj.weight": "model-00019-of-00101.safetensors", + "model.layers.18.mlp.experts.68.gate_proj.weight": "model-00019-of-00101.safetensors", + "model.layers.18.mlp.experts.68.up_proj.weight": "model-00019-of-00101.safetensors", + "model.layers.18.mlp.experts.68.down_proj.weight": "model-00019-of-00101.safetensors", + "model.layers.18.mlp.experts.69.gate_proj.weight": "model-00019-of-00101.safetensors", + "model.layers.18.mlp.experts.69.up_proj.weight": "model-00019-of-00101.safetensors", + "model.layers.18.mlp.experts.69.down_proj.weight": "model-00019-of-00101.safetensors", + "model.layers.18.mlp.experts.70.gate_proj.weight": "model-00019-of-00101.safetensors", + "model.layers.18.mlp.experts.70.up_proj.weight": "model-00019-of-00101.safetensors", + "model.layers.18.mlp.experts.70.down_proj.weight": "model-00019-of-00101.safetensors", + "model.layers.18.mlp.experts.71.gate_proj.weight": "model-00019-of-00101.safetensors", + "model.layers.18.mlp.experts.71.up_proj.weight": "model-00019-of-00101.safetensors", + "model.layers.18.mlp.experts.71.down_proj.weight": "model-00019-of-00101.safetensors", + "model.layers.18.mlp.experts.72.gate_proj.weight": "model-00019-of-00101.safetensors", + "model.layers.18.mlp.experts.72.up_proj.weight": "model-00019-of-00101.safetensors", + "model.layers.18.mlp.experts.72.down_proj.weight": "model-00019-of-00101.safetensors", + "model.layers.18.mlp.experts.73.gate_proj.weight": "model-00019-of-00101.safetensors", + "model.layers.18.mlp.experts.73.up_proj.weight": "model-00019-of-00101.safetensors", + "model.layers.18.mlp.experts.73.down_proj.weight": "model-00019-of-00101.safetensors", + "model.layers.18.mlp.experts.74.gate_proj.weight": "model-00019-of-00101.safetensors", + "model.layers.18.mlp.experts.74.up_proj.weight": "model-00019-of-00101.safetensors", + "model.layers.18.mlp.experts.74.down_proj.weight": "model-00019-of-00101.safetensors", + "model.layers.18.mlp.experts.75.gate_proj.weight": "model-00019-of-00101.safetensors", + "model.layers.18.mlp.experts.75.up_proj.weight": "model-00019-of-00101.safetensors", + "model.layers.18.mlp.experts.75.down_proj.weight": "model-00019-of-00101.safetensors", + "model.layers.18.mlp.experts.76.gate_proj.weight": "model-00019-of-00101.safetensors", + "model.layers.18.mlp.experts.76.up_proj.weight": "model-00019-of-00101.safetensors", + "model.layers.18.mlp.experts.76.down_proj.weight": "model-00019-of-00101.safetensors", + "model.layers.18.mlp.experts.77.gate_proj.weight": "model-00019-of-00101.safetensors", + "model.layers.18.mlp.experts.77.up_proj.weight": "model-00019-of-00101.safetensors", + "model.layers.18.mlp.experts.77.down_proj.weight": "model-00019-of-00101.safetensors", + "model.layers.18.mlp.experts.78.gate_proj.weight": "model-00019-of-00101.safetensors", + "model.layers.18.mlp.experts.78.up_proj.weight": "model-00019-of-00101.safetensors", + "model.layers.18.mlp.experts.78.down_proj.weight": "model-00019-of-00101.safetensors", + "model.layers.18.mlp.experts.79.gate_proj.weight": "model-00019-of-00101.safetensors", + "model.layers.18.mlp.experts.79.up_proj.weight": "model-00019-of-00101.safetensors", + "model.layers.18.mlp.experts.79.down_proj.weight": "model-00019-of-00101.safetensors", + "model.layers.18.mlp.experts.80.gate_proj.weight": "model-00019-of-00101.safetensors", + "model.layers.18.mlp.experts.80.up_proj.weight": "model-00019-of-00101.safetensors", + "model.layers.18.mlp.experts.80.down_proj.weight": "model-00019-of-00101.safetensors", + "model.layers.18.mlp.experts.81.gate_proj.weight": "model-00019-of-00101.safetensors", + "model.layers.18.mlp.experts.81.up_proj.weight": "model-00019-of-00101.safetensors", + "model.layers.18.mlp.experts.81.down_proj.weight": "model-00019-of-00101.safetensors", + "model.layers.18.mlp.experts.82.gate_proj.weight": "model-00019-of-00101.safetensors", + "model.layers.18.mlp.experts.82.up_proj.weight": "model-00019-of-00101.safetensors", + "model.layers.18.mlp.experts.82.down_proj.weight": "model-00019-of-00101.safetensors", + "model.layers.18.mlp.experts.83.gate_proj.weight": "model-00019-of-00101.safetensors", + "model.layers.18.mlp.experts.83.up_proj.weight": "model-00019-of-00101.safetensors", + "model.layers.18.mlp.experts.83.down_proj.weight": "model-00019-of-00101.safetensors", + "model.layers.18.mlp.experts.84.gate_proj.weight": "model-00019-of-00101.safetensors", + "model.layers.18.mlp.experts.84.up_proj.weight": "model-00019-of-00101.safetensors", + "model.layers.18.mlp.experts.84.down_proj.weight": "model-00019-of-00101.safetensors", + "model.layers.18.mlp.experts.85.gate_proj.weight": "model-00019-of-00101.safetensors", + "model.layers.18.mlp.experts.85.up_proj.weight": "model-00019-of-00101.safetensors", + "model.layers.18.mlp.experts.85.down_proj.weight": "model-00019-of-00101.safetensors", + "model.layers.18.mlp.experts.86.gate_proj.weight": "model-00019-of-00101.safetensors", + "model.layers.18.mlp.experts.86.up_proj.weight": "model-00019-of-00101.safetensors", + "model.layers.18.mlp.experts.86.down_proj.weight": "model-00019-of-00101.safetensors", + "model.layers.18.mlp.experts.87.gate_proj.weight": "model-00019-of-00101.safetensors", + "model.layers.18.mlp.experts.87.up_proj.weight": "model-00019-of-00101.safetensors", + "model.layers.18.mlp.experts.87.down_proj.weight": "model-00019-of-00101.safetensors", + "model.layers.18.mlp.experts.88.gate_proj.weight": "model-00019-of-00101.safetensors", + "model.layers.18.mlp.experts.88.up_proj.weight": "model-00019-of-00101.safetensors", + "model.layers.18.mlp.experts.88.down_proj.weight": "model-00019-of-00101.safetensors", + "model.layers.18.mlp.experts.89.gate_proj.weight": "model-00019-of-00101.safetensors", + "model.layers.18.mlp.experts.89.up_proj.weight": "model-00019-of-00101.safetensors", + "model.layers.18.mlp.experts.89.down_proj.weight": "model-00019-of-00101.safetensors", + "model.layers.18.mlp.experts.90.gate_proj.weight": "model-00019-of-00101.safetensors", + "model.layers.18.mlp.experts.90.up_proj.weight": "model-00019-of-00101.safetensors", + "model.layers.18.mlp.experts.90.down_proj.weight": "model-00019-of-00101.safetensors", + "model.layers.18.mlp.experts.91.gate_proj.weight": "model-00019-of-00101.safetensors", + "model.layers.18.mlp.experts.91.up_proj.weight": "model-00019-of-00101.safetensors", + "model.layers.18.mlp.experts.91.down_proj.weight": "model-00019-of-00101.safetensors", + "model.layers.18.mlp.experts.92.gate_proj.weight": "model-00019-of-00101.safetensors", + "model.layers.18.mlp.experts.92.up_proj.weight": "model-00019-of-00101.safetensors", + "model.layers.18.mlp.experts.92.down_proj.weight": "model-00019-of-00101.safetensors", + "model.layers.18.mlp.experts.93.gate_proj.weight": "model-00019-of-00101.safetensors", + "model.layers.18.mlp.experts.93.up_proj.weight": "model-00019-of-00101.safetensors", + "model.layers.18.mlp.experts.93.down_proj.weight": "model-00019-of-00101.safetensors", + "model.layers.18.mlp.experts.94.gate_proj.weight": "model-00019-of-00101.safetensors", + "model.layers.18.mlp.experts.94.up_proj.weight": "model-00019-of-00101.safetensors", + "model.layers.18.mlp.experts.94.down_proj.weight": "model-00019-of-00101.safetensors", + "model.layers.18.mlp.experts.95.gate_proj.weight": "model-00019-of-00101.safetensors", + "model.layers.18.mlp.experts.95.up_proj.weight": "model-00019-of-00101.safetensors", + "model.layers.18.mlp.experts.95.down_proj.weight": "model-00019-of-00101.safetensors", + "model.layers.18.mlp.experts.96.gate_proj.weight": "model-00019-of-00101.safetensors", + "model.layers.18.mlp.experts.96.up_proj.weight": "model-00019-of-00101.safetensors", + "model.layers.18.mlp.experts.96.down_proj.weight": "model-00019-of-00101.safetensors", + "model.layers.18.mlp.experts.97.gate_proj.weight": "model-00019-of-00101.safetensors", + "model.layers.18.mlp.experts.97.up_proj.weight": "model-00019-of-00101.safetensors", + "model.layers.18.mlp.experts.97.down_proj.weight": "model-00019-of-00101.safetensors", + "model.layers.18.mlp.experts.98.gate_proj.weight": "model-00019-of-00101.safetensors", + "model.layers.18.mlp.experts.98.up_proj.weight": "model-00019-of-00101.safetensors", + "model.layers.18.mlp.experts.98.down_proj.weight": "model-00019-of-00101.safetensors", + "model.layers.18.mlp.experts.99.gate_proj.weight": "model-00019-of-00101.safetensors", + "model.layers.18.mlp.experts.99.up_proj.weight": "model-00019-of-00101.safetensors", + "model.layers.18.mlp.experts.99.down_proj.weight": "model-00019-of-00101.safetensors", + "model.layers.18.mlp.experts.100.gate_proj.weight": "model-00019-of-00101.safetensors", + "model.layers.18.mlp.experts.100.up_proj.weight": "model-00019-of-00101.safetensors", + "model.layers.18.mlp.experts.100.down_proj.weight": "model-00019-of-00101.safetensors", + "model.layers.18.mlp.experts.101.gate_proj.weight": "model-00019-of-00101.safetensors", + "model.layers.18.mlp.experts.101.up_proj.weight": "model-00019-of-00101.safetensors", + "model.layers.18.mlp.experts.101.down_proj.weight": "model-00019-of-00101.safetensors", + "model.layers.18.mlp.experts.102.gate_proj.weight": "model-00019-of-00101.safetensors", + "model.layers.18.mlp.experts.102.up_proj.weight": "model-00019-of-00101.safetensors", + "model.layers.18.mlp.experts.102.down_proj.weight": "model-00019-of-00101.safetensors", + "model.layers.18.mlp.experts.103.gate_proj.weight": "model-00019-of-00101.safetensors", + "model.layers.18.mlp.experts.103.up_proj.weight": "model-00019-of-00101.safetensors", + "model.layers.18.mlp.experts.103.down_proj.weight": "model-00019-of-00101.safetensors", + "model.layers.18.mlp.experts.104.gate_proj.weight": "model-00019-of-00101.safetensors", + "model.layers.18.mlp.experts.104.up_proj.weight": "model-00019-of-00101.safetensors", + "model.layers.18.mlp.experts.104.down_proj.weight": "model-00019-of-00101.safetensors", + "model.layers.18.mlp.experts.105.gate_proj.weight": "model-00019-of-00101.safetensors", + "model.layers.18.mlp.experts.105.up_proj.weight": "model-00019-of-00101.safetensors", + "model.layers.18.mlp.experts.105.down_proj.weight": "model-00019-of-00101.safetensors", + "model.layers.18.mlp.experts.106.gate_proj.weight": "model-00019-of-00101.safetensors", + "model.layers.18.mlp.experts.106.up_proj.weight": "model-00019-of-00101.safetensors", + "model.layers.18.mlp.experts.106.down_proj.weight": "model-00019-of-00101.safetensors", + "model.layers.18.mlp.experts.107.gate_proj.weight": "model-00019-of-00101.safetensors", + "model.layers.18.mlp.experts.107.up_proj.weight": "model-00019-of-00101.safetensors", + "model.layers.18.mlp.experts.107.down_proj.weight": "model-00019-of-00101.safetensors", + "model.layers.18.mlp.experts.108.gate_proj.weight": "model-00019-of-00101.safetensors", + "model.layers.18.mlp.experts.108.up_proj.weight": "model-00019-of-00101.safetensors", + "model.layers.18.mlp.experts.108.down_proj.weight": "model-00019-of-00101.safetensors", + "model.layers.18.mlp.experts.109.gate_proj.weight": "model-00019-of-00101.safetensors", + "model.layers.18.mlp.experts.109.up_proj.weight": "model-00019-of-00101.safetensors", + "model.layers.18.mlp.experts.109.down_proj.weight": "model-00019-of-00101.safetensors", + "model.layers.18.mlp.experts.110.gate_proj.weight": "model-00019-of-00101.safetensors", + "model.layers.18.mlp.experts.110.up_proj.weight": "model-00019-of-00101.safetensors", + "model.layers.18.mlp.experts.110.down_proj.weight": "model-00019-of-00101.safetensors", + "model.layers.18.mlp.experts.111.gate_proj.weight": "model-00019-of-00101.safetensors", + "model.layers.18.mlp.experts.111.up_proj.weight": "model-00019-of-00101.safetensors", + "model.layers.18.mlp.experts.111.down_proj.weight": "model-00019-of-00101.safetensors", + "model.layers.18.mlp.experts.112.gate_proj.weight": "model-00019-of-00101.safetensors", + "model.layers.18.mlp.experts.112.up_proj.weight": "model-00019-of-00101.safetensors", + "model.layers.18.mlp.experts.112.down_proj.weight": "model-00019-of-00101.safetensors", + "model.layers.18.mlp.experts.113.gate_proj.weight": "model-00019-of-00101.safetensors", + "model.layers.18.mlp.experts.113.up_proj.weight": "model-00019-of-00101.safetensors", + "model.layers.18.mlp.experts.113.down_proj.weight": "model-00019-of-00101.safetensors", + "model.layers.18.mlp.experts.114.gate_proj.weight": "model-00019-of-00101.safetensors", + "model.layers.18.mlp.experts.114.up_proj.weight": "model-00019-of-00101.safetensors", + "model.layers.18.mlp.experts.114.down_proj.weight": "model-00019-of-00101.safetensors", + "model.layers.18.mlp.experts.115.gate_proj.weight": "model-00019-of-00101.safetensors", + "model.layers.18.mlp.experts.115.up_proj.weight": "model-00019-of-00101.safetensors", + "model.layers.18.mlp.experts.115.down_proj.weight": "model-00019-of-00101.safetensors", + "model.layers.18.mlp.experts.116.gate_proj.weight": "model-00019-of-00101.safetensors", + "model.layers.18.mlp.experts.116.up_proj.weight": "model-00019-of-00101.safetensors", + "model.layers.18.mlp.experts.116.down_proj.weight": "model-00019-of-00101.safetensors", + "model.layers.18.mlp.experts.117.gate_proj.weight": "model-00019-of-00101.safetensors", + "model.layers.18.mlp.experts.117.up_proj.weight": "model-00019-of-00101.safetensors", + "model.layers.18.mlp.experts.117.down_proj.weight": "model-00019-of-00101.safetensors", + "model.layers.18.mlp.experts.118.gate_proj.weight": "model-00019-of-00101.safetensors", + "model.layers.18.mlp.experts.118.up_proj.weight": "model-00019-of-00101.safetensors", + "model.layers.18.mlp.experts.118.down_proj.weight": "model-00019-of-00101.safetensors", + "model.layers.18.mlp.experts.119.gate_proj.weight": "model-00019-of-00101.safetensors", + "model.layers.18.mlp.experts.119.up_proj.weight": "model-00019-of-00101.safetensors", + "model.layers.18.mlp.experts.119.down_proj.weight": "model-00019-of-00101.safetensors", + "model.layers.18.mlp.gate.weight": "model-00019-of-00101.safetensors", + "model.layers.18.mlp.gate.e_score_correction_bias": "model-00019-of-00101.safetensors", + "model.layers.18.mlp.shared_experts.gate_proj.weight": "model-00019-of-00101.safetensors", + "model.layers.18.mlp.shared_experts.up_proj.weight": "model-00019-of-00101.safetensors", + "model.layers.18.mlp.shared_experts.down_proj.weight": "model-00019-of-00101.safetensors", + "model.layers.18.input_layernorm.weight": "model-00019-of-00101.safetensors", + "model.layers.18.post_attention_layernorm.weight": "model-00019-of-00101.safetensors", + "model.layers.19.self_attn.q_proj.weight": "model-00019-of-00101.safetensors", + "model.layers.19.self_attn.q_proj.bias": "model-00019-of-00101.safetensors", + "model.layers.19.self_attn.k_proj.weight": "model-00019-of-00101.safetensors", + "model.layers.19.self_attn.k_proj.bias": "model-00019-of-00101.safetensors", + "model.layers.19.self_attn.v_proj.weight": "model-00019-of-00101.safetensors", + "model.layers.19.self_attn.v_proj.bias": "model-00019-of-00101.safetensors", + "model.layers.19.self_attn.o_proj.weight": "model-00019-of-00101.safetensors", + "model.layers.19.self_attn.q_norm.weight": "model-00019-of-00101.safetensors", + "model.layers.19.self_attn.k_norm.weight": "model-00019-of-00101.safetensors", + "model.layers.19.mlp.experts.0.gate_proj.weight": "model-00019-of-00101.safetensors", + "model.layers.19.mlp.experts.0.up_proj.weight": "model-00019-of-00101.safetensors", + "model.layers.19.mlp.experts.0.down_proj.weight": "model-00019-of-00101.safetensors", + "model.layers.19.mlp.experts.1.gate_proj.weight": "model-00019-of-00101.safetensors", + "model.layers.19.mlp.experts.1.up_proj.weight": "model-00019-of-00101.safetensors", + "model.layers.19.mlp.experts.1.down_proj.weight": "model-00019-of-00101.safetensors", + "model.layers.19.mlp.experts.2.gate_proj.weight": "model-00019-of-00101.safetensors", + "model.layers.19.mlp.experts.2.up_proj.weight": "model-00019-of-00101.safetensors", + "model.layers.19.mlp.experts.2.down_proj.weight": "model-00019-of-00101.safetensors", + "model.layers.19.mlp.experts.3.gate_proj.weight": "model-00019-of-00101.safetensors", + "model.layers.19.mlp.experts.3.up_proj.weight": "model-00019-of-00101.safetensors", + "model.layers.19.mlp.experts.3.down_proj.weight": "model-00019-of-00101.safetensors", + "model.layers.19.mlp.experts.4.gate_proj.weight": "model-00019-of-00101.safetensors", + "model.layers.19.mlp.experts.4.up_proj.weight": "model-00019-of-00101.safetensors", + "model.layers.19.mlp.experts.4.down_proj.weight": "model-00019-of-00101.safetensors", + "model.layers.19.mlp.experts.5.gate_proj.weight": "model-00019-of-00101.safetensors", + "model.layers.19.mlp.experts.5.up_proj.weight": "model-00019-of-00101.safetensors", + "model.layers.19.mlp.experts.5.down_proj.weight": "model-00019-of-00101.safetensors", + "model.layers.19.mlp.experts.6.gate_proj.weight": "model-00019-of-00101.safetensors", + "model.layers.19.mlp.experts.6.up_proj.weight": "model-00019-of-00101.safetensors", + "model.layers.19.mlp.experts.6.down_proj.weight": "model-00019-of-00101.safetensors", + "model.layers.19.mlp.experts.7.gate_proj.weight": "model-00019-of-00101.safetensors", + "model.layers.19.mlp.experts.7.up_proj.weight": "model-00019-of-00101.safetensors", + "model.layers.19.mlp.experts.7.down_proj.weight": "model-00019-of-00101.safetensors", + "model.layers.19.mlp.experts.8.gate_proj.weight": "model-00019-of-00101.safetensors", + "model.layers.19.mlp.experts.8.up_proj.weight": "model-00019-of-00101.safetensors", + "model.layers.19.mlp.experts.8.down_proj.weight": "model-00019-of-00101.safetensors", + "model.layers.19.mlp.experts.9.gate_proj.weight": "model-00019-of-00101.safetensors", + "model.layers.19.mlp.experts.9.up_proj.weight": "model-00019-of-00101.safetensors", + "model.layers.19.mlp.experts.9.down_proj.weight": "model-00019-of-00101.safetensors", + "model.layers.19.mlp.experts.10.gate_proj.weight": "model-00019-of-00101.safetensors", + "model.layers.19.mlp.experts.10.up_proj.weight": "model-00019-of-00101.safetensors", + "model.layers.19.mlp.experts.10.down_proj.weight": "model-00019-of-00101.safetensors", + "model.layers.19.mlp.experts.11.gate_proj.weight": "model-00019-of-00101.safetensors", + "model.layers.19.mlp.experts.11.up_proj.weight": "model-00019-of-00101.safetensors", + "model.layers.19.mlp.experts.11.down_proj.weight": "model-00019-of-00101.safetensors", + "model.layers.19.mlp.experts.12.gate_proj.weight": "model-00019-of-00101.safetensors", + "model.layers.19.mlp.experts.12.up_proj.weight": "model-00019-of-00101.safetensors", + "model.layers.19.mlp.experts.12.down_proj.weight": "model-00019-of-00101.safetensors", + "model.layers.19.mlp.experts.13.gate_proj.weight": "model-00019-of-00101.safetensors", + "model.layers.19.mlp.experts.13.up_proj.weight": "model-00019-of-00101.safetensors", + "model.layers.19.mlp.experts.13.down_proj.weight": "model-00019-of-00101.safetensors", + "model.layers.19.mlp.experts.14.gate_proj.weight": "model-00019-of-00101.safetensors", + "model.layers.19.mlp.experts.14.up_proj.weight": "model-00019-of-00101.safetensors", + "model.layers.19.mlp.experts.14.down_proj.weight": "model-00019-of-00101.safetensors", + "model.layers.19.mlp.experts.15.gate_proj.weight": "model-00019-of-00101.safetensors", + "model.layers.19.mlp.experts.15.up_proj.weight": "model-00019-of-00101.safetensors", + "model.layers.19.mlp.experts.15.down_proj.weight": "model-00019-of-00101.safetensors", + "model.layers.19.mlp.experts.16.gate_proj.weight": "model-00019-of-00101.safetensors", + "model.layers.19.mlp.experts.16.up_proj.weight": "model-00019-of-00101.safetensors", + "model.layers.19.mlp.experts.16.down_proj.weight": "model-00019-of-00101.safetensors", + "model.layers.19.mlp.experts.17.gate_proj.weight": "model-00019-of-00101.safetensors", + "model.layers.19.mlp.experts.17.up_proj.weight": "model-00019-of-00101.safetensors", + "model.layers.19.mlp.experts.17.down_proj.weight": "model-00019-of-00101.safetensors", + "model.layers.19.mlp.experts.18.gate_proj.weight": "model-00019-of-00101.safetensors", + "model.layers.19.mlp.experts.18.up_proj.weight": "model-00019-of-00101.safetensors", + "model.layers.19.mlp.experts.18.down_proj.weight": "model-00019-of-00101.safetensors", + "model.layers.19.mlp.experts.19.gate_proj.weight": "model-00019-of-00101.safetensors", + "model.layers.19.mlp.experts.19.up_proj.weight": "model-00019-of-00101.safetensors", + "model.layers.19.mlp.experts.19.down_proj.weight": "model-00019-of-00101.safetensors", + "model.layers.19.mlp.experts.20.gate_proj.weight": "model-00019-of-00101.safetensors", + "model.layers.19.mlp.experts.20.up_proj.weight": "model-00019-of-00101.safetensors", + "model.layers.19.mlp.experts.20.down_proj.weight": "model-00019-of-00101.safetensors", + "model.layers.19.mlp.experts.21.gate_proj.weight": "model-00019-of-00101.safetensors", + "model.layers.19.mlp.experts.21.up_proj.weight": "model-00019-of-00101.safetensors", + "model.layers.19.mlp.experts.21.down_proj.weight": "model-00019-of-00101.safetensors", + "model.layers.19.mlp.experts.22.gate_proj.weight": "model-00019-of-00101.safetensors", + "model.layers.19.mlp.experts.22.up_proj.weight": "model-00019-of-00101.safetensors", + "model.layers.19.mlp.experts.22.down_proj.weight": "model-00019-of-00101.safetensors", + "model.layers.19.mlp.experts.23.gate_proj.weight": "model-00019-of-00101.safetensors", + "model.layers.19.mlp.experts.23.up_proj.weight": "model-00019-of-00101.safetensors", + "model.layers.19.mlp.experts.23.down_proj.weight": "model-00019-of-00101.safetensors", + "model.layers.19.mlp.experts.24.gate_proj.weight": "model-00019-of-00101.safetensors", + "model.layers.19.mlp.experts.24.up_proj.weight": "model-00019-of-00101.safetensors", + "model.layers.19.mlp.experts.24.down_proj.weight": "model-00019-of-00101.safetensors", + "model.layers.19.mlp.experts.25.gate_proj.weight": "model-00019-of-00101.safetensors", + "model.layers.19.mlp.experts.25.up_proj.weight": "model-00019-of-00101.safetensors", + "model.layers.19.mlp.experts.25.down_proj.weight": "model-00019-of-00101.safetensors", + "model.layers.19.mlp.experts.26.gate_proj.weight": "model-00019-of-00101.safetensors", + "model.layers.19.mlp.experts.26.up_proj.weight": "model-00019-of-00101.safetensors", + "model.layers.19.mlp.experts.26.down_proj.weight": "model-00019-of-00101.safetensors", + "model.layers.19.mlp.experts.27.gate_proj.weight": "model-00019-of-00101.safetensors", + "model.layers.19.mlp.experts.27.up_proj.weight": "model-00019-of-00101.safetensors", + "model.layers.19.mlp.experts.27.down_proj.weight": "model-00019-of-00101.safetensors", + "model.layers.19.mlp.experts.28.gate_proj.weight": "model-00019-of-00101.safetensors", + "model.layers.19.mlp.experts.28.up_proj.weight": "model-00019-of-00101.safetensors", + "model.layers.19.mlp.experts.28.down_proj.weight": "model-00019-of-00101.safetensors", + "model.layers.19.mlp.experts.29.gate_proj.weight": "model-00019-of-00101.safetensors", + "model.layers.19.mlp.experts.29.up_proj.weight": "model-00019-of-00101.safetensors", + "model.layers.19.mlp.experts.29.down_proj.weight": "model-00019-of-00101.safetensors", + "model.layers.19.mlp.experts.30.gate_proj.weight": "model-00019-of-00101.safetensors", + "model.layers.19.mlp.experts.30.up_proj.weight": "model-00019-of-00101.safetensors", + "model.layers.19.mlp.experts.30.down_proj.weight": "model-00019-of-00101.safetensors", + "model.layers.19.mlp.experts.31.gate_proj.weight": "model-00019-of-00101.safetensors", + "model.layers.19.mlp.experts.31.up_proj.weight": "model-00019-of-00101.safetensors", + "model.layers.19.mlp.experts.31.down_proj.weight": "model-00019-of-00101.safetensors", + "model.layers.19.mlp.experts.32.gate_proj.weight": "model-00019-of-00101.safetensors", + "model.layers.19.mlp.experts.32.up_proj.weight": "model-00019-of-00101.safetensors", + "model.layers.19.mlp.experts.32.down_proj.weight": "model-00019-of-00101.safetensors", + "model.layers.19.mlp.experts.33.gate_proj.weight": "model-00019-of-00101.safetensors", + "model.layers.19.mlp.experts.33.up_proj.weight": "model-00019-of-00101.safetensors", + "model.layers.19.mlp.experts.33.down_proj.weight": "model-00019-of-00101.safetensors", + "model.layers.19.mlp.experts.34.gate_proj.weight": "model-00019-of-00101.safetensors", + "model.layers.19.mlp.experts.34.up_proj.weight": "model-00019-of-00101.safetensors", + "model.layers.19.mlp.experts.34.down_proj.weight": "model-00019-of-00101.safetensors", + "model.layers.19.mlp.experts.35.gate_proj.weight": "model-00019-of-00101.safetensors", + "model.layers.19.mlp.experts.35.up_proj.weight": "model-00019-of-00101.safetensors", + "model.layers.19.mlp.experts.35.down_proj.weight": "model-00019-of-00101.safetensors", + "model.layers.19.mlp.experts.36.gate_proj.weight": "model-00019-of-00101.safetensors", + "model.layers.19.mlp.experts.36.up_proj.weight": "model-00019-of-00101.safetensors", + "model.layers.19.mlp.experts.36.down_proj.weight": "model-00019-of-00101.safetensors", + "model.layers.19.mlp.experts.37.gate_proj.weight": "model-00019-of-00101.safetensors", + "model.layers.19.mlp.experts.37.up_proj.weight": "model-00019-of-00101.safetensors", + "model.layers.19.mlp.experts.37.down_proj.weight": "model-00019-of-00101.safetensors", + "model.layers.19.mlp.experts.38.gate_proj.weight": "model-00019-of-00101.safetensors", + "model.layers.19.mlp.experts.38.up_proj.weight": "model-00019-of-00101.safetensors", + "model.layers.19.mlp.experts.38.down_proj.weight": "model-00019-of-00101.safetensors", + "model.layers.19.mlp.experts.39.gate_proj.weight": "model-00019-of-00101.safetensors", + "model.layers.19.mlp.experts.39.up_proj.weight": "model-00019-of-00101.safetensors", + "model.layers.19.mlp.experts.39.down_proj.weight": "model-00019-of-00101.safetensors", + "model.layers.19.mlp.experts.40.gate_proj.weight": "model-00019-of-00101.safetensors", + "model.layers.19.mlp.experts.40.up_proj.weight": "model-00019-of-00101.safetensors", + "model.layers.19.mlp.experts.40.down_proj.weight": "model-00019-of-00101.safetensors", + "model.layers.19.mlp.experts.41.gate_proj.weight": "model-00019-of-00101.safetensors", + "model.layers.19.mlp.experts.41.up_proj.weight": "model-00019-of-00101.safetensors", + "model.layers.19.mlp.experts.41.down_proj.weight": "model-00019-of-00101.safetensors", + "model.layers.19.mlp.experts.42.gate_proj.weight": "model-00019-of-00101.safetensors", + "model.layers.19.mlp.experts.42.up_proj.weight": "model-00019-of-00101.safetensors", + "model.layers.19.mlp.experts.42.down_proj.weight": "model-00019-of-00101.safetensors", + "model.layers.19.mlp.experts.43.gate_proj.weight": "model-00019-of-00101.safetensors", + "model.layers.19.mlp.experts.43.up_proj.weight": "model-00019-of-00101.safetensors", + "model.layers.19.mlp.experts.43.down_proj.weight": "model-00019-of-00101.safetensors", + "model.layers.19.mlp.experts.44.gate_proj.weight": "model-00019-of-00101.safetensors", + "model.layers.19.mlp.experts.44.up_proj.weight": "model-00019-of-00101.safetensors", + "model.layers.19.mlp.experts.44.down_proj.weight": "model-00019-of-00101.safetensors", + "model.layers.19.mlp.experts.45.gate_proj.weight": "model-00019-of-00101.safetensors", + "model.layers.19.mlp.experts.45.up_proj.weight": "model-00019-of-00101.safetensors", + "model.layers.19.mlp.experts.45.down_proj.weight": "model-00019-of-00101.safetensors", + "model.layers.19.mlp.experts.46.gate_proj.weight": "model-00019-of-00101.safetensors", + "model.layers.19.mlp.experts.46.up_proj.weight": "model-00019-of-00101.safetensors", + "model.layers.19.mlp.experts.46.down_proj.weight": "model-00019-of-00101.safetensors", + "model.layers.19.mlp.experts.47.gate_proj.weight": "model-00019-of-00101.safetensors", + "model.layers.19.mlp.experts.47.up_proj.weight": "model-00019-of-00101.safetensors", + "model.layers.19.mlp.experts.47.down_proj.weight": "model-00020-of-00101.safetensors", + "model.layers.19.mlp.experts.48.gate_proj.weight": "model-00020-of-00101.safetensors", + "model.layers.19.mlp.experts.48.up_proj.weight": "model-00020-of-00101.safetensors", + "model.layers.19.mlp.experts.48.down_proj.weight": "model-00020-of-00101.safetensors", + "model.layers.19.mlp.experts.49.gate_proj.weight": "model-00020-of-00101.safetensors", + "model.layers.19.mlp.experts.49.up_proj.weight": "model-00020-of-00101.safetensors", + "model.layers.19.mlp.experts.49.down_proj.weight": "model-00020-of-00101.safetensors", + "model.layers.19.mlp.experts.50.gate_proj.weight": "model-00020-of-00101.safetensors", + "model.layers.19.mlp.experts.50.up_proj.weight": "model-00020-of-00101.safetensors", + "model.layers.19.mlp.experts.50.down_proj.weight": "model-00020-of-00101.safetensors", + "model.layers.19.mlp.experts.51.gate_proj.weight": "model-00020-of-00101.safetensors", + "model.layers.19.mlp.experts.51.up_proj.weight": "model-00020-of-00101.safetensors", + "model.layers.19.mlp.experts.51.down_proj.weight": "model-00020-of-00101.safetensors", + "model.layers.19.mlp.experts.52.gate_proj.weight": "model-00020-of-00101.safetensors", + "model.layers.19.mlp.experts.52.up_proj.weight": "model-00020-of-00101.safetensors", + "model.layers.19.mlp.experts.52.down_proj.weight": "model-00020-of-00101.safetensors", + "model.layers.19.mlp.experts.53.gate_proj.weight": "model-00020-of-00101.safetensors", + "model.layers.19.mlp.experts.53.up_proj.weight": "model-00020-of-00101.safetensors", + "model.layers.19.mlp.experts.53.down_proj.weight": "model-00020-of-00101.safetensors", + "model.layers.19.mlp.experts.54.gate_proj.weight": "model-00020-of-00101.safetensors", + "model.layers.19.mlp.experts.54.up_proj.weight": "model-00020-of-00101.safetensors", + "model.layers.19.mlp.experts.54.down_proj.weight": "model-00020-of-00101.safetensors", + "model.layers.19.mlp.experts.55.gate_proj.weight": "model-00020-of-00101.safetensors", + "model.layers.19.mlp.experts.55.up_proj.weight": "model-00020-of-00101.safetensors", + "model.layers.19.mlp.experts.55.down_proj.weight": "model-00020-of-00101.safetensors", + "model.layers.19.mlp.experts.56.gate_proj.weight": "model-00020-of-00101.safetensors", + "model.layers.19.mlp.experts.56.up_proj.weight": "model-00020-of-00101.safetensors", + "model.layers.19.mlp.experts.56.down_proj.weight": "model-00020-of-00101.safetensors", + "model.layers.19.mlp.experts.57.gate_proj.weight": "model-00020-of-00101.safetensors", + "model.layers.19.mlp.experts.57.up_proj.weight": "model-00020-of-00101.safetensors", + "model.layers.19.mlp.experts.57.down_proj.weight": "model-00020-of-00101.safetensors", + "model.layers.19.mlp.experts.58.gate_proj.weight": "model-00020-of-00101.safetensors", + "model.layers.19.mlp.experts.58.up_proj.weight": "model-00020-of-00101.safetensors", + "model.layers.19.mlp.experts.58.down_proj.weight": "model-00020-of-00101.safetensors", + "model.layers.19.mlp.experts.59.gate_proj.weight": "model-00020-of-00101.safetensors", + "model.layers.19.mlp.experts.59.up_proj.weight": "model-00020-of-00101.safetensors", + "model.layers.19.mlp.experts.59.down_proj.weight": "model-00020-of-00101.safetensors", + "model.layers.19.mlp.experts.60.gate_proj.weight": "model-00020-of-00101.safetensors", + "model.layers.19.mlp.experts.60.up_proj.weight": "model-00020-of-00101.safetensors", + "model.layers.19.mlp.experts.60.down_proj.weight": "model-00020-of-00101.safetensors", + "model.layers.19.mlp.experts.61.gate_proj.weight": "model-00020-of-00101.safetensors", + "model.layers.19.mlp.experts.61.up_proj.weight": "model-00020-of-00101.safetensors", + "model.layers.19.mlp.experts.61.down_proj.weight": "model-00020-of-00101.safetensors", + "model.layers.19.mlp.experts.62.gate_proj.weight": "model-00020-of-00101.safetensors", + "model.layers.19.mlp.experts.62.up_proj.weight": "model-00020-of-00101.safetensors", + "model.layers.19.mlp.experts.62.down_proj.weight": "model-00020-of-00101.safetensors", + "model.layers.19.mlp.experts.63.gate_proj.weight": "model-00020-of-00101.safetensors", + "model.layers.19.mlp.experts.63.up_proj.weight": "model-00020-of-00101.safetensors", + "model.layers.19.mlp.experts.63.down_proj.weight": "model-00020-of-00101.safetensors", + "model.layers.19.mlp.experts.64.gate_proj.weight": "model-00020-of-00101.safetensors", + "model.layers.19.mlp.experts.64.up_proj.weight": "model-00020-of-00101.safetensors", + "model.layers.19.mlp.experts.64.down_proj.weight": "model-00020-of-00101.safetensors", + "model.layers.19.mlp.experts.65.gate_proj.weight": "model-00020-of-00101.safetensors", + "model.layers.19.mlp.experts.65.up_proj.weight": "model-00020-of-00101.safetensors", + "model.layers.19.mlp.experts.65.down_proj.weight": "model-00020-of-00101.safetensors", + "model.layers.19.mlp.experts.66.gate_proj.weight": "model-00020-of-00101.safetensors", + "model.layers.19.mlp.experts.66.up_proj.weight": "model-00020-of-00101.safetensors", + "model.layers.19.mlp.experts.66.down_proj.weight": "model-00020-of-00101.safetensors", + "model.layers.19.mlp.experts.67.gate_proj.weight": "model-00020-of-00101.safetensors", + "model.layers.19.mlp.experts.67.up_proj.weight": "model-00020-of-00101.safetensors", + "model.layers.19.mlp.experts.67.down_proj.weight": "model-00020-of-00101.safetensors", + "model.layers.19.mlp.experts.68.gate_proj.weight": "model-00020-of-00101.safetensors", + "model.layers.19.mlp.experts.68.up_proj.weight": "model-00020-of-00101.safetensors", + "model.layers.19.mlp.experts.68.down_proj.weight": "model-00020-of-00101.safetensors", + "model.layers.19.mlp.experts.69.gate_proj.weight": "model-00020-of-00101.safetensors", + "model.layers.19.mlp.experts.69.up_proj.weight": "model-00020-of-00101.safetensors", + "model.layers.19.mlp.experts.69.down_proj.weight": "model-00020-of-00101.safetensors", + "model.layers.19.mlp.experts.70.gate_proj.weight": "model-00020-of-00101.safetensors", + "model.layers.19.mlp.experts.70.up_proj.weight": "model-00020-of-00101.safetensors", + "model.layers.19.mlp.experts.70.down_proj.weight": "model-00020-of-00101.safetensors", + "model.layers.19.mlp.experts.71.gate_proj.weight": "model-00020-of-00101.safetensors", + "model.layers.19.mlp.experts.71.up_proj.weight": "model-00020-of-00101.safetensors", + "model.layers.19.mlp.experts.71.down_proj.weight": "model-00020-of-00101.safetensors", + "model.layers.19.mlp.experts.72.gate_proj.weight": "model-00020-of-00101.safetensors", + "model.layers.19.mlp.experts.72.up_proj.weight": "model-00020-of-00101.safetensors", + "model.layers.19.mlp.experts.72.down_proj.weight": "model-00020-of-00101.safetensors", + "model.layers.19.mlp.experts.73.gate_proj.weight": "model-00020-of-00101.safetensors", + "model.layers.19.mlp.experts.73.up_proj.weight": "model-00020-of-00101.safetensors", + "model.layers.19.mlp.experts.73.down_proj.weight": "model-00020-of-00101.safetensors", + "model.layers.19.mlp.experts.74.gate_proj.weight": "model-00020-of-00101.safetensors", + "model.layers.19.mlp.experts.74.up_proj.weight": "model-00020-of-00101.safetensors", + "model.layers.19.mlp.experts.74.down_proj.weight": "model-00020-of-00101.safetensors", + "model.layers.19.mlp.experts.75.gate_proj.weight": "model-00020-of-00101.safetensors", + "model.layers.19.mlp.experts.75.up_proj.weight": "model-00020-of-00101.safetensors", + "model.layers.19.mlp.experts.75.down_proj.weight": "model-00020-of-00101.safetensors", + "model.layers.19.mlp.experts.76.gate_proj.weight": "model-00020-of-00101.safetensors", + "model.layers.19.mlp.experts.76.up_proj.weight": "model-00020-of-00101.safetensors", + "model.layers.19.mlp.experts.76.down_proj.weight": "model-00020-of-00101.safetensors", + "model.layers.19.mlp.experts.77.gate_proj.weight": "model-00020-of-00101.safetensors", + "model.layers.19.mlp.experts.77.up_proj.weight": "model-00020-of-00101.safetensors", + "model.layers.19.mlp.experts.77.down_proj.weight": "model-00020-of-00101.safetensors", + "model.layers.19.mlp.experts.78.gate_proj.weight": "model-00020-of-00101.safetensors", + "model.layers.19.mlp.experts.78.up_proj.weight": "model-00020-of-00101.safetensors", + "model.layers.19.mlp.experts.78.down_proj.weight": "model-00020-of-00101.safetensors", + "model.layers.19.mlp.experts.79.gate_proj.weight": "model-00020-of-00101.safetensors", + "model.layers.19.mlp.experts.79.up_proj.weight": "model-00020-of-00101.safetensors", + "model.layers.19.mlp.experts.79.down_proj.weight": "model-00020-of-00101.safetensors", + "model.layers.19.mlp.experts.80.gate_proj.weight": "model-00020-of-00101.safetensors", + "model.layers.19.mlp.experts.80.up_proj.weight": "model-00020-of-00101.safetensors", + "model.layers.19.mlp.experts.80.down_proj.weight": "model-00020-of-00101.safetensors", + "model.layers.19.mlp.experts.81.gate_proj.weight": "model-00020-of-00101.safetensors", + "model.layers.19.mlp.experts.81.up_proj.weight": "model-00020-of-00101.safetensors", + "model.layers.19.mlp.experts.81.down_proj.weight": "model-00020-of-00101.safetensors", + "model.layers.19.mlp.experts.82.gate_proj.weight": "model-00020-of-00101.safetensors", + "model.layers.19.mlp.experts.82.up_proj.weight": "model-00020-of-00101.safetensors", + "model.layers.19.mlp.experts.82.down_proj.weight": "model-00020-of-00101.safetensors", + "model.layers.19.mlp.experts.83.gate_proj.weight": "model-00020-of-00101.safetensors", + "model.layers.19.mlp.experts.83.up_proj.weight": "model-00020-of-00101.safetensors", + "model.layers.19.mlp.experts.83.down_proj.weight": "model-00020-of-00101.safetensors", + "model.layers.19.mlp.experts.84.gate_proj.weight": "model-00020-of-00101.safetensors", + "model.layers.19.mlp.experts.84.up_proj.weight": "model-00020-of-00101.safetensors", + "model.layers.19.mlp.experts.84.down_proj.weight": "model-00020-of-00101.safetensors", + "model.layers.19.mlp.experts.85.gate_proj.weight": "model-00020-of-00101.safetensors", + "model.layers.19.mlp.experts.85.up_proj.weight": "model-00020-of-00101.safetensors", + "model.layers.19.mlp.experts.85.down_proj.weight": "model-00020-of-00101.safetensors", + "model.layers.19.mlp.experts.86.gate_proj.weight": "model-00020-of-00101.safetensors", + "model.layers.19.mlp.experts.86.up_proj.weight": "model-00020-of-00101.safetensors", + "model.layers.19.mlp.experts.86.down_proj.weight": "model-00020-of-00101.safetensors", + "model.layers.19.mlp.experts.87.gate_proj.weight": "model-00020-of-00101.safetensors", + "model.layers.19.mlp.experts.87.up_proj.weight": "model-00020-of-00101.safetensors", + "model.layers.19.mlp.experts.87.down_proj.weight": "model-00020-of-00101.safetensors", + "model.layers.19.mlp.experts.88.gate_proj.weight": "model-00020-of-00101.safetensors", + "model.layers.19.mlp.experts.88.up_proj.weight": "model-00020-of-00101.safetensors", + "model.layers.19.mlp.experts.88.down_proj.weight": "model-00020-of-00101.safetensors", + "model.layers.19.mlp.experts.89.gate_proj.weight": "model-00020-of-00101.safetensors", + "model.layers.19.mlp.experts.89.up_proj.weight": "model-00020-of-00101.safetensors", + "model.layers.19.mlp.experts.89.down_proj.weight": "model-00020-of-00101.safetensors", + "model.layers.19.mlp.experts.90.gate_proj.weight": "model-00020-of-00101.safetensors", + "model.layers.19.mlp.experts.90.up_proj.weight": "model-00020-of-00101.safetensors", + "model.layers.19.mlp.experts.90.down_proj.weight": "model-00020-of-00101.safetensors", + "model.layers.19.mlp.experts.91.gate_proj.weight": "model-00020-of-00101.safetensors", + "model.layers.19.mlp.experts.91.up_proj.weight": "model-00020-of-00101.safetensors", + "model.layers.19.mlp.experts.91.down_proj.weight": "model-00020-of-00101.safetensors", + "model.layers.19.mlp.experts.92.gate_proj.weight": "model-00020-of-00101.safetensors", + "model.layers.19.mlp.experts.92.up_proj.weight": "model-00020-of-00101.safetensors", + "model.layers.19.mlp.experts.92.down_proj.weight": "model-00020-of-00101.safetensors", + "model.layers.19.mlp.experts.93.gate_proj.weight": "model-00020-of-00101.safetensors", + "model.layers.19.mlp.experts.93.up_proj.weight": "model-00020-of-00101.safetensors", + "model.layers.19.mlp.experts.93.down_proj.weight": "model-00020-of-00101.safetensors", + "model.layers.19.mlp.experts.94.gate_proj.weight": "model-00020-of-00101.safetensors", + "model.layers.19.mlp.experts.94.up_proj.weight": "model-00020-of-00101.safetensors", + "model.layers.19.mlp.experts.94.down_proj.weight": "model-00020-of-00101.safetensors", + "model.layers.19.mlp.experts.95.gate_proj.weight": "model-00020-of-00101.safetensors", + "model.layers.19.mlp.experts.95.up_proj.weight": "model-00020-of-00101.safetensors", + "model.layers.19.mlp.experts.95.down_proj.weight": "model-00020-of-00101.safetensors", + "model.layers.19.mlp.experts.96.gate_proj.weight": "model-00020-of-00101.safetensors", + "model.layers.19.mlp.experts.96.up_proj.weight": "model-00020-of-00101.safetensors", + "model.layers.19.mlp.experts.96.down_proj.weight": "model-00020-of-00101.safetensors", + "model.layers.19.mlp.experts.97.gate_proj.weight": "model-00020-of-00101.safetensors", + "model.layers.19.mlp.experts.97.up_proj.weight": "model-00020-of-00101.safetensors", + "model.layers.19.mlp.experts.97.down_proj.weight": "model-00020-of-00101.safetensors", + "model.layers.19.mlp.experts.98.gate_proj.weight": "model-00020-of-00101.safetensors", + "model.layers.19.mlp.experts.98.up_proj.weight": "model-00020-of-00101.safetensors", + "model.layers.19.mlp.experts.98.down_proj.weight": "model-00020-of-00101.safetensors", + "model.layers.19.mlp.experts.99.gate_proj.weight": "model-00020-of-00101.safetensors", + "model.layers.19.mlp.experts.99.up_proj.weight": "model-00020-of-00101.safetensors", + "model.layers.19.mlp.experts.99.down_proj.weight": "model-00020-of-00101.safetensors", + "model.layers.19.mlp.experts.100.gate_proj.weight": "model-00020-of-00101.safetensors", + "model.layers.19.mlp.experts.100.up_proj.weight": "model-00020-of-00101.safetensors", + "model.layers.19.mlp.experts.100.down_proj.weight": "model-00020-of-00101.safetensors", + "model.layers.19.mlp.experts.101.gate_proj.weight": "model-00020-of-00101.safetensors", + "model.layers.19.mlp.experts.101.up_proj.weight": "model-00020-of-00101.safetensors", + "model.layers.19.mlp.experts.101.down_proj.weight": "model-00020-of-00101.safetensors", + "model.layers.19.mlp.experts.102.gate_proj.weight": "model-00020-of-00101.safetensors", + "model.layers.19.mlp.experts.102.up_proj.weight": "model-00020-of-00101.safetensors", + "model.layers.19.mlp.experts.102.down_proj.weight": "model-00020-of-00101.safetensors", + "model.layers.19.mlp.experts.103.gate_proj.weight": "model-00020-of-00101.safetensors", + "model.layers.19.mlp.experts.103.up_proj.weight": "model-00020-of-00101.safetensors", + "model.layers.19.mlp.experts.103.down_proj.weight": "model-00020-of-00101.safetensors", + "model.layers.19.mlp.experts.104.gate_proj.weight": "model-00020-of-00101.safetensors", + "model.layers.19.mlp.experts.104.up_proj.weight": "model-00020-of-00101.safetensors", + "model.layers.19.mlp.experts.104.down_proj.weight": "model-00020-of-00101.safetensors", + "model.layers.19.mlp.experts.105.gate_proj.weight": "model-00020-of-00101.safetensors", + "model.layers.19.mlp.experts.105.up_proj.weight": "model-00020-of-00101.safetensors", + "model.layers.19.mlp.experts.105.down_proj.weight": "model-00020-of-00101.safetensors", + "model.layers.19.mlp.experts.106.gate_proj.weight": "model-00020-of-00101.safetensors", + "model.layers.19.mlp.experts.106.up_proj.weight": "model-00020-of-00101.safetensors", + "model.layers.19.mlp.experts.106.down_proj.weight": "model-00020-of-00101.safetensors", + "model.layers.19.mlp.experts.107.gate_proj.weight": "model-00020-of-00101.safetensors", + "model.layers.19.mlp.experts.107.up_proj.weight": "model-00020-of-00101.safetensors", + "model.layers.19.mlp.experts.107.down_proj.weight": "model-00020-of-00101.safetensors", + "model.layers.19.mlp.experts.108.gate_proj.weight": "model-00020-of-00101.safetensors", + "model.layers.19.mlp.experts.108.up_proj.weight": "model-00020-of-00101.safetensors", + "model.layers.19.mlp.experts.108.down_proj.weight": "model-00020-of-00101.safetensors", + "model.layers.19.mlp.experts.109.gate_proj.weight": "model-00020-of-00101.safetensors", + "model.layers.19.mlp.experts.109.up_proj.weight": "model-00020-of-00101.safetensors", + "model.layers.19.mlp.experts.109.down_proj.weight": "model-00020-of-00101.safetensors", + "model.layers.19.mlp.experts.110.gate_proj.weight": "model-00020-of-00101.safetensors", + "model.layers.19.mlp.experts.110.up_proj.weight": "model-00020-of-00101.safetensors", + "model.layers.19.mlp.experts.110.down_proj.weight": "model-00020-of-00101.safetensors", + "model.layers.19.mlp.experts.111.gate_proj.weight": "model-00020-of-00101.safetensors", + "model.layers.19.mlp.experts.111.up_proj.weight": "model-00020-of-00101.safetensors", + "model.layers.19.mlp.experts.111.down_proj.weight": "model-00020-of-00101.safetensors", + "model.layers.19.mlp.experts.112.gate_proj.weight": "model-00020-of-00101.safetensors", + "model.layers.19.mlp.experts.112.up_proj.weight": "model-00020-of-00101.safetensors", + "model.layers.19.mlp.experts.112.down_proj.weight": "model-00020-of-00101.safetensors", + "model.layers.19.mlp.experts.113.gate_proj.weight": "model-00020-of-00101.safetensors", + "model.layers.19.mlp.experts.113.up_proj.weight": "model-00020-of-00101.safetensors", + "model.layers.19.mlp.experts.113.down_proj.weight": "model-00020-of-00101.safetensors", + "model.layers.19.mlp.experts.114.gate_proj.weight": "model-00020-of-00101.safetensors", + "model.layers.19.mlp.experts.114.up_proj.weight": "model-00020-of-00101.safetensors", + "model.layers.19.mlp.experts.114.down_proj.weight": "model-00020-of-00101.safetensors", + "model.layers.19.mlp.experts.115.gate_proj.weight": "model-00020-of-00101.safetensors", + "model.layers.19.mlp.experts.115.up_proj.weight": "model-00020-of-00101.safetensors", + "model.layers.19.mlp.experts.115.down_proj.weight": "model-00020-of-00101.safetensors", + "model.layers.19.mlp.experts.116.gate_proj.weight": "model-00020-of-00101.safetensors", + "model.layers.19.mlp.experts.116.up_proj.weight": "model-00020-of-00101.safetensors", + "model.layers.19.mlp.experts.116.down_proj.weight": "model-00020-of-00101.safetensors", + "model.layers.19.mlp.experts.117.gate_proj.weight": "model-00020-of-00101.safetensors", + "model.layers.19.mlp.experts.117.up_proj.weight": "model-00020-of-00101.safetensors", + "model.layers.19.mlp.experts.117.down_proj.weight": "model-00020-of-00101.safetensors", + "model.layers.19.mlp.experts.118.gate_proj.weight": "model-00020-of-00101.safetensors", + "model.layers.19.mlp.experts.118.up_proj.weight": "model-00020-of-00101.safetensors", + "model.layers.19.mlp.experts.118.down_proj.weight": "model-00020-of-00101.safetensors", + "model.layers.19.mlp.experts.119.gate_proj.weight": "model-00020-of-00101.safetensors", + "model.layers.19.mlp.experts.119.up_proj.weight": "model-00020-of-00101.safetensors", + "model.layers.19.mlp.experts.119.down_proj.weight": "model-00020-of-00101.safetensors", + "model.layers.19.mlp.gate.weight": "model-00020-of-00101.safetensors", + "model.layers.19.mlp.gate.e_score_correction_bias": "model-00020-of-00101.safetensors", + "model.layers.19.mlp.shared_experts.gate_proj.weight": "model-00020-of-00101.safetensors", + "model.layers.19.mlp.shared_experts.up_proj.weight": "model-00020-of-00101.safetensors", + "model.layers.19.mlp.shared_experts.down_proj.weight": "model-00020-of-00101.safetensors", + "model.layers.19.input_layernorm.weight": "model-00020-of-00101.safetensors", + "model.layers.19.post_attention_layernorm.weight": "model-00020-of-00101.safetensors", + "model.layers.20.self_attn.q_proj.weight": "model-00020-of-00101.safetensors", + "model.layers.20.self_attn.q_proj.bias": "model-00020-of-00101.safetensors", + "model.layers.20.self_attn.k_proj.weight": "model-00020-of-00101.safetensors", + "model.layers.20.self_attn.k_proj.bias": "model-00020-of-00101.safetensors", + "model.layers.20.self_attn.v_proj.weight": "model-00020-of-00101.safetensors", + "model.layers.20.self_attn.v_proj.bias": "model-00020-of-00101.safetensors", + "model.layers.20.self_attn.o_proj.weight": "model-00020-of-00101.safetensors", + "model.layers.20.self_attn.q_norm.weight": "model-00020-of-00101.safetensors", + "model.layers.20.self_attn.k_norm.weight": "model-00020-of-00101.safetensors", + "model.layers.20.mlp.experts.0.gate_proj.weight": "model-00020-of-00101.safetensors", + "model.layers.20.mlp.experts.0.up_proj.weight": "model-00020-of-00101.safetensors", + "model.layers.20.mlp.experts.0.down_proj.weight": "model-00020-of-00101.safetensors", + "model.layers.20.mlp.experts.1.gate_proj.weight": "model-00020-of-00101.safetensors", + "model.layers.20.mlp.experts.1.up_proj.weight": "model-00020-of-00101.safetensors", + "model.layers.20.mlp.experts.1.down_proj.weight": "model-00020-of-00101.safetensors", + "model.layers.20.mlp.experts.2.gate_proj.weight": "model-00020-of-00101.safetensors", + "model.layers.20.mlp.experts.2.up_proj.weight": "model-00020-of-00101.safetensors", + "model.layers.20.mlp.experts.2.down_proj.weight": "model-00020-of-00101.safetensors", + "model.layers.20.mlp.experts.3.gate_proj.weight": "model-00020-of-00101.safetensors", + "model.layers.20.mlp.experts.3.up_proj.weight": "model-00020-of-00101.safetensors", + "model.layers.20.mlp.experts.3.down_proj.weight": "model-00020-of-00101.safetensors", + "model.layers.20.mlp.experts.4.gate_proj.weight": "model-00020-of-00101.safetensors", + "model.layers.20.mlp.experts.4.up_proj.weight": "model-00020-of-00101.safetensors", + "model.layers.20.mlp.experts.4.down_proj.weight": "model-00020-of-00101.safetensors", + "model.layers.20.mlp.experts.5.gate_proj.weight": "model-00020-of-00101.safetensors", + "model.layers.20.mlp.experts.5.up_proj.weight": "model-00020-of-00101.safetensors", + "model.layers.20.mlp.experts.5.down_proj.weight": "model-00020-of-00101.safetensors", + "model.layers.20.mlp.experts.6.gate_proj.weight": "model-00020-of-00101.safetensors", + "model.layers.20.mlp.experts.6.up_proj.weight": "model-00020-of-00101.safetensors", + "model.layers.20.mlp.experts.6.down_proj.weight": "model-00020-of-00101.safetensors", + "model.layers.20.mlp.experts.7.gate_proj.weight": "model-00020-of-00101.safetensors", + "model.layers.20.mlp.experts.7.up_proj.weight": "model-00020-of-00101.safetensors", + "model.layers.20.mlp.experts.7.down_proj.weight": "model-00020-of-00101.safetensors", + "model.layers.20.mlp.experts.8.gate_proj.weight": "model-00020-of-00101.safetensors", + "model.layers.20.mlp.experts.8.up_proj.weight": "model-00020-of-00101.safetensors", + "model.layers.20.mlp.experts.8.down_proj.weight": "model-00020-of-00101.safetensors", + "model.layers.20.mlp.experts.9.gate_proj.weight": "model-00020-of-00101.safetensors", + "model.layers.20.mlp.experts.9.up_proj.weight": "model-00020-of-00101.safetensors", + "model.layers.20.mlp.experts.9.down_proj.weight": "model-00020-of-00101.safetensors", + "model.layers.20.mlp.experts.10.gate_proj.weight": "model-00020-of-00101.safetensors", + "model.layers.20.mlp.experts.10.up_proj.weight": "model-00020-of-00101.safetensors", + "model.layers.20.mlp.experts.10.down_proj.weight": "model-00020-of-00101.safetensors", + "model.layers.20.mlp.experts.11.gate_proj.weight": "model-00020-of-00101.safetensors", + "model.layers.20.mlp.experts.11.up_proj.weight": "model-00020-of-00101.safetensors", + "model.layers.20.mlp.experts.11.down_proj.weight": "model-00020-of-00101.safetensors", + "model.layers.20.mlp.experts.12.gate_proj.weight": "model-00020-of-00101.safetensors", + "model.layers.20.mlp.experts.12.up_proj.weight": "model-00020-of-00101.safetensors", + "model.layers.20.mlp.experts.12.down_proj.weight": "model-00020-of-00101.safetensors", + "model.layers.20.mlp.experts.13.gate_proj.weight": "model-00020-of-00101.safetensors", + "model.layers.20.mlp.experts.13.up_proj.weight": "model-00020-of-00101.safetensors", + "model.layers.20.mlp.experts.13.down_proj.weight": "model-00020-of-00101.safetensors", + "model.layers.20.mlp.experts.14.gate_proj.weight": "model-00020-of-00101.safetensors", + "model.layers.20.mlp.experts.14.up_proj.weight": "model-00020-of-00101.safetensors", + "model.layers.20.mlp.experts.14.down_proj.weight": "model-00020-of-00101.safetensors", + "model.layers.20.mlp.experts.15.gate_proj.weight": "model-00020-of-00101.safetensors", + "model.layers.20.mlp.experts.15.up_proj.weight": "model-00020-of-00101.safetensors", + "model.layers.20.mlp.experts.15.down_proj.weight": "model-00020-of-00101.safetensors", + "model.layers.20.mlp.experts.16.gate_proj.weight": "model-00020-of-00101.safetensors", + "model.layers.20.mlp.experts.16.up_proj.weight": "model-00020-of-00101.safetensors", + "model.layers.20.mlp.experts.16.down_proj.weight": "model-00020-of-00101.safetensors", + "model.layers.20.mlp.experts.17.gate_proj.weight": "model-00020-of-00101.safetensors", + "model.layers.20.mlp.experts.17.up_proj.weight": "model-00020-of-00101.safetensors", + "model.layers.20.mlp.experts.17.down_proj.weight": "model-00020-of-00101.safetensors", + "model.layers.20.mlp.experts.18.gate_proj.weight": "model-00020-of-00101.safetensors", + "model.layers.20.mlp.experts.18.up_proj.weight": "model-00020-of-00101.safetensors", + "model.layers.20.mlp.experts.18.down_proj.weight": "model-00020-of-00101.safetensors", + "model.layers.20.mlp.experts.19.gate_proj.weight": "model-00020-of-00101.safetensors", + "model.layers.20.mlp.experts.19.up_proj.weight": "model-00020-of-00101.safetensors", + "model.layers.20.mlp.experts.19.down_proj.weight": "model-00020-of-00101.safetensors", + "model.layers.20.mlp.experts.20.gate_proj.weight": "model-00020-of-00101.safetensors", + "model.layers.20.mlp.experts.20.up_proj.weight": "model-00020-of-00101.safetensors", + "model.layers.20.mlp.experts.20.down_proj.weight": "model-00020-of-00101.safetensors", + "model.layers.20.mlp.experts.21.gate_proj.weight": "model-00020-of-00101.safetensors", + "model.layers.20.mlp.experts.21.up_proj.weight": "model-00020-of-00101.safetensors", + "model.layers.20.mlp.experts.21.down_proj.weight": "model-00020-of-00101.safetensors", + "model.layers.20.mlp.experts.22.gate_proj.weight": "model-00020-of-00101.safetensors", + "model.layers.20.mlp.experts.22.up_proj.weight": "model-00020-of-00101.safetensors", + "model.layers.20.mlp.experts.22.down_proj.weight": "model-00020-of-00101.safetensors", + "model.layers.20.mlp.experts.23.gate_proj.weight": "model-00020-of-00101.safetensors", + "model.layers.20.mlp.experts.23.up_proj.weight": "model-00020-of-00101.safetensors", + "model.layers.20.mlp.experts.23.down_proj.weight": "model-00020-of-00101.safetensors", + "model.layers.20.mlp.experts.24.gate_proj.weight": "model-00020-of-00101.safetensors", + "model.layers.20.mlp.experts.24.up_proj.weight": "model-00020-of-00101.safetensors", + "model.layers.20.mlp.experts.24.down_proj.weight": "model-00020-of-00101.safetensors", + "model.layers.20.mlp.experts.25.gate_proj.weight": "model-00020-of-00101.safetensors", + "model.layers.20.mlp.experts.25.up_proj.weight": "model-00020-of-00101.safetensors", + "model.layers.20.mlp.experts.25.down_proj.weight": "model-00020-of-00101.safetensors", + "model.layers.20.mlp.experts.26.gate_proj.weight": "model-00020-of-00101.safetensors", + "model.layers.20.mlp.experts.26.up_proj.weight": "model-00020-of-00101.safetensors", + "model.layers.20.mlp.experts.26.down_proj.weight": "model-00020-of-00101.safetensors", + "model.layers.20.mlp.experts.27.gate_proj.weight": "model-00020-of-00101.safetensors", + "model.layers.20.mlp.experts.27.up_proj.weight": "model-00020-of-00101.safetensors", + "model.layers.20.mlp.experts.27.down_proj.weight": "model-00020-of-00101.safetensors", + "model.layers.20.mlp.experts.28.gate_proj.weight": "model-00020-of-00101.safetensors", + "model.layers.20.mlp.experts.28.up_proj.weight": "model-00020-of-00101.safetensors", + "model.layers.20.mlp.experts.28.down_proj.weight": "model-00020-of-00101.safetensors", + "model.layers.20.mlp.experts.29.gate_proj.weight": "model-00020-of-00101.safetensors", + "model.layers.20.mlp.experts.29.up_proj.weight": "model-00020-of-00101.safetensors", + "model.layers.20.mlp.experts.29.down_proj.weight": "model-00020-of-00101.safetensors", + "model.layers.20.mlp.experts.30.gate_proj.weight": "model-00020-of-00101.safetensors", + "model.layers.20.mlp.experts.30.up_proj.weight": "model-00020-of-00101.safetensors", + "model.layers.20.mlp.experts.30.down_proj.weight": "model-00020-of-00101.safetensors", + "model.layers.20.mlp.experts.31.gate_proj.weight": "model-00020-of-00101.safetensors", + "model.layers.20.mlp.experts.31.up_proj.weight": "model-00020-of-00101.safetensors", + "model.layers.20.mlp.experts.31.down_proj.weight": "model-00020-of-00101.safetensors", + "model.layers.20.mlp.experts.32.gate_proj.weight": "model-00020-of-00101.safetensors", + "model.layers.20.mlp.experts.32.up_proj.weight": "model-00020-of-00101.safetensors", + "model.layers.20.mlp.experts.32.down_proj.weight": "model-00020-of-00101.safetensors", + "model.layers.20.mlp.experts.33.gate_proj.weight": "model-00020-of-00101.safetensors", + "model.layers.20.mlp.experts.33.up_proj.weight": "model-00020-of-00101.safetensors", + "model.layers.20.mlp.experts.33.down_proj.weight": "model-00020-of-00101.safetensors", + "model.layers.20.mlp.experts.34.gate_proj.weight": "model-00020-of-00101.safetensors", + "model.layers.20.mlp.experts.34.up_proj.weight": "model-00021-of-00101.safetensors", + "model.layers.20.mlp.experts.34.down_proj.weight": "model-00021-of-00101.safetensors", + "model.layers.20.mlp.experts.35.gate_proj.weight": "model-00021-of-00101.safetensors", + "model.layers.20.mlp.experts.35.up_proj.weight": "model-00021-of-00101.safetensors", + "model.layers.20.mlp.experts.35.down_proj.weight": "model-00021-of-00101.safetensors", + "model.layers.20.mlp.experts.36.gate_proj.weight": "model-00021-of-00101.safetensors", + "model.layers.20.mlp.experts.36.up_proj.weight": "model-00021-of-00101.safetensors", + "model.layers.20.mlp.experts.36.down_proj.weight": "model-00021-of-00101.safetensors", + "model.layers.20.mlp.experts.37.gate_proj.weight": "model-00021-of-00101.safetensors", + "model.layers.20.mlp.experts.37.up_proj.weight": "model-00021-of-00101.safetensors", + "model.layers.20.mlp.experts.37.down_proj.weight": "model-00021-of-00101.safetensors", + "model.layers.20.mlp.experts.38.gate_proj.weight": "model-00021-of-00101.safetensors", + "model.layers.20.mlp.experts.38.up_proj.weight": "model-00021-of-00101.safetensors", + "model.layers.20.mlp.experts.38.down_proj.weight": "model-00021-of-00101.safetensors", + "model.layers.20.mlp.experts.39.gate_proj.weight": "model-00021-of-00101.safetensors", + "model.layers.20.mlp.experts.39.up_proj.weight": "model-00021-of-00101.safetensors", + "model.layers.20.mlp.experts.39.down_proj.weight": "model-00021-of-00101.safetensors", + "model.layers.20.mlp.experts.40.gate_proj.weight": "model-00021-of-00101.safetensors", + "model.layers.20.mlp.experts.40.up_proj.weight": "model-00021-of-00101.safetensors", + "model.layers.20.mlp.experts.40.down_proj.weight": "model-00021-of-00101.safetensors", + "model.layers.20.mlp.experts.41.gate_proj.weight": "model-00021-of-00101.safetensors", + "model.layers.20.mlp.experts.41.up_proj.weight": "model-00021-of-00101.safetensors", + "model.layers.20.mlp.experts.41.down_proj.weight": "model-00021-of-00101.safetensors", + "model.layers.20.mlp.experts.42.gate_proj.weight": "model-00021-of-00101.safetensors", + "model.layers.20.mlp.experts.42.up_proj.weight": "model-00021-of-00101.safetensors", + "model.layers.20.mlp.experts.42.down_proj.weight": "model-00021-of-00101.safetensors", + "model.layers.20.mlp.experts.43.gate_proj.weight": "model-00021-of-00101.safetensors", + "model.layers.20.mlp.experts.43.up_proj.weight": "model-00021-of-00101.safetensors", + "model.layers.20.mlp.experts.43.down_proj.weight": "model-00021-of-00101.safetensors", + "model.layers.20.mlp.experts.44.gate_proj.weight": "model-00021-of-00101.safetensors", + "model.layers.20.mlp.experts.44.up_proj.weight": "model-00021-of-00101.safetensors", + "model.layers.20.mlp.experts.44.down_proj.weight": "model-00021-of-00101.safetensors", + "model.layers.20.mlp.experts.45.gate_proj.weight": "model-00021-of-00101.safetensors", + "model.layers.20.mlp.experts.45.up_proj.weight": "model-00021-of-00101.safetensors", + "model.layers.20.mlp.experts.45.down_proj.weight": "model-00021-of-00101.safetensors", + "model.layers.20.mlp.experts.46.gate_proj.weight": "model-00021-of-00101.safetensors", + "model.layers.20.mlp.experts.46.up_proj.weight": "model-00021-of-00101.safetensors", + "model.layers.20.mlp.experts.46.down_proj.weight": "model-00021-of-00101.safetensors", + "model.layers.20.mlp.experts.47.gate_proj.weight": "model-00021-of-00101.safetensors", + "model.layers.20.mlp.experts.47.up_proj.weight": "model-00021-of-00101.safetensors", + "model.layers.20.mlp.experts.47.down_proj.weight": "model-00021-of-00101.safetensors", + "model.layers.20.mlp.experts.48.gate_proj.weight": "model-00021-of-00101.safetensors", + "model.layers.20.mlp.experts.48.up_proj.weight": "model-00021-of-00101.safetensors", + "model.layers.20.mlp.experts.48.down_proj.weight": "model-00021-of-00101.safetensors", + "model.layers.20.mlp.experts.49.gate_proj.weight": "model-00021-of-00101.safetensors", + "model.layers.20.mlp.experts.49.up_proj.weight": "model-00021-of-00101.safetensors", + "model.layers.20.mlp.experts.49.down_proj.weight": "model-00021-of-00101.safetensors", + "model.layers.20.mlp.experts.50.gate_proj.weight": "model-00021-of-00101.safetensors", + "model.layers.20.mlp.experts.50.up_proj.weight": "model-00021-of-00101.safetensors", + "model.layers.20.mlp.experts.50.down_proj.weight": "model-00021-of-00101.safetensors", + "model.layers.20.mlp.experts.51.gate_proj.weight": "model-00021-of-00101.safetensors", + "model.layers.20.mlp.experts.51.up_proj.weight": "model-00021-of-00101.safetensors", + "model.layers.20.mlp.experts.51.down_proj.weight": "model-00021-of-00101.safetensors", + "model.layers.20.mlp.experts.52.gate_proj.weight": "model-00021-of-00101.safetensors", + "model.layers.20.mlp.experts.52.up_proj.weight": "model-00021-of-00101.safetensors", + "model.layers.20.mlp.experts.52.down_proj.weight": "model-00021-of-00101.safetensors", + "model.layers.20.mlp.experts.53.gate_proj.weight": "model-00021-of-00101.safetensors", + "model.layers.20.mlp.experts.53.up_proj.weight": "model-00021-of-00101.safetensors", + "model.layers.20.mlp.experts.53.down_proj.weight": "model-00021-of-00101.safetensors", + "model.layers.20.mlp.experts.54.gate_proj.weight": "model-00021-of-00101.safetensors", + "model.layers.20.mlp.experts.54.up_proj.weight": "model-00021-of-00101.safetensors", + "model.layers.20.mlp.experts.54.down_proj.weight": "model-00021-of-00101.safetensors", + "model.layers.20.mlp.experts.55.gate_proj.weight": "model-00021-of-00101.safetensors", + "model.layers.20.mlp.experts.55.up_proj.weight": "model-00021-of-00101.safetensors", + "model.layers.20.mlp.experts.55.down_proj.weight": "model-00021-of-00101.safetensors", + "model.layers.20.mlp.experts.56.gate_proj.weight": "model-00021-of-00101.safetensors", + "model.layers.20.mlp.experts.56.up_proj.weight": "model-00021-of-00101.safetensors", + "model.layers.20.mlp.experts.56.down_proj.weight": "model-00021-of-00101.safetensors", + "model.layers.20.mlp.experts.57.gate_proj.weight": "model-00021-of-00101.safetensors", + "model.layers.20.mlp.experts.57.up_proj.weight": "model-00021-of-00101.safetensors", + "model.layers.20.mlp.experts.57.down_proj.weight": "model-00021-of-00101.safetensors", + "model.layers.20.mlp.experts.58.gate_proj.weight": "model-00021-of-00101.safetensors", + "model.layers.20.mlp.experts.58.up_proj.weight": "model-00021-of-00101.safetensors", + "model.layers.20.mlp.experts.58.down_proj.weight": "model-00021-of-00101.safetensors", + "model.layers.20.mlp.experts.59.gate_proj.weight": "model-00021-of-00101.safetensors", + "model.layers.20.mlp.experts.59.up_proj.weight": "model-00021-of-00101.safetensors", + "model.layers.20.mlp.experts.59.down_proj.weight": "model-00021-of-00101.safetensors", + "model.layers.20.mlp.experts.60.gate_proj.weight": "model-00021-of-00101.safetensors", + "model.layers.20.mlp.experts.60.up_proj.weight": "model-00021-of-00101.safetensors", + "model.layers.20.mlp.experts.60.down_proj.weight": "model-00021-of-00101.safetensors", + "model.layers.20.mlp.experts.61.gate_proj.weight": "model-00021-of-00101.safetensors", + "model.layers.20.mlp.experts.61.up_proj.weight": "model-00021-of-00101.safetensors", + "model.layers.20.mlp.experts.61.down_proj.weight": "model-00021-of-00101.safetensors", + "model.layers.20.mlp.experts.62.gate_proj.weight": "model-00021-of-00101.safetensors", + "model.layers.20.mlp.experts.62.up_proj.weight": "model-00021-of-00101.safetensors", + "model.layers.20.mlp.experts.62.down_proj.weight": "model-00021-of-00101.safetensors", + "model.layers.20.mlp.experts.63.gate_proj.weight": "model-00021-of-00101.safetensors", + "model.layers.20.mlp.experts.63.up_proj.weight": "model-00021-of-00101.safetensors", + "model.layers.20.mlp.experts.63.down_proj.weight": "model-00021-of-00101.safetensors", + "model.layers.20.mlp.experts.64.gate_proj.weight": "model-00021-of-00101.safetensors", + "model.layers.20.mlp.experts.64.up_proj.weight": "model-00021-of-00101.safetensors", + "model.layers.20.mlp.experts.64.down_proj.weight": "model-00021-of-00101.safetensors", + "model.layers.20.mlp.experts.65.gate_proj.weight": "model-00021-of-00101.safetensors", + "model.layers.20.mlp.experts.65.up_proj.weight": "model-00021-of-00101.safetensors", + "model.layers.20.mlp.experts.65.down_proj.weight": "model-00021-of-00101.safetensors", + "model.layers.20.mlp.experts.66.gate_proj.weight": "model-00021-of-00101.safetensors", + "model.layers.20.mlp.experts.66.up_proj.weight": "model-00021-of-00101.safetensors", + "model.layers.20.mlp.experts.66.down_proj.weight": "model-00021-of-00101.safetensors", + "model.layers.20.mlp.experts.67.gate_proj.weight": "model-00021-of-00101.safetensors", + "model.layers.20.mlp.experts.67.up_proj.weight": "model-00021-of-00101.safetensors", + "model.layers.20.mlp.experts.67.down_proj.weight": "model-00021-of-00101.safetensors", + "model.layers.20.mlp.experts.68.gate_proj.weight": "model-00021-of-00101.safetensors", + "model.layers.20.mlp.experts.68.up_proj.weight": "model-00021-of-00101.safetensors", + "model.layers.20.mlp.experts.68.down_proj.weight": "model-00021-of-00101.safetensors", + "model.layers.20.mlp.experts.69.gate_proj.weight": "model-00021-of-00101.safetensors", + "model.layers.20.mlp.experts.69.up_proj.weight": "model-00021-of-00101.safetensors", + "model.layers.20.mlp.experts.69.down_proj.weight": "model-00021-of-00101.safetensors", + "model.layers.20.mlp.experts.70.gate_proj.weight": "model-00021-of-00101.safetensors", + "model.layers.20.mlp.experts.70.up_proj.weight": "model-00021-of-00101.safetensors", + "model.layers.20.mlp.experts.70.down_proj.weight": "model-00021-of-00101.safetensors", + "model.layers.20.mlp.experts.71.gate_proj.weight": "model-00021-of-00101.safetensors", + "model.layers.20.mlp.experts.71.up_proj.weight": "model-00021-of-00101.safetensors", + "model.layers.20.mlp.experts.71.down_proj.weight": "model-00021-of-00101.safetensors", + "model.layers.20.mlp.experts.72.gate_proj.weight": "model-00021-of-00101.safetensors", + "model.layers.20.mlp.experts.72.up_proj.weight": "model-00021-of-00101.safetensors", + "model.layers.20.mlp.experts.72.down_proj.weight": "model-00021-of-00101.safetensors", + "model.layers.20.mlp.experts.73.gate_proj.weight": "model-00021-of-00101.safetensors", + "model.layers.20.mlp.experts.73.up_proj.weight": "model-00021-of-00101.safetensors", + "model.layers.20.mlp.experts.73.down_proj.weight": "model-00021-of-00101.safetensors", + "model.layers.20.mlp.experts.74.gate_proj.weight": "model-00021-of-00101.safetensors", + "model.layers.20.mlp.experts.74.up_proj.weight": "model-00021-of-00101.safetensors", + "model.layers.20.mlp.experts.74.down_proj.weight": "model-00021-of-00101.safetensors", + "model.layers.20.mlp.experts.75.gate_proj.weight": "model-00021-of-00101.safetensors", + "model.layers.20.mlp.experts.75.up_proj.weight": "model-00021-of-00101.safetensors", + "model.layers.20.mlp.experts.75.down_proj.weight": "model-00021-of-00101.safetensors", + "model.layers.20.mlp.experts.76.gate_proj.weight": "model-00021-of-00101.safetensors", + "model.layers.20.mlp.experts.76.up_proj.weight": "model-00021-of-00101.safetensors", + "model.layers.20.mlp.experts.76.down_proj.weight": "model-00021-of-00101.safetensors", + "model.layers.20.mlp.experts.77.gate_proj.weight": "model-00021-of-00101.safetensors", + "model.layers.20.mlp.experts.77.up_proj.weight": "model-00021-of-00101.safetensors", + "model.layers.20.mlp.experts.77.down_proj.weight": "model-00021-of-00101.safetensors", + "model.layers.20.mlp.experts.78.gate_proj.weight": "model-00021-of-00101.safetensors", + "model.layers.20.mlp.experts.78.up_proj.weight": "model-00021-of-00101.safetensors", + "model.layers.20.mlp.experts.78.down_proj.weight": "model-00021-of-00101.safetensors", + "model.layers.20.mlp.experts.79.gate_proj.weight": "model-00021-of-00101.safetensors", + "model.layers.20.mlp.experts.79.up_proj.weight": "model-00021-of-00101.safetensors", + "model.layers.20.mlp.experts.79.down_proj.weight": "model-00021-of-00101.safetensors", + "model.layers.20.mlp.experts.80.gate_proj.weight": "model-00021-of-00101.safetensors", + "model.layers.20.mlp.experts.80.up_proj.weight": "model-00021-of-00101.safetensors", + "model.layers.20.mlp.experts.80.down_proj.weight": "model-00021-of-00101.safetensors", + "model.layers.20.mlp.experts.81.gate_proj.weight": "model-00021-of-00101.safetensors", + "model.layers.20.mlp.experts.81.up_proj.weight": "model-00021-of-00101.safetensors", + "model.layers.20.mlp.experts.81.down_proj.weight": "model-00021-of-00101.safetensors", + "model.layers.20.mlp.experts.82.gate_proj.weight": "model-00021-of-00101.safetensors", + "model.layers.20.mlp.experts.82.up_proj.weight": "model-00021-of-00101.safetensors", + "model.layers.20.mlp.experts.82.down_proj.weight": "model-00021-of-00101.safetensors", + "model.layers.20.mlp.experts.83.gate_proj.weight": "model-00021-of-00101.safetensors", + "model.layers.20.mlp.experts.83.up_proj.weight": "model-00021-of-00101.safetensors", + "model.layers.20.mlp.experts.83.down_proj.weight": "model-00021-of-00101.safetensors", + "model.layers.20.mlp.experts.84.gate_proj.weight": "model-00021-of-00101.safetensors", + "model.layers.20.mlp.experts.84.up_proj.weight": "model-00021-of-00101.safetensors", + "model.layers.20.mlp.experts.84.down_proj.weight": "model-00021-of-00101.safetensors", + "model.layers.20.mlp.experts.85.gate_proj.weight": "model-00021-of-00101.safetensors", + "model.layers.20.mlp.experts.85.up_proj.weight": "model-00021-of-00101.safetensors", + "model.layers.20.mlp.experts.85.down_proj.weight": "model-00021-of-00101.safetensors", + "model.layers.20.mlp.experts.86.gate_proj.weight": "model-00021-of-00101.safetensors", + "model.layers.20.mlp.experts.86.up_proj.weight": "model-00021-of-00101.safetensors", + "model.layers.20.mlp.experts.86.down_proj.weight": "model-00021-of-00101.safetensors", + "model.layers.20.mlp.experts.87.gate_proj.weight": "model-00021-of-00101.safetensors", + "model.layers.20.mlp.experts.87.up_proj.weight": "model-00021-of-00101.safetensors", + "model.layers.20.mlp.experts.87.down_proj.weight": "model-00021-of-00101.safetensors", + "model.layers.20.mlp.experts.88.gate_proj.weight": "model-00021-of-00101.safetensors", + "model.layers.20.mlp.experts.88.up_proj.weight": "model-00021-of-00101.safetensors", + "model.layers.20.mlp.experts.88.down_proj.weight": "model-00021-of-00101.safetensors", + "model.layers.20.mlp.experts.89.gate_proj.weight": "model-00021-of-00101.safetensors", + "model.layers.20.mlp.experts.89.up_proj.weight": "model-00021-of-00101.safetensors", + "model.layers.20.mlp.experts.89.down_proj.weight": "model-00021-of-00101.safetensors", + "model.layers.20.mlp.experts.90.gate_proj.weight": "model-00021-of-00101.safetensors", + "model.layers.20.mlp.experts.90.up_proj.weight": "model-00021-of-00101.safetensors", + "model.layers.20.mlp.experts.90.down_proj.weight": "model-00021-of-00101.safetensors", + "model.layers.20.mlp.experts.91.gate_proj.weight": "model-00021-of-00101.safetensors", + "model.layers.20.mlp.experts.91.up_proj.weight": "model-00021-of-00101.safetensors", + "model.layers.20.mlp.experts.91.down_proj.weight": "model-00021-of-00101.safetensors", + "model.layers.20.mlp.experts.92.gate_proj.weight": "model-00021-of-00101.safetensors", + "model.layers.20.mlp.experts.92.up_proj.weight": "model-00021-of-00101.safetensors", + "model.layers.20.mlp.experts.92.down_proj.weight": "model-00021-of-00101.safetensors", + "model.layers.20.mlp.experts.93.gate_proj.weight": "model-00021-of-00101.safetensors", + "model.layers.20.mlp.experts.93.up_proj.weight": "model-00021-of-00101.safetensors", + "model.layers.20.mlp.experts.93.down_proj.weight": "model-00021-of-00101.safetensors", + "model.layers.20.mlp.experts.94.gate_proj.weight": "model-00021-of-00101.safetensors", + "model.layers.20.mlp.experts.94.up_proj.weight": "model-00021-of-00101.safetensors", + "model.layers.20.mlp.experts.94.down_proj.weight": "model-00021-of-00101.safetensors", + "model.layers.20.mlp.experts.95.gate_proj.weight": "model-00021-of-00101.safetensors", + "model.layers.20.mlp.experts.95.up_proj.weight": "model-00021-of-00101.safetensors", + "model.layers.20.mlp.experts.95.down_proj.weight": "model-00021-of-00101.safetensors", + "model.layers.20.mlp.experts.96.gate_proj.weight": "model-00021-of-00101.safetensors", + "model.layers.20.mlp.experts.96.up_proj.weight": "model-00021-of-00101.safetensors", + "model.layers.20.mlp.experts.96.down_proj.weight": "model-00021-of-00101.safetensors", + "model.layers.20.mlp.experts.97.gate_proj.weight": "model-00021-of-00101.safetensors", + "model.layers.20.mlp.experts.97.up_proj.weight": "model-00021-of-00101.safetensors", + "model.layers.20.mlp.experts.97.down_proj.weight": "model-00021-of-00101.safetensors", + "model.layers.20.mlp.experts.98.gate_proj.weight": "model-00021-of-00101.safetensors", + "model.layers.20.mlp.experts.98.up_proj.weight": "model-00021-of-00101.safetensors", + "model.layers.20.mlp.experts.98.down_proj.weight": "model-00021-of-00101.safetensors", + "model.layers.20.mlp.experts.99.gate_proj.weight": "model-00021-of-00101.safetensors", + "model.layers.20.mlp.experts.99.up_proj.weight": "model-00021-of-00101.safetensors", + "model.layers.20.mlp.experts.99.down_proj.weight": "model-00021-of-00101.safetensors", + "model.layers.20.mlp.experts.100.gate_proj.weight": "model-00021-of-00101.safetensors", + "model.layers.20.mlp.experts.100.up_proj.weight": "model-00021-of-00101.safetensors", + "model.layers.20.mlp.experts.100.down_proj.weight": "model-00021-of-00101.safetensors", + "model.layers.20.mlp.experts.101.gate_proj.weight": "model-00021-of-00101.safetensors", + "model.layers.20.mlp.experts.101.up_proj.weight": "model-00021-of-00101.safetensors", + "model.layers.20.mlp.experts.101.down_proj.weight": "model-00021-of-00101.safetensors", + "model.layers.20.mlp.experts.102.gate_proj.weight": "model-00021-of-00101.safetensors", + "model.layers.20.mlp.experts.102.up_proj.weight": "model-00021-of-00101.safetensors", + "model.layers.20.mlp.experts.102.down_proj.weight": "model-00021-of-00101.safetensors", + "model.layers.20.mlp.experts.103.gate_proj.weight": "model-00021-of-00101.safetensors", + "model.layers.20.mlp.experts.103.up_proj.weight": "model-00021-of-00101.safetensors", + "model.layers.20.mlp.experts.103.down_proj.weight": "model-00021-of-00101.safetensors", + "model.layers.20.mlp.experts.104.gate_proj.weight": "model-00021-of-00101.safetensors", + "model.layers.20.mlp.experts.104.up_proj.weight": "model-00021-of-00101.safetensors", + "model.layers.20.mlp.experts.104.down_proj.weight": "model-00021-of-00101.safetensors", + "model.layers.20.mlp.experts.105.gate_proj.weight": "model-00021-of-00101.safetensors", + "model.layers.20.mlp.experts.105.up_proj.weight": "model-00021-of-00101.safetensors", + "model.layers.20.mlp.experts.105.down_proj.weight": "model-00021-of-00101.safetensors", + "model.layers.20.mlp.experts.106.gate_proj.weight": "model-00021-of-00101.safetensors", + "model.layers.20.mlp.experts.106.up_proj.weight": "model-00021-of-00101.safetensors", + "model.layers.20.mlp.experts.106.down_proj.weight": "model-00021-of-00101.safetensors", + "model.layers.20.mlp.experts.107.gate_proj.weight": "model-00021-of-00101.safetensors", + "model.layers.20.mlp.experts.107.up_proj.weight": "model-00021-of-00101.safetensors", + "model.layers.20.mlp.experts.107.down_proj.weight": "model-00021-of-00101.safetensors", + "model.layers.20.mlp.experts.108.gate_proj.weight": "model-00021-of-00101.safetensors", + "model.layers.20.mlp.experts.108.up_proj.weight": "model-00021-of-00101.safetensors", + "model.layers.20.mlp.experts.108.down_proj.weight": "model-00021-of-00101.safetensors", + "model.layers.20.mlp.experts.109.gate_proj.weight": "model-00021-of-00101.safetensors", + "model.layers.20.mlp.experts.109.up_proj.weight": "model-00021-of-00101.safetensors", + "model.layers.20.mlp.experts.109.down_proj.weight": "model-00021-of-00101.safetensors", + "model.layers.20.mlp.experts.110.gate_proj.weight": "model-00021-of-00101.safetensors", + "model.layers.20.mlp.experts.110.up_proj.weight": "model-00021-of-00101.safetensors", + "model.layers.20.mlp.experts.110.down_proj.weight": "model-00021-of-00101.safetensors", + "model.layers.20.mlp.experts.111.gate_proj.weight": "model-00021-of-00101.safetensors", + "model.layers.20.mlp.experts.111.up_proj.weight": "model-00021-of-00101.safetensors", + "model.layers.20.mlp.experts.111.down_proj.weight": "model-00021-of-00101.safetensors", + "model.layers.20.mlp.experts.112.gate_proj.weight": "model-00021-of-00101.safetensors", + "model.layers.20.mlp.experts.112.up_proj.weight": "model-00021-of-00101.safetensors", + "model.layers.20.mlp.experts.112.down_proj.weight": "model-00021-of-00101.safetensors", + "model.layers.20.mlp.experts.113.gate_proj.weight": "model-00021-of-00101.safetensors", + "model.layers.20.mlp.experts.113.up_proj.weight": "model-00021-of-00101.safetensors", + "model.layers.20.mlp.experts.113.down_proj.weight": "model-00021-of-00101.safetensors", + "model.layers.20.mlp.experts.114.gate_proj.weight": "model-00021-of-00101.safetensors", + "model.layers.20.mlp.experts.114.up_proj.weight": "model-00021-of-00101.safetensors", + "model.layers.20.mlp.experts.114.down_proj.weight": "model-00021-of-00101.safetensors", + "model.layers.20.mlp.experts.115.gate_proj.weight": "model-00021-of-00101.safetensors", + "model.layers.20.mlp.experts.115.up_proj.weight": "model-00021-of-00101.safetensors", + "model.layers.20.mlp.experts.115.down_proj.weight": "model-00021-of-00101.safetensors", + "model.layers.20.mlp.experts.116.gate_proj.weight": "model-00021-of-00101.safetensors", + "model.layers.20.mlp.experts.116.up_proj.weight": "model-00021-of-00101.safetensors", + "model.layers.20.mlp.experts.116.down_proj.weight": "model-00021-of-00101.safetensors", + "model.layers.20.mlp.experts.117.gate_proj.weight": "model-00021-of-00101.safetensors", + "model.layers.20.mlp.experts.117.up_proj.weight": "model-00021-of-00101.safetensors", + "model.layers.20.mlp.experts.117.down_proj.weight": "model-00021-of-00101.safetensors", + "model.layers.20.mlp.experts.118.gate_proj.weight": "model-00021-of-00101.safetensors", + "model.layers.20.mlp.experts.118.up_proj.weight": "model-00021-of-00101.safetensors", + "model.layers.20.mlp.experts.118.down_proj.weight": "model-00021-of-00101.safetensors", + "model.layers.20.mlp.experts.119.gate_proj.weight": "model-00021-of-00101.safetensors", + "model.layers.20.mlp.experts.119.up_proj.weight": "model-00021-of-00101.safetensors", + "model.layers.20.mlp.experts.119.down_proj.weight": "model-00021-of-00101.safetensors", + "model.layers.20.mlp.gate.weight": "model-00021-of-00101.safetensors", + "model.layers.20.mlp.gate.e_score_correction_bias": "model-00021-of-00101.safetensors", + "model.layers.20.mlp.shared_experts.gate_proj.weight": "model-00021-of-00101.safetensors", + "model.layers.20.mlp.shared_experts.up_proj.weight": "model-00021-of-00101.safetensors", + "model.layers.20.mlp.shared_experts.down_proj.weight": "model-00021-of-00101.safetensors", + "model.layers.20.input_layernorm.weight": "model-00021-of-00101.safetensors", + "model.layers.20.post_attention_layernorm.weight": "model-00021-of-00101.safetensors", + "model.layers.21.self_attn.q_proj.weight": "model-00021-of-00101.safetensors", + "model.layers.21.self_attn.q_proj.bias": "model-00021-of-00101.safetensors", + "model.layers.21.self_attn.k_proj.weight": "model-00021-of-00101.safetensors", + "model.layers.21.self_attn.k_proj.bias": "model-00021-of-00101.safetensors", + "model.layers.21.self_attn.v_proj.weight": "model-00021-of-00101.safetensors", + "model.layers.21.self_attn.v_proj.bias": "model-00021-of-00101.safetensors", + "model.layers.21.self_attn.o_proj.weight": "model-00021-of-00101.safetensors", + "model.layers.21.self_attn.q_norm.weight": "model-00021-of-00101.safetensors", + "model.layers.21.self_attn.k_norm.weight": "model-00021-of-00101.safetensors", + "model.layers.21.mlp.experts.0.gate_proj.weight": "model-00021-of-00101.safetensors", + "model.layers.21.mlp.experts.0.up_proj.weight": "model-00021-of-00101.safetensors", + "model.layers.21.mlp.experts.0.down_proj.weight": "model-00021-of-00101.safetensors", + "model.layers.21.mlp.experts.1.gate_proj.weight": "model-00021-of-00101.safetensors", + "model.layers.21.mlp.experts.1.up_proj.weight": "model-00021-of-00101.safetensors", + "model.layers.21.mlp.experts.1.down_proj.weight": "model-00021-of-00101.safetensors", + "model.layers.21.mlp.experts.2.gate_proj.weight": "model-00021-of-00101.safetensors", + "model.layers.21.mlp.experts.2.up_proj.weight": "model-00021-of-00101.safetensors", + "model.layers.21.mlp.experts.2.down_proj.weight": "model-00021-of-00101.safetensors", + "model.layers.21.mlp.experts.3.gate_proj.weight": "model-00021-of-00101.safetensors", + "model.layers.21.mlp.experts.3.up_proj.weight": "model-00021-of-00101.safetensors", + "model.layers.21.mlp.experts.3.down_proj.weight": "model-00021-of-00101.safetensors", + "model.layers.21.mlp.experts.4.gate_proj.weight": "model-00021-of-00101.safetensors", + "model.layers.21.mlp.experts.4.up_proj.weight": "model-00021-of-00101.safetensors", + "model.layers.21.mlp.experts.4.down_proj.weight": "model-00021-of-00101.safetensors", + "model.layers.21.mlp.experts.5.gate_proj.weight": "model-00021-of-00101.safetensors", + "model.layers.21.mlp.experts.5.up_proj.weight": "model-00021-of-00101.safetensors", + "model.layers.21.mlp.experts.5.down_proj.weight": "model-00021-of-00101.safetensors", + "model.layers.21.mlp.experts.6.gate_proj.weight": "model-00021-of-00101.safetensors", + "model.layers.21.mlp.experts.6.up_proj.weight": "model-00021-of-00101.safetensors", + "model.layers.21.mlp.experts.6.down_proj.weight": "model-00021-of-00101.safetensors", + "model.layers.21.mlp.experts.7.gate_proj.weight": "model-00021-of-00101.safetensors", + "model.layers.21.mlp.experts.7.up_proj.weight": "model-00021-of-00101.safetensors", + "model.layers.21.mlp.experts.7.down_proj.weight": "model-00021-of-00101.safetensors", + "model.layers.21.mlp.experts.8.gate_proj.weight": "model-00021-of-00101.safetensors", + "model.layers.21.mlp.experts.8.up_proj.weight": "model-00021-of-00101.safetensors", + "model.layers.21.mlp.experts.8.down_proj.weight": "model-00021-of-00101.safetensors", + "model.layers.21.mlp.experts.9.gate_proj.weight": "model-00021-of-00101.safetensors", + "model.layers.21.mlp.experts.9.up_proj.weight": "model-00021-of-00101.safetensors", + "model.layers.21.mlp.experts.9.down_proj.weight": "model-00021-of-00101.safetensors", + "model.layers.21.mlp.experts.10.gate_proj.weight": "model-00021-of-00101.safetensors", + "model.layers.21.mlp.experts.10.up_proj.weight": "model-00021-of-00101.safetensors", + "model.layers.21.mlp.experts.10.down_proj.weight": "model-00021-of-00101.safetensors", + "model.layers.21.mlp.experts.11.gate_proj.weight": "model-00021-of-00101.safetensors", + "model.layers.21.mlp.experts.11.up_proj.weight": "model-00021-of-00101.safetensors", + "model.layers.21.mlp.experts.11.down_proj.weight": "model-00021-of-00101.safetensors", + "model.layers.21.mlp.experts.12.gate_proj.weight": "model-00021-of-00101.safetensors", + "model.layers.21.mlp.experts.12.up_proj.weight": "model-00021-of-00101.safetensors", + "model.layers.21.mlp.experts.12.down_proj.weight": "model-00021-of-00101.safetensors", + "model.layers.21.mlp.experts.13.gate_proj.weight": "model-00021-of-00101.safetensors", + "model.layers.21.mlp.experts.13.up_proj.weight": "model-00021-of-00101.safetensors", + "model.layers.21.mlp.experts.13.down_proj.weight": "model-00021-of-00101.safetensors", + "model.layers.21.mlp.experts.14.gate_proj.weight": "model-00021-of-00101.safetensors", + "model.layers.21.mlp.experts.14.up_proj.weight": "model-00021-of-00101.safetensors", + "model.layers.21.mlp.experts.14.down_proj.weight": "model-00021-of-00101.safetensors", + "model.layers.21.mlp.experts.15.gate_proj.weight": "model-00021-of-00101.safetensors", + "model.layers.21.mlp.experts.15.up_proj.weight": "model-00021-of-00101.safetensors", + "model.layers.21.mlp.experts.15.down_proj.weight": "model-00021-of-00101.safetensors", + "model.layers.21.mlp.experts.16.gate_proj.weight": "model-00021-of-00101.safetensors", + "model.layers.21.mlp.experts.16.up_proj.weight": "model-00021-of-00101.safetensors", + "model.layers.21.mlp.experts.16.down_proj.weight": "model-00021-of-00101.safetensors", + "model.layers.21.mlp.experts.17.gate_proj.weight": "model-00021-of-00101.safetensors", + "model.layers.21.mlp.experts.17.up_proj.weight": "model-00021-of-00101.safetensors", + "model.layers.21.mlp.experts.17.down_proj.weight": "model-00021-of-00101.safetensors", + "model.layers.21.mlp.experts.18.gate_proj.weight": "model-00021-of-00101.safetensors", + "model.layers.21.mlp.experts.18.up_proj.weight": "model-00021-of-00101.safetensors", + "model.layers.21.mlp.experts.18.down_proj.weight": "model-00021-of-00101.safetensors", + "model.layers.21.mlp.experts.19.gate_proj.weight": "model-00021-of-00101.safetensors", + "model.layers.21.mlp.experts.19.up_proj.weight": "model-00021-of-00101.safetensors", + "model.layers.21.mlp.experts.19.down_proj.weight": "model-00021-of-00101.safetensors", + "model.layers.21.mlp.experts.20.gate_proj.weight": "model-00021-of-00101.safetensors", + "model.layers.21.mlp.experts.20.up_proj.weight": "model-00021-of-00101.safetensors", + "model.layers.21.mlp.experts.20.down_proj.weight": "model-00021-of-00101.safetensors", + "model.layers.21.mlp.experts.21.gate_proj.weight": "model-00022-of-00101.safetensors", + "model.layers.21.mlp.experts.21.up_proj.weight": "model-00022-of-00101.safetensors", + "model.layers.21.mlp.experts.21.down_proj.weight": "model-00022-of-00101.safetensors", + "model.layers.21.mlp.experts.22.gate_proj.weight": "model-00022-of-00101.safetensors", + "model.layers.21.mlp.experts.22.up_proj.weight": "model-00022-of-00101.safetensors", + "model.layers.21.mlp.experts.22.down_proj.weight": "model-00022-of-00101.safetensors", + "model.layers.21.mlp.experts.23.gate_proj.weight": "model-00022-of-00101.safetensors", + "model.layers.21.mlp.experts.23.up_proj.weight": "model-00022-of-00101.safetensors", + "model.layers.21.mlp.experts.23.down_proj.weight": "model-00022-of-00101.safetensors", + "model.layers.21.mlp.experts.24.gate_proj.weight": "model-00022-of-00101.safetensors", + "model.layers.21.mlp.experts.24.up_proj.weight": "model-00022-of-00101.safetensors", + "model.layers.21.mlp.experts.24.down_proj.weight": "model-00022-of-00101.safetensors", + "model.layers.21.mlp.experts.25.gate_proj.weight": "model-00022-of-00101.safetensors", + "model.layers.21.mlp.experts.25.up_proj.weight": "model-00022-of-00101.safetensors", + "model.layers.21.mlp.experts.25.down_proj.weight": "model-00022-of-00101.safetensors", + "model.layers.21.mlp.experts.26.gate_proj.weight": "model-00022-of-00101.safetensors", + "model.layers.21.mlp.experts.26.up_proj.weight": "model-00022-of-00101.safetensors", + "model.layers.21.mlp.experts.26.down_proj.weight": "model-00022-of-00101.safetensors", + "model.layers.21.mlp.experts.27.gate_proj.weight": "model-00022-of-00101.safetensors", + "model.layers.21.mlp.experts.27.up_proj.weight": "model-00022-of-00101.safetensors", + "model.layers.21.mlp.experts.27.down_proj.weight": "model-00022-of-00101.safetensors", + "model.layers.21.mlp.experts.28.gate_proj.weight": "model-00022-of-00101.safetensors", + "model.layers.21.mlp.experts.28.up_proj.weight": "model-00022-of-00101.safetensors", + "model.layers.21.mlp.experts.28.down_proj.weight": "model-00022-of-00101.safetensors", + "model.layers.21.mlp.experts.29.gate_proj.weight": "model-00022-of-00101.safetensors", + "model.layers.21.mlp.experts.29.up_proj.weight": "model-00022-of-00101.safetensors", + "model.layers.21.mlp.experts.29.down_proj.weight": "model-00022-of-00101.safetensors", + "model.layers.21.mlp.experts.30.gate_proj.weight": "model-00022-of-00101.safetensors", + "model.layers.21.mlp.experts.30.up_proj.weight": "model-00022-of-00101.safetensors", + "model.layers.21.mlp.experts.30.down_proj.weight": "model-00022-of-00101.safetensors", + "model.layers.21.mlp.experts.31.gate_proj.weight": "model-00022-of-00101.safetensors", + "model.layers.21.mlp.experts.31.up_proj.weight": "model-00022-of-00101.safetensors", + "model.layers.21.mlp.experts.31.down_proj.weight": "model-00022-of-00101.safetensors", + "model.layers.21.mlp.experts.32.gate_proj.weight": "model-00022-of-00101.safetensors", + "model.layers.21.mlp.experts.32.up_proj.weight": "model-00022-of-00101.safetensors", + "model.layers.21.mlp.experts.32.down_proj.weight": "model-00022-of-00101.safetensors", + "model.layers.21.mlp.experts.33.gate_proj.weight": "model-00022-of-00101.safetensors", + "model.layers.21.mlp.experts.33.up_proj.weight": "model-00022-of-00101.safetensors", + "model.layers.21.mlp.experts.33.down_proj.weight": "model-00022-of-00101.safetensors", + "model.layers.21.mlp.experts.34.gate_proj.weight": "model-00022-of-00101.safetensors", + "model.layers.21.mlp.experts.34.up_proj.weight": "model-00022-of-00101.safetensors", + "model.layers.21.mlp.experts.34.down_proj.weight": "model-00022-of-00101.safetensors", + "model.layers.21.mlp.experts.35.gate_proj.weight": "model-00022-of-00101.safetensors", + "model.layers.21.mlp.experts.35.up_proj.weight": "model-00022-of-00101.safetensors", + "model.layers.21.mlp.experts.35.down_proj.weight": "model-00022-of-00101.safetensors", + "model.layers.21.mlp.experts.36.gate_proj.weight": "model-00022-of-00101.safetensors", + "model.layers.21.mlp.experts.36.up_proj.weight": "model-00022-of-00101.safetensors", + "model.layers.21.mlp.experts.36.down_proj.weight": "model-00022-of-00101.safetensors", + "model.layers.21.mlp.experts.37.gate_proj.weight": "model-00022-of-00101.safetensors", + "model.layers.21.mlp.experts.37.up_proj.weight": "model-00022-of-00101.safetensors", + "model.layers.21.mlp.experts.37.down_proj.weight": "model-00022-of-00101.safetensors", + "model.layers.21.mlp.experts.38.gate_proj.weight": "model-00022-of-00101.safetensors", + "model.layers.21.mlp.experts.38.up_proj.weight": "model-00022-of-00101.safetensors", + "model.layers.21.mlp.experts.38.down_proj.weight": "model-00022-of-00101.safetensors", + "model.layers.21.mlp.experts.39.gate_proj.weight": "model-00022-of-00101.safetensors", + "model.layers.21.mlp.experts.39.up_proj.weight": "model-00022-of-00101.safetensors", + "model.layers.21.mlp.experts.39.down_proj.weight": "model-00022-of-00101.safetensors", + "model.layers.21.mlp.experts.40.gate_proj.weight": "model-00022-of-00101.safetensors", + "model.layers.21.mlp.experts.40.up_proj.weight": "model-00022-of-00101.safetensors", + "model.layers.21.mlp.experts.40.down_proj.weight": "model-00022-of-00101.safetensors", + "model.layers.21.mlp.experts.41.gate_proj.weight": "model-00022-of-00101.safetensors", + "model.layers.21.mlp.experts.41.up_proj.weight": "model-00022-of-00101.safetensors", + "model.layers.21.mlp.experts.41.down_proj.weight": "model-00022-of-00101.safetensors", + "model.layers.21.mlp.experts.42.gate_proj.weight": "model-00022-of-00101.safetensors", + "model.layers.21.mlp.experts.42.up_proj.weight": "model-00022-of-00101.safetensors", + "model.layers.21.mlp.experts.42.down_proj.weight": "model-00022-of-00101.safetensors", + "model.layers.21.mlp.experts.43.gate_proj.weight": "model-00022-of-00101.safetensors", + "model.layers.21.mlp.experts.43.up_proj.weight": "model-00022-of-00101.safetensors", + "model.layers.21.mlp.experts.43.down_proj.weight": "model-00022-of-00101.safetensors", + "model.layers.21.mlp.experts.44.gate_proj.weight": "model-00022-of-00101.safetensors", + "model.layers.21.mlp.experts.44.up_proj.weight": "model-00022-of-00101.safetensors", + "model.layers.21.mlp.experts.44.down_proj.weight": "model-00022-of-00101.safetensors", + "model.layers.21.mlp.experts.45.gate_proj.weight": "model-00022-of-00101.safetensors", + "model.layers.21.mlp.experts.45.up_proj.weight": "model-00022-of-00101.safetensors", + "model.layers.21.mlp.experts.45.down_proj.weight": "model-00022-of-00101.safetensors", + "model.layers.21.mlp.experts.46.gate_proj.weight": "model-00022-of-00101.safetensors", + "model.layers.21.mlp.experts.46.up_proj.weight": "model-00022-of-00101.safetensors", + "model.layers.21.mlp.experts.46.down_proj.weight": "model-00022-of-00101.safetensors", + "model.layers.21.mlp.experts.47.gate_proj.weight": "model-00022-of-00101.safetensors", + "model.layers.21.mlp.experts.47.up_proj.weight": "model-00022-of-00101.safetensors", + "model.layers.21.mlp.experts.47.down_proj.weight": "model-00022-of-00101.safetensors", + "model.layers.21.mlp.experts.48.gate_proj.weight": "model-00022-of-00101.safetensors", + "model.layers.21.mlp.experts.48.up_proj.weight": "model-00022-of-00101.safetensors", + "model.layers.21.mlp.experts.48.down_proj.weight": "model-00022-of-00101.safetensors", + "model.layers.21.mlp.experts.49.gate_proj.weight": "model-00022-of-00101.safetensors", + "model.layers.21.mlp.experts.49.up_proj.weight": "model-00022-of-00101.safetensors", + "model.layers.21.mlp.experts.49.down_proj.weight": "model-00022-of-00101.safetensors", + "model.layers.21.mlp.experts.50.gate_proj.weight": "model-00022-of-00101.safetensors", + "model.layers.21.mlp.experts.50.up_proj.weight": "model-00022-of-00101.safetensors", + "model.layers.21.mlp.experts.50.down_proj.weight": "model-00022-of-00101.safetensors", + "model.layers.21.mlp.experts.51.gate_proj.weight": "model-00022-of-00101.safetensors", + "model.layers.21.mlp.experts.51.up_proj.weight": "model-00022-of-00101.safetensors", + "model.layers.21.mlp.experts.51.down_proj.weight": "model-00022-of-00101.safetensors", + "model.layers.21.mlp.experts.52.gate_proj.weight": "model-00022-of-00101.safetensors", + "model.layers.21.mlp.experts.52.up_proj.weight": "model-00022-of-00101.safetensors", + "model.layers.21.mlp.experts.52.down_proj.weight": "model-00022-of-00101.safetensors", + "model.layers.21.mlp.experts.53.gate_proj.weight": "model-00022-of-00101.safetensors", + "model.layers.21.mlp.experts.53.up_proj.weight": "model-00022-of-00101.safetensors", + "model.layers.21.mlp.experts.53.down_proj.weight": "model-00022-of-00101.safetensors", + "model.layers.21.mlp.experts.54.gate_proj.weight": "model-00022-of-00101.safetensors", + "model.layers.21.mlp.experts.54.up_proj.weight": "model-00022-of-00101.safetensors", + "model.layers.21.mlp.experts.54.down_proj.weight": "model-00022-of-00101.safetensors", + "model.layers.21.mlp.experts.55.gate_proj.weight": "model-00022-of-00101.safetensors", + "model.layers.21.mlp.experts.55.up_proj.weight": "model-00022-of-00101.safetensors", + "model.layers.21.mlp.experts.55.down_proj.weight": "model-00022-of-00101.safetensors", + "model.layers.21.mlp.experts.56.gate_proj.weight": "model-00022-of-00101.safetensors", + "model.layers.21.mlp.experts.56.up_proj.weight": "model-00022-of-00101.safetensors", + "model.layers.21.mlp.experts.56.down_proj.weight": "model-00022-of-00101.safetensors", + "model.layers.21.mlp.experts.57.gate_proj.weight": "model-00022-of-00101.safetensors", + "model.layers.21.mlp.experts.57.up_proj.weight": "model-00022-of-00101.safetensors", + "model.layers.21.mlp.experts.57.down_proj.weight": "model-00022-of-00101.safetensors", + "model.layers.21.mlp.experts.58.gate_proj.weight": "model-00022-of-00101.safetensors", + "model.layers.21.mlp.experts.58.up_proj.weight": "model-00022-of-00101.safetensors", + "model.layers.21.mlp.experts.58.down_proj.weight": "model-00022-of-00101.safetensors", + "model.layers.21.mlp.experts.59.gate_proj.weight": "model-00022-of-00101.safetensors", + "model.layers.21.mlp.experts.59.up_proj.weight": "model-00022-of-00101.safetensors", + "model.layers.21.mlp.experts.59.down_proj.weight": "model-00022-of-00101.safetensors", + "model.layers.21.mlp.experts.60.gate_proj.weight": "model-00022-of-00101.safetensors", + "model.layers.21.mlp.experts.60.up_proj.weight": "model-00022-of-00101.safetensors", + "model.layers.21.mlp.experts.60.down_proj.weight": "model-00022-of-00101.safetensors", + "model.layers.21.mlp.experts.61.gate_proj.weight": "model-00022-of-00101.safetensors", + "model.layers.21.mlp.experts.61.up_proj.weight": "model-00022-of-00101.safetensors", + "model.layers.21.mlp.experts.61.down_proj.weight": "model-00022-of-00101.safetensors", + "model.layers.21.mlp.experts.62.gate_proj.weight": "model-00022-of-00101.safetensors", + "model.layers.21.mlp.experts.62.up_proj.weight": "model-00022-of-00101.safetensors", + "model.layers.21.mlp.experts.62.down_proj.weight": "model-00022-of-00101.safetensors", + "model.layers.21.mlp.experts.63.gate_proj.weight": "model-00022-of-00101.safetensors", + "model.layers.21.mlp.experts.63.up_proj.weight": "model-00022-of-00101.safetensors", + "model.layers.21.mlp.experts.63.down_proj.weight": "model-00022-of-00101.safetensors", + "model.layers.21.mlp.experts.64.gate_proj.weight": "model-00022-of-00101.safetensors", + "model.layers.21.mlp.experts.64.up_proj.weight": "model-00022-of-00101.safetensors", + "model.layers.21.mlp.experts.64.down_proj.weight": "model-00022-of-00101.safetensors", + "model.layers.21.mlp.experts.65.gate_proj.weight": "model-00022-of-00101.safetensors", + "model.layers.21.mlp.experts.65.up_proj.weight": "model-00022-of-00101.safetensors", + "model.layers.21.mlp.experts.65.down_proj.weight": "model-00022-of-00101.safetensors", + "model.layers.21.mlp.experts.66.gate_proj.weight": "model-00022-of-00101.safetensors", + "model.layers.21.mlp.experts.66.up_proj.weight": "model-00022-of-00101.safetensors", + "model.layers.21.mlp.experts.66.down_proj.weight": "model-00022-of-00101.safetensors", + "model.layers.21.mlp.experts.67.gate_proj.weight": "model-00022-of-00101.safetensors", + "model.layers.21.mlp.experts.67.up_proj.weight": "model-00022-of-00101.safetensors", + "model.layers.21.mlp.experts.67.down_proj.weight": "model-00022-of-00101.safetensors", + "model.layers.21.mlp.experts.68.gate_proj.weight": "model-00022-of-00101.safetensors", + "model.layers.21.mlp.experts.68.up_proj.weight": "model-00022-of-00101.safetensors", + "model.layers.21.mlp.experts.68.down_proj.weight": "model-00022-of-00101.safetensors", + "model.layers.21.mlp.experts.69.gate_proj.weight": "model-00022-of-00101.safetensors", + "model.layers.21.mlp.experts.69.up_proj.weight": "model-00022-of-00101.safetensors", + "model.layers.21.mlp.experts.69.down_proj.weight": "model-00022-of-00101.safetensors", + "model.layers.21.mlp.experts.70.gate_proj.weight": "model-00022-of-00101.safetensors", + "model.layers.21.mlp.experts.70.up_proj.weight": "model-00022-of-00101.safetensors", + "model.layers.21.mlp.experts.70.down_proj.weight": "model-00022-of-00101.safetensors", + "model.layers.21.mlp.experts.71.gate_proj.weight": "model-00022-of-00101.safetensors", + "model.layers.21.mlp.experts.71.up_proj.weight": "model-00022-of-00101.safetensors", + "model.layers.21.mlp.experts.71.down_proj.weight": "model-00022-of-00101.safetensors", + "model.layers.21.mlp.experts.72.gate_proj.weight": "model-00022-of-00101.safetensors", + "model.layers.21.mlp.experts.72.up_proj.weight": "model-00022-of-00101.safetensors", + "model.layers.21.mlp.experts.72.down_proj.weight": "model-00022-of-00101.safetensors", + "model.layers.21.mlp.experts.73.gate_proj.weight": "model-00022-of-00101.safetensors", + "model.layers.21.mlp.experts.73.up_proj.weight": "model-00022-of-00101.safetensors", + "model.layers.21.mlp.experts.73.down_proj.weight": "model-00022-of-00101.safetensors", + "model.layers.21.mlp.experts.74.gate_proj.weight": "model-00022-of-00101.safetensors", + "model.layers.21.mlp.experts.74.up_proj.weight": "model-00022-of-00101.safetensors", + "model.layers.21.mlp.experts.74.down_proj.weight": "model-00022-of-00101.safetensors", + "model.layers.21.mlp.experts.75.gate_proj.weight": "model-00022-of-00101.safetensors", + "model.layers.21.mlp.experts.75.up_proj.weight": "model-00022-of-00101.safetensors", + "model.layers.21.mlp.experts.75.down_proj.weight": "model-00022-of-00101.safetensors", + "model.layers.21.mlp.experts.76.gate_proj.weight": "model-00022-of-00101.safetensors", + "model.layers.21.mlp.experts.76.up_proj.weight": "model-00022-of-00101.safetensors", + "model.layers.21.mlp.experts.76.down_proj.weight": "model-00022-of-00101.safetensors", + "model.layers.21.mlp.experts.77.gate_proj.weight": "model-00022-of-00101.safetensors", + "model.layers.21.mlp.experts.77.up_proj.weight": "model-00022-of-00101.safetensors", + "model.layers.21.mlp.experts.77.down_proj.weight": "model-00022-of-00101.safetensors", + "model.layers.21.mlp.experts.78.gate_proj.weight": "model-00022-of-00101.safetensors", + "model.layers.21.mlp.experts.78.up_proj.weight": "model-00022-of-00101.safetensors", + "model.layers.21.mlp.experts.78.down_proj.weight": "model-00022-of-00101.safetensors", + "model.layers.21.mlp.experts.79.gate_proj.weight": "model-00022-of-00101.safetensors", + "model.layers.21.mlp.experts.79.up_proj.weight": "model-00022-of-00101.safetensors", + "model.layers.21.mlp.experts.79.down_proj.weight": "model-00022-of-00101.safetensors", + "model.layers.21.mlp.experts.80.gate_proj.weight": "model-00022-of-00101.safetensors", + "model.layers.21.mlp.experts.80.up_proj.weight": "model-00022-of-00101.safetensors", + "model.layers.21.mlp.experts.80.down_proj.weight": "model-00022-of-00101.safetensors", + "model.layers.21.mlp.experts.81.gate_proj.weight": "model-00022-of-00101.safetensors", + "model.layers.21.mlp.experts.81.up_proj.weight": "model-00022-of-00101.safetensors", + "model.layers.21.mlp.experts.81.down_proj.weight": "model-00022-of-00101.safetensors", + "model.layers.21.mlp.experts.82.gate_proj.weight": "model-00022-of-00101.safetensors", + "model.layers.21.mlp.experts.82.up_proj.weight": "model-00022-of-00101.safetensors", + "model.layers.21.mlp.experts.82.down_proj.weight": "model-00022-of-00101.safetensors", + "model.layers.21.mlp.experts.83.gate_proj.weight": "model-00022-of-00101.safetensors", + "model.layers.21.mlp.experts.83.up_proj.weight": "model-00022-of-00101.safetensors", + "model.layers.21.mlp.experts.83.down_proj.weight": "model-00022-of-00101.safetensors", + "model.layers.21.mlp.experts.84.gate_proj.weight": "model-00022-of-00101.safetensors", + "model.layers.21.mlp.experts.84.up_proj.weight": "model-00022-of-00101.safetensors", + "model.layers.21.mlp.experts.84.down_proj.weight": "model-00022-of-00101.safetensors", + "model.layers.21.mlp.experts.85.gate_proj.weight": "model-00022-of-00101.safetensors", + "model.layers.21.mlp.experts.85.up_proj.weight": "model-00022-of-00101.safetensors", + "model.layers.21.mlp.experts.85.down_proj.weight": "model-00022-of-00101.safetensors", + "model.layers.21.mlp.experts.86.gate_proj.weight": "model-00022-of-00101.safetensors", + "model.layers.21.mlp.experts.86.up_proj.weight": "model-00022-of-00101.safetensors", + "model.layers.21.mlp.experts.86.down_proj.weight": "model-00022-of-00101.safetensors", + "model.layers.21.mlp.experts.87.gate_proj.weight": "model-00022-of-00101.safetensors", + "model.layers.21.mlp.experts.87.up_proj.weight": "model-00022-of-00101.safetensors", + "model.layers.21.mlp.experts.87.down_proj.weight": "model-00022-of-00101.safetensors", + "model.layers.21.mlp.experts.88.gate_proj.weight": "model-00022-of-00101.safetensors", + "model.layers.21.mlp.experts.88.up_proj.weight": "model-00022-of-00101.safetensors", + "model.layers.21.mlp.experts.88.down_proj.weight": "model-00022-of-00101.safetensors", + "model.layers.21.mlp.experts.89.gate_proj.weight": "model-00022-of-00101.safetensors", + "model.layers.21.mlp.experts.89.up_proj.weight": "model-00022-of-00101.safetensors", + "model.layers.21.mlp.experts.89.down_proj.weight": "model-00022-of-00101.safetensors", + "model.layers.21.mlp.experts.90.gate_proj.weight": "model-00022-of-00101.safetensors", + "model.layers.21.mlp.experts.90.up_proj.weight": "model-00022-of-00101.safetensors", + "model.layers.21.mlp.experts.90.down_proj.weight": "model-00022-of-00101.safetensors", + "model.layers.21.mlp.experts.91.gate_proj.weight": "model-00022-of-00101.safetensors", + "model.layers.21.mlp.experts.91.up_proj.weight": "model-00022-of-00101.safetensors", + "model.layers.21.mlp.experts.91.down_proj.weight": "model-00022-of-00101.safetensors", + "model.layers.21.mlp.experts.92.gate_proj.weight": "model-00022-of-00101.safetensors", + "model.layers.21.mlp.experts.92.up_proj.weight": "model-00022-of-00101.safetensors", + "model.layers.21.mlp.experts.92.down_proj.weight": "model-00022-of-00101.safetensors", + "model.layers.21.mlp.experts.93.gate_proj.weight": "model-00022-of-00101.safetensors", + "model.layers.21.mlp.experts.93.up_proj.weight": "model-00022-of-00101.safetensors", + "model.layers.21.mlp.experts.93.down_proj.weight": "model-00022-of-00101.safetensors", + "model.layers.21.mlp.experts.94.gate_proj.weight": "model-00022-of-00101.safetensors", + "model.layers.21.mlp.experts.94.up_proj.weight": "model-00022-of-00101.safetensors", + "model.layers.21.mlp.experts.94.down_proj.weight": "model-00022-of-00101.safetensors", + "model.layers.21.mlp.experts.95.gate_proj.weight": "model-00022-of-00101.safetensors", + "model.layers.21.mlp.experts.95.up_proj.weight": "model-00022-of-00101.safetensors", + "model.layers.21.mlp.experts.95.down_proj.weight": "model-00022-of-00101.safetensors", + "model.layers.21.mlp.experts.96.gate_proj.weight": "model-00022-of-00101.safetensors", + "model.layers.21.mlp.experts.96.up_proj.weight": "model-00022-of-00101.safetensors", + "model.layers.21.mlp.experts.96.down_proj.weight": "model-00022-of-00101.safetensors", + "model.layers.21.mlp.experts.97.gate_proj.weight": "model-00022-of-00101.safetensors", + "model.layers.21.mlp.experts.97.up_proj.weight": "model-00022-of-00101.safetensors", + "model.layers.21.mlp.experts.97.down_proj.weight": "model-00022-of-00101.safetensors", + "model.layers.21.mlp.experts.98.gate_proj.weight": "model-00022-of-00101.safetensors", + "model.layers.21.mlp.experts.98.up_proj.weight": "model-00022-of-00101.safetensors", + "model.layers.21.mlp.experts.98.down_proj.weight": "model-00022-of-00101.safetensors", + "model.layers.21.mlp.experts.99.gate_proj.weight": "model-00022-of-00101.safetensors", + "model.layers.21.mlp.experts.99.up_proj.weight": "model-00022-of-00101.safetensors", + "model.layers.21.mlp.experts.99.down_proj.weight": "model-00022-of-00101.safetensors", + "model.layers.21.mlp.experts.100.gate_proj.weight": "model-00022-of-00101.safetensors", + "model.layers.21.mlp.experts.100.up_proj.weight": "model-00022-of-00101.safetensors", + "model.layers.21.mlp.experts.100.down_proj.weight": "model-00022-of-00101.safetensors", + "model.layers.21.mlp.experts.101.gate_proj.weight": "model-00022-of-00101.safetensors", + "model.layers.21.mlp.experts.101.up_proj.weight": "model-00022-of-00101.safetensors", + "model.layers.21.mlp.experts.101.down_proj.weight": "model-00022-of-00101.safetensors", + "model.layers.21.mlp.experts.102.gate_proj.weight": "model-00022-of-00101.safetensors", + "model.layers.21.mlp.experts.102.up_proj.weight": "model-00022-of-00101.safetensors", + "model.layers.21.mlp.experts.102.down_proj.weight": "model-00022-of-00101.safetensors", + "model.layers.21.mlp.experts.103.gate_proj.weight": "model-00022-of-00101.safetensors", + "model.layers.21.mlp.experts.103.up_proj.weight": "model-00022-of-00101.safetensors", + "model.layers.21.mlp.experts.103.down_proj.weight": "model-00022-of-00101.safetensors", + "model.layers.21.mlp.experts.104.gate_proj.weight": "model-00022-of-00101.safetensors", + "model.layers.21.mlp.experts.104.up_proj.weight": "model-00022-of-00101.safetensors", + "model.layers.21.mlp.experts.104.down_proj.weight": "model-00022-of-00101.safetensors", + "model.layers.21.mlp.experts.105.gate_proj.weight": "model-00022-of-00101.safetensors", + "model.layers.21.mlp.experts.105.up_proj.weight": "model-00022-of-00101.safetensors", + "model.layers.21.mlp.experts.105.down_proj.weight": "model-00022-of-00101.safetensors", + "model.layers.21.mlp.experts.106.gate_proj.weight": "model-00022-of-00101.safetensors", + "model.layers.21.mlp.experts.106.up_proj.weight": "model-00022-of-00101.safetensors", + "model.layers.21.mlp.experts.106.down_proj.weight": "model-00022-of-00101.safetensors", + "model.layers.21.mlp.experts.107.gate_proj.weight": "model-00022-of-00101.safetensors", + "model.layers.21.mlp.experts.107.up_proj.weight": "model-00022-of-00101.safetensors", + "model.layers.21.mlp.experts.107.down_proj.weight": "model-00022-of-00101.safetensors", + "model.layers.21.mlp.experts.108.gate_proj.weight": "model-00022-of-00101.safetensors", + "model.layers.21.mlp.experts.108.up_proj.weight": "model-00022-of-00101.safetensors", + "model.layers.21.mlp.experts.108.down_proj.weight": "model-00022-of-00101.safetensors", + "model.layers.21.mlp.experts.109.gate_proj.weight": "model-00022-of-00101.safetensors", + "model.layers.21.mlp.experts.109.up_proj.weight": "model-00022-of-00101.safetensors", + "model.layers.21.mlp.experts.109.down_proj.weight": "model-00022-of-00101.safetensors", + "model.layers.21.mlp.experts.110.gate_proj.weight": "model-00022-of-00101.safetensors", + "model.layers.21.mlp.experts.110.up_proj.weight": "model-00022-of-00101.safetensors", + "model.layers.21.mlp.experts.110.down_proj.weight": "model-00022-of-00101.safetensors", + "model.layers.21.mlp.experts.111.gate_proj.weight": "model-00022-of-00101.safetensors", + "model.layers.21.mlp.experts.111.up_proj.weight": "model-00022-of-00101.safetensors", + "model.layers.21.mlp.experts.111.down_proj.weight": "model-00022-of-00101.safetensors", + "model.layers.21.mlp.experts.112.gate_proj.weight": "model-00022-of-00101.safetensors", + "model.layers.21.mlp.experts.112.up_proj.weight": "model-00022-of-00101.safetensors", + "model.layers.21.mlp.experts.112.down_proj.weight": "model-00022-of-00101.safetensors", + "model.layers.21.mlp.experts.113.gate_proj.weight": "model-00022-of-00101.safetensors", + "model.layers.21.mlp.experts.113.up_proj.weight": "model-00022-of-00101.safetensors", + "model.layers.21.mlp.experts.113.down_proj.weight": "model-00022-of-00101.safetensors", + "model.layers.21.mlp.experts.114.gate_proj.weight": "model-00022-of-00101.safetensors", + "model.layers.21.mlp.experts.114.up_proj.weight": "model-00022-of-00101.safetensors", + "model.layers.21.mlp.experts.114.down_proj.weight": "model-00022-of-00101.safetensors", + "model.layers.21.mlp.experts.115.gate_proj.weight": "model-00022-of-00101.safetensors", + "model.layers.21.mlp.experts.115.up_proj.weight": "model-00022-of-00101.safetensors", + "model.layers.21.mlp.experts.115.down_proj.weight": "model-00022-of-00101.safetensors", + "model.layers.21.mlp.experts.116.gate_proj.weight": "model-00022-of-00101.safetensors", + "model.layers.21.mlp.experts.116.up_proj.weight": "model-00022-of-00101.safetensors", + "model.layers.21.mlp.experts.116.down_proj.weight": "model-00022-of-00101.safetensors", + "model.layers.21.mlp.experts.117.gate_proj.weight": "model-00022-of-00101.safetensors", + "model.layers.21.mlp.experts.117.up_proj.weight": "model-00022-of-00101.safetensors", + "model.layers.21.mlp.experts.117.down_proj.weight": "model-00022-of-00101.safetensors", + "model.layers.21.mlp.experts.118.gate_proj.weight": "model-00022-of-00101.safetensors", + "model.layers.21.mlp.experts.118.up_proj.weight": "model-00022-of-00101.safetensors", + "model.layers.21.mlp.experts.118.down_proj.weight": "model-00022-of-00101.safetensors", + "model.layers.21.mlp.experts.119.gate_proj.weight": "model-00022-of-00101.safetensors", + "model.layers.21.mlp.experts.119.up_proj.weight": "model-00022-of-00101.safetensors", + "model.layers.21.mlp.experts.119.down_proj.weight": "model-00022-of-00101.safetensors", + "model.layers.21.mlp.gate.weight": "model-00022-of-00101.safetensors", + "model.layers.21.mlp.gate.e_score_correction_bias": "model-00022-of-00101.safetensors", + "model.layers.21.mlp.shared_experts.gate_proj.weight": "model-00022-of-00101.safetensors", + "model.layers.21.mlp.shared_experts.up_proj.weight": "model-00022-of-00101.safetensors", + "model.layers.21.mlp.shared_experts.down_proj.weight": "model-00022-of-00101.safetensors", + "model.layers.21.input_layernorm.weight": "model-00022-of-00101.safetensors", + "model.layers.21.post_attention_layernorm.weight": "model-00022-of-00101.safetensors", + "model.layers.22.self_attn.q_proj.weight": "model-00022-of-00101.safetensors", + "model.layers.22.self_attn.q_proj.bias": "model-00022-of-00101.safetensors", + "model.layers.22.self_attn.k_proj.weight": "model-00022-of-00101.safetensors", + "model.layers.22.self_attn.k_proj.bias": "model-00022-of-00101.safetensors", + "model.layers.22.self_attn.v_proj.weight": "model-00022-of-00101.safetensors", + "model.layers.22.self_attn.v_proj.bias": "model-00022-of-00101.safetensors", + "model.layers.22.self_attn.o_proj.weight": "model-00022-of-00101.safetensors", + "model.layers.22.self_attn.q_norm.weight": "model-00022-of-00101.safetensors", + "model.layers.22.self_attn.k_norm.weight": "model-00022-of-00101.safetensors", + "model.layers.22.mlp.experts.0.gate_proj.weight": "model-00022-of-00101.safetensors", + "model.layers.22.mlp.experts.0.up_proj.weight": "model-00022-of-00101.safetensors", + "model.layers.22.mlp.experts.0.down_proj.weight": "model-00022-of-00101.safetensors", + "model.layers.22.mlp.experts.1.gate_proj.weight": "model-00022-of-00101.safetensors", + "model.layers.22.mlp.experts.1.up_proj.weight": "model-00022-of-00101.safetensors", + "model.layers.22.mlp.experts.1.down_proj.weight": "model-00022-of-00101.safetensors", + "model.layers.22.mlp.experts.2.gate_proj.weight": "model-00022-of-00101.safetensors", + "model.layers.22.mlp.experts.2.up_proj.weight": "model-00022-of-00101.safetensors", + "model.layers.22.mlp.experts.2.down_proj.weight": "model-00022-of-00101.safetensors", + "model.layers.22.mlp.experts.3.gate_proj.weight": "model-00022-of-00101.safetensors", + "model.layers.22.mlp.experts.3.up_proj.weight": "model-00022-of-00101.safetensors", + "model.layers.22.mlp.experts.3.down_proj.weight": "model-00022-of-00101.safetensors", + "model.layers.22.mlp.experts.4.gate_proj.weight": "model-00022-of-00101.safetensors", + "model.layers.22.mlp.experts.4.up_proj.weight": "model-00022-of-00101.safetensors", + "model.layers.22.mlp.experts.4.down_proj.weight": "model-00022-of-00101.safetensors", + "model.layers.22.mlp.experts.5.gate_proj.weight": "model-00022-of-00101.safetensors", + "model.layers.22.mlp.experts.5.up_proj.weight": "model-00022-of-00101.safetensors", + "model.layers.22.mlp.experts.5.down_proj.weight": "model-00022-of-00101.safetensors", + "model.layers.22.mlp.experts.6.gate_proj.weight": "model-00022-of-00101.safetensors", + "model.layers.22.mlp.experts.6.up_proj.weight": "model-00022-of-00101.safetensors", + "model.layers.22.mlp.experts.6.down_proj.weight": "model-00022-of-00101.safetensors", + "model.layers.22.mlp.experts.7.gate_proj.weight": "model-00022-of-00101.safetensors", + "model.layers.22.mlp.experts.7.up_proj.weight": "model-00022-of-00101.safetensors", + "model.layers.22.mlp.experts.7.down_proj.weight": "model-00023-of-00101.safetensors", + "model.layers.22.mlp.experts.8.gate_proj.weight": "model-00023-of-00101.safetensors", + "model.layers.22.mlp.experts.8.up_proj.weight": "model-00023-of-00101.safetensors", + "model.layers.22.mlp.experts.8.down_proj.weight": "model-00023-of-00101.safetensors", + "model.layers.22.mlp.experts.9.gate_proj.weight": "model-00023-of-00101.safetensors", + "model.layers.22.mlp.experts.9.up_proj.weight": "model-00023-of-00101.safetensors", + "model.layers.22.mlp.experts.9.down_proj.weight": "model-00023-of-00101.safetensors", + "model.layers.22.mlp.experts.10.gate_proj.weight": "model-00023-of-00101.safetensors", + "model.layers.22.mlp.experts.10.up_proj.weight": "model-00023-of-00101.safetensors", + "model.layers.22.mlp.experts.10.down_proj.weight": "model-00023-of-00101.safetensors", + "model.layers.22.mlp.experts.11.gate_proj.weight": "model-00023-of-00101.safetensors", + "model.layers.22.mlp.experts.11.up_proj.weight": "model-00023-of-00101.safetensors", + "model.layers.22.mlp.experts.11.down_proj.weight": "model-00023-of-00101.safetensors", + "model.layers.22.mlp.experts.12.gate_proj.weight": "model-00023-of-00101.safetensors", + "model.layers.22.mlp.experts.12.up_proj.weight": "model-00023-of-00101.safetensors", + "model.layers.22.mlp.experts.12.down_proj.weight": "model-00023-of-00101.safetensors", + "model.layers.22.mlp.experts.13.gate_proj.weight": "model-00023-of-00101.safetensors", + "model.layers.22.mlp.experts.13.up_proj.weight": "model-00023-of-00101.safetensors", + "model.layers.22.mlp.experts.13.down_proj.weight": "model-00023-of-00101.safetensors", + "model.layers.22.mlp.experts.14.gate_proj.weight": "model-00023-of-00101.safetensors", + "model.layers.22.mlp.experts.14.up_proj.weight": "model-00023-of-00101.safetensors", + "model.layers.22.mlp.experts.14.down_proj.weight": "model-00023-of-00101.safetensors", + "model.layers.22.mlp.experts.15.gate_proj.weight": "model-00023-of-00101.safetensors", + "model.layers.22.mlp.experts.15.up_proj.weight": "model-00023-of-00101.safetensors", + "model.layers.22.mlp.experts.15.down_proj.weight": "model-00023-of-00101.safetensors", + "model.layers.22.mlp.experts.16.gate_proj.weight": "model-00023-of-00101.safetensors", + "model.layers.22.mlp.experts.16.up_proj.weight": "model-00023-of-00101.safetensors", + "model.layers.22.mlp.experts.16.down_proj.weight": "model-00023-of-00101.safetensors", + "model.layers.22.mlp.experts.17.gate_proj.weight": "model-00023-of-00101.safetensors", + "model.layers.22.mlp.experts.17.up_proj.weight": "model-00023-of-00101.safetensors", + "model.layers.22.mlp.experts.17.down_proj.weight": "model-00023-of-00101.safetensors", + "model.layers.22.mlp.experts.18.gate_proj.weight": "model-00023-of-00101.safetensors", + "model.layers.22.mlp.experts.18.up_proj.weight": "model-00023-of-00101.safetensors", + "model.layers.22.mlp.experts.18.down_proj.weight": "model-00023-of-00101.safetensors", + "model.layers.22.mlp.experts.19.gate_proj.weight": "model-00023-of-00101.safetensors", + "model.layers.22.mlp.experts.19.up_proj.weight": "model-00023-of-00101.safetensors", + "model.layers.22.mlp.experts.19.down_proj.weight": "model-00023-of-00101.safetensors", + "model.layers.22.mlp.experts.20.gate_proj.weight": "model-00023-of-00101.safetensors", + "model.layers.22.mlp.experts.20.up_proj.weight": "model-00023-of-00101.safetensors", + "model.layers.22.mlp.experts.20.down_proj.weight": "model-00023-of-00101.safetensors", + "model.layers.22.mlp.experts.21.gate_proj.weight": "model-00023-of-00101.safetensors", + "model.layers.22.mlp.experts.21.up_proj.weight": "model-00023-of-00101.safetensors", + "model.layers.22.mlp.experts.21.down_proj.weight": "model-00023-of-00101.safetensors", + "model.layers.22.mlp.experts.22.gate_proj.weight": "model-00023-of-00101.safetensors", + "model.layers.22.mlp.experts.22.up_proj.weight": "model-00023-of-00101.safetensors", + "model.layers.22.mlp.experts.22.down_proj.weight": "model-00023-of-00101.safetensors", + "model.layers.22.mlp.experts.23.gate_proj.weight": "model-00023-of-00101.safetensors", + "model.layers.22.mlp.experts.23.up_proj.weight": "model-00023-of-00101.safetensors", + "model.layers.22.mlp.experts.23.down_proj.weight": "model-00023-of-00101.safetensors", + "model.layers.22.mlp.experts.24.gate_proj.weight": "model-00023-of-00101.safetensors", + "model.layers.22.mlp.experts.24.up_proj.weight": "model-00023-of-00101.safetensors", + "model.layers.22.mlp.experts.24.down_proj.weight": "model-00023-of-00101.safetensors", + "model.layers.22.mlp.experts.25.gate_proj.weight": "model-00023-of-00101.safetensors", + "model.layers.22.mlp.experts.25.up_proj.weight": "model-00023-of-00101.safetensors", + "model.layers.22.mlp.experts.25.down_proj.weight": "model-00023-of-00101.safetensors", + "model.layers.22.mlp.experts.26.gate_proj.weight": "model-00023-of-00101.safetensors", + "model.layers.22.mlp.experts.26.up_proj.weight": "model-00023-of-00101.safetensors", + "model.layers.22.mlp.experts.26.down_proj.weight": "model-00023-of-00101.safetensors", + "model.layers.22.mlp.experts.27.gate_proj.weight": "model-00023-of-00101.safetensors", + "model.layers.22.mlp.experts.27.up_proj.weight": "model-00023-of-00101.safetensors", + "model.layers.22.mlp.experts.27.down_proj.weight": "model-00023-of-00101.safetensors", + "model.layers.22.mlp.experts.28.gate_proj.weight": "model-00023-of-00101.safetensors", + "model.layers.22.mlp.experts.28.up_proj.weight": "model-00023-of-00101.safetensors", + "model.layers.22.mlp.experts.28.down_proj.weight": "model-00023-of-00101.safetensors", + "model.layers.22.mlp.experts.29.gate_proj.weight": "model-00023-of-00101.safetensors", + "model.layers.22.mlp.experts.29.up_proj.weight": "model-00023-of-00101.safetensors", + "model.layers.22.mlp.experts.29.down_proj.weight": "model-00023-of-00101.safetensors", + "model.layers.22.mlp.experts.30.gate_proj.weight": "model-00023-of-00101.safetensors", + "model.layers.22.mlp.experts.30.up_proj.weight": "model-00023-of-00101.safetensors", + "model.layers.22.mlp.experts.30.down_proj.weight": "model-00023-of-00101.safetensors", + "model.layers.22.mlp.experts.31.gate_proj.weight": "model-00023-of-00101.safetensors", + "model.layers.22.mlp.experts.31.up_proj.weight": "model-00023-of-00101.safetensors", + "model.layers.22.mlp.experts.31.down_proj.weight": "model-00023-of-00101.safetensors", + "model.layers.22.mlp.experts.32.gate_proj.weight": "model-00023-of-00101.safetensors", + "model.layers.22.mlp.experts.32.up_proj.weight": "model-00023-of-00101.safetensors", + "model.layers.22.mlp.experts.32.down_proj.weight": "model-00023-of-00101.safetensors", + "model.layers.22.mlp.experts.33.gate_proj.weight": "model-00023-of-00101.safetensors", + "model.layers.22.mlp.experts.33.up_proj.weight": "model-00023-of-00101.safetensors", + "model.layers.22.mlp.experts.33.down_proj.weight": "model-00023-of-00101.safetensors", + "model.layers.22.mlp.experts.34.gate_proj.weight": "model-00023-of-00101.safetensors", + "model.layers.22.mlp.experts.34.up_proj.weight": "model-00023-of-00101.safetensors", + "model.layers.22.mlp.experts.34.down_proj.weight": "model-00023-of-00101.safetensors", + "model.layers.22.mlp.experts.35.gate_proj.weight": "model-00023-of-00101.safetensors", + "model.layers.22.mlp.experts.35.up_proj.weight": "model-00023-of-00101.safetensors", + "model.layers.22.mlp.experts.35.down_proj.weight": "model-00023-of-00101.safetensors", + "model.layers.22.mlp.experts.36.gate_proj.weight": "model-00023-of-00101.safetensors", + "model.layers.22.mlp.experts.36.up_proj.weight": "model-00023-of-00101.safetensors", + "model.layers.22.mlp.experts.36.down_proj.weight": "model-00023-of-00101.safetensors", + "model.layers.22.mlp.experts.37.gate_proj.weight": "model-00023-of-00101.safetensors", + "model.layers.22.mlp.experts.37.up_proj.weight": "model-00023-of-00101.safetensors", + "model.layers.22.mlp.experts.37.down_proj.weight": "model-00023-of-00101.safetensors", + "model.layers.22.mlp.experts.38.gate_proj.weight": "model-00023-of-00101.safetensors", + "model.layers.22.mlp.experts.38.up_proj.weight": "model-00023-of-00101.safetensors", + "model.layers.22.mlp.experts.38.down_proj.weight": "model-00023-of-00101.safetensors", + "model.layers.22.mlp.experts.39.gate_proj.weight": "model-00023-of-00101.safetensors", + "model.layers.22.mlp.experts.39.up_proj.weight": "model-00023-of-00101.safetensors", + "model.layers.22.mlp.experts.39.down_proj.weight": "model-00023-of-00101.safetensors", + "model.layers.22.mlp.experts.40.gate_proj.weight": "model-00023-of-00101.safetensors", + "model.layers.22.mlp.experts.40.up_proj.weight": "model-00023-of-00101.safetensors", + "model.layers.22.mlp.experts.40.down_proj.weight": "model-00023-of-00101.safetensors", + "model.layers.22.mlp.experts.41.gate_proj.weight": "model-00023-of-00101.safetensors", + "model.layers.22.mlp.experts.41.up_proj.weight": "model-00023-of-00101.safetensors", + "model.layers.22.mlp.experts.41.down_proj.weight": "model-00023-of-00101.safetensors", + "model.layers.22.mlp.experts.42.gate_proj.weight": "model-00023-of-00101.safetensors", + "model.layers.22.mlp.experts.42.up_proj.weight": "model-00023-of-00101.safetensors", + "model.layers.22.mlp.experts.42.down_proj.weight": "model-00023-of-00101.safetensors", + "model.layers.22.mlp.experts.43.gate_proj.weight": "model-00023-of-00101.safetensors", + "model.layers.22.mlp.experts.43.up_proj.weight": "model-00023-of-00101.safetensors", + "model.layers.22.mlp.experts.43.down_proj.weight": "model-00023-of-00101.safetensors", + "model.layers.22.mlp.experts.44.gate_proj.weight": "model-00023-of-00101.safetensors", + "model.layers.22.mlp.experts.44.up_proj.weight": "model-00023-of-00101.safetensors", + "model.layers.22.mlp.experts.44.down_proj.weight": "model-00023-of-00101.safetensors", + "model.layers.22.mlp.experts.45.gate_proj.weight": "model-00023-of-00101.safetensors", + "model.layers.22.mlp.experts.45.up_proj.weight": "model-00023-of-00101.safetensors", + "model.layers.22.mlp.experts.45.down_proj.weight": "model-00023-of-00101.safetensors", + "model.layers.22.mlp.experts.46.gate_proj.weight": "model-00023-of-00101.safetensors", + "model.layers.22.mlp.experts.46.up_proj.weight": "model-00023-of-00101.safetensors", + "model.layers.22.mlp.experts.46.down_proj.weight": "model-00023-of-00101.safetensors", + "model.layers.22.mlp.experts.47.gate_proj.weight": "model-00023-of-00101.safetensors", + "model.layers.22.mlp.experts.47.up_proj.weight": "model-00023-of-00101.safetensors", + "model.layers.22.mlp.experts.47.down_proj.weight": "model-00023-of-00101.safetensors", + "model.layers.22.mlp.experts.48.gate_proj.weight": "model-00023-of-00101.safetensors", + "model.layers.22.mlp.experts.48.up_proj.weight": "model-00023-of-00101.safetensors", + "model.layers.22.mlp.experts.48.down_proj.weight": "model-00023-of-00101.safetensors", + "model.layers.22.mlp.experts.49.gate_proj.weight": "model-00023-of-00101.safetensors", + "model.layers.22.mlp.experts.49.up_proj.weight": "model-00023-of-00101.safetensors", + "model.layers.22.mlp.experts.49.down_proj.weight": "model-00023-of-00101.safetensors", + "model.layers.22.mlp.experts.50.gate_proj.weight": "model-00023-of-00101.safetensors", + "model.layers.22.mlp.experts.50.up_proj.weight": "model-00023-of-00101.safetensors", + "model.layers.22.mlp.experts.50.down_proj.weight": "model-00023-of-00101.safetensors", + "model.layers.22.mlp.experts.51.gate_proj.weight": "model-00023-of-00101.safetensors", + "model.layers.22.mlp.experts.51.up_proj.weight": "model-00023-of-00101.safetensors", + "model.layers.22.mlp.experts.51.down_proj.weight": "model-00023-of-00101.safetensors", + "model.layers.22.mlp.experts.52.gate_proj.weight": "model-00023-of-00101.safetensors", + "model.layers.22.mlp.experts.52.up_proj.weight": "model-00023-of-00101.safetensors", + "model.layers.22.mlp.experts.52.down_proj.weight": "model-00023-of-00101.safetensors", + "model.layers.22.mlp.experts.53.gate_proj.weight": "model-00023-of-00101.safetensors", + "model.layers.22.mlp.experts.53.up_proj.weight": "model-00023-of-00101.safetensors", + "model.layers.22.mlp.experts.53.down_proj.weight": "model-00023-of-00101.safetensors", + "model.layers.22.mlp.experts.54.gate_proj.weight": "model-00023-of-00101.safetensors", + "model.layers.22.mlp.experts.54.up_proj.weight": "model-00023-of-00101.safetensors", + "model.layers.22.mlp.experts.54.down_proj.weight": "model-00023-of-00101.safetensors", + "model.layers.22.mlp.experts.55.gate_proj.weight": "model-00023-of-00101.safetensors", + "model.layers.22.mlp.experts.55.up_proj.weight": "model-00023-of-00101.safetensors", + "model.layers.22.mlp.experts.55.down_proj.weight": "model-00023-of-00101.safetensors", + "model.layers.22.mlp.experts.56.gate_proj.weight": "model-00023-of-00101.safetensors", + "model.layers.22.mlp.experts.56.up_proj.weight": "model-00023-of-00101.safetensors", + "model.layers.22.mlp.experts.56.down_proj.weight": "model-00023-of-00101.safetensors", + "model.layers.22.mlp.experts.57.gate_proj.weight": "model-00023-of-00101.safetensors", + "model.layers.22.mlp.experts.57.up_proj.weight": "model-00023-of-00101.safetensors", + "model.layers.22.mlp.experts.57.down_proj.weight": "model-00023-of-00101.safetensors", + "model.layers.22.mlp.experts.58.gate_proj.weight": "model-00023-of-00101.safetensors", + "model.layers.22.mlp.experts.58.up_proj.weight": "model-00023-of-00101.safetensors", + "model.layers.22.mlp.experts.58.down_proj.weight": "model-00023-of-00101.safetensors", + "model.layers.22.mlp.experts.59.gate_proj.weight": "model-00023-of-00101.safetensors", + "model.layers.22.mlp.experts.59.up_proj.weight": "model-00023-of-00101.safetensors", + "model.layers.22.mlp.experts.59.down_proj.weight": "model-00023-of-00101.safetensors", + "model.layers.22.mlp.experts.60.gate_proj.weight": "model-00023-of-00101.safetensors", + "model.layers.22.mlp.experts.60.up_proj.weight": "model-00023-of-00101.safetensors", + "model.layers.22.mlp.experts.60.down_proj.weight": "model-00023-of-00101.safetensors", + "model.layers.22.mlp.experts.61.gate_proj.weight": "model-00023-of-00101.safetensors", + "model.layers.22.mlp.experts.61.up_proj.weight": "model-00023-of-00101.safetensors", + "model.layers.22.mlp.experts.61.down_proj.weight": "model-00023-of-00101.safetensors", + "model.layers.22.mlp.experts.62.gate_proj.weight": "model-00023-of-00101.safetensors", + "model.layers.22.mlp.experts.62.up_proj.weight": "model-00023-of-00101.safetensors", + "model.layers.22.mlp.experts.62.down_proj.weight": "model-00023-of-00101.safetensors", + "model.layers.22.mlp.experts.63.gate_proj.weight": "model-00023-of-00101.safetensors", + "model.layers.22.mlp.experts.63.up_proj.weight": "model-00023-of-00101.safetensors", + "model.layers.22.mlp.experts.63.down_proj.weight": "model-00023-of-00101.safetensors", + "model.layers.22.mlp.experts.64.gate_proj.weight": "model-00023-of-00101.safetensors", + "model.layers.22.mlp.experts.64.up_proj.weight": "model-00023-of-00101.safetensors", + "model.layers.22.mlp.experts.64.down_proj.weight": "model-00023-of-00101.safetensors", + "model.layers.22.mlp.experts.65.gate_proj.weight": "model-00023-of-00101.safetensors", + "model.layers.22.mlp.experts.65.up_proj.weight": "model-00023-of-00101.safetensors", + "model.layers.22.mlp.experts.65.down_proj.weight": "model-00023-of-00101.safetensors", + "model.layers.22.mlp.experts.66.gate_proj.weight": "model-00023-of-00101.safetensors", + "model.layers.22.mlp.experts.66.up_proj.weight": "model-00023-of-00101.safetensors", + "model.layers.22.mlp.experts.66.down_proj.weight": "model-00023-of-00101.safetensors", + "model.layers.22.mlp.experts.67.gate_proj.weight": "model-00023-of-00101.safetensors", + "model.layers.22.mlp.experts.67.up_proj.weight": "model-00023-of-00101.safetensors", + "model.layers.22.mlp.experts.67.down_proj.weight": "model-00023-of-00101.safetensors", + "model.layers.22.mlp.experts.68.gate_proj.weight": "model-00023-of-00101.safetensors", + "model.layers.22.mlp.experts.68.up_proj.weight": "model-00023-of-00101.safetensors", + "model.layers.22.mlp.experts.68.down_proj.weight": "model-00023-of-00101.safetensors", + "model.layers.22.mlp.experts.69.gate_proj.weight": "model-00023-of-00101.safetensors", + "model.layers.22.mlp.experts.69.up_proj.weight": "model-00023-of-00101.safetensors", + "model.layers.22.mlp.experts.69.down_proj.weight": "model-00023-of-00101.safetensors", + "model.layers.22.mlp.experts.70.gate_proj.weight": "model-00023-of-00101.safetensors", + "model.layers.22.mlp.experts.70.up_proj.weight": "model-00023-of-00101.safetensors", + "model.layers.22.mlp.experts.70.down_proj.weight": "model-00023-of-00101.safetensors", + "model.layers.22.mlp.experts.71.gate_proj.weight": "model-00023-of-00101.safetensors", + "model.layers.22.mlp.experts.71.up_proj.weight": "model-00023-of-00101.safetensors", + "model.layers.22.mlp.experts.71.down_proj.weight": "model-00023-of-00101.safetensors", + "model.layers.22.mlp.experts.72.gate_proj.weight": "model-00023-of-00101.safetensors", + "model.layers.22.mlp.experts.72.up_proj.weight": "model-00023-of-00101.safetensors", + "model.layers.22.mlp.experts.72.down_proj.weight": "model-00023-of-00101.safetensors", + "model.layers.22.mlp.experts.73.gate_proj.weight": "model-00023-of-00101.safetensors", + "model.layers.22.mlp.experts.73.up_proj.weight": "model-00023-of-00101.safetensors", + "model.layers.22.mlp.experts.73.down_proj.weight": "model-00023-of-00101.safetensors", + "model.layers.22.mlp.experts.74.gate_proj.weight": "model-00023-of-00101.safetensors", + "model.layers.22.mlp.experts.74.up_proj.weight": "model-00023-of-00101.safetensors", + "model.layers.22.mlp.experts.74.down_proj.weight": "model-00023-of-00101.safetensors", + "model.layers.22.mlp.experts.75.gate_proj.weight": "model-00023-of-00101.safetensors", + "model.layers.22.mlp.experts.75.up_proj.weight": "model-00023-of-00101.safetensors", + "model.layers.22.mlp.experts.75.down_proj.weight": "model-00023-of-00101.safetensors", + "model.layers.22.mlp.experts.76.gate_proj.weight": "model-00023-of-00101.safetensors", + "model.layers.22.mlp.experts.76.up_proj.weight": "model-00023-of-00101.safetensors", + "model.layers.22.mlp.experts.76.down_proj.weight": "model-00023-of-00101.safetensors", + "model.layers.22.mlp.experts.77.gate_proj.weight": "model-00023-of-00101.safetensors", + "model.layers.22.mlp.experts.77.up_proj.weight": "model-00023-of-00101.safetensors", + "model.layers.22.mlp.experts.77.down_proj.weight": "model-00023-of-00101.safetensors", + "model.layers.22.mlp.experts.78.gate_proj.weight": "model-00023-of-00101.safetensors", + "model.layers.22.mlp.experts.78.up_proj.weight": "model-00023-of-00101.safetensors", + "model.layers.22.mlp.experts.78.down_proj.weight": "model-00023-of-00101.safetensors", + "model.layers.22.mlp.experts.79.gate_proj.weight": "model-00023-of-00101.safetensors", + "model.layers.22.mlp.experts.79.up_proj.weight": "model-00023-of-00101.safetensors", + "model.layers.22.mlp.experts.79.down_proj.weight": "model-00023-of-00101.safetensors", + "model.layers.22.mlp.experts.80.gate_proj.weight": "model-00023-of-00101.safetensors", + "model.layers.22.mlp.experts.80.up_proj.weight": "model-00023-of-00101.safetensors", + "model.layers.22.mlp.experts.80.down_proj.weight": "model-00023-of-00101.safetensors", + "model.layers.22.mlp.experts.81.gate_proj.weight": "model-00023-of-00101.safetensors", + "model.layers.22.mlp.experts.81.up_proj.weight": "model-00023-of-00101.safetensors", + "model.layers.22.mlp.experts.81.down_proj.weight": "model-00023-of-00101.safetensors", + "model.layers.22.mlp.experts.82.gate_proj.weight": "model-00023-of-00101.safetensors", + "model.layers.22.mlp.experts.82.up_proj.weight": "model-00023-of-00101.safetensors", + "model.layers.22.mlp.experts.82.down_proj.weight": "model-00023-of-00101.safetensors", + "model.layers.22.mlp.experts.83.gate_proj.weight": "model-00023-of-00101.safetensors", + "model.layers.22.mlp.experts.83.up_proj.weight": "model-00023-of-00101.safetensors", + "model.layers.22.mlp.experts.83.down_proj.weight": "model-00023-of-00101.safetensors", + "model.layers.22.mlp.experts.84.gate_proj.weight": "model-00023-of-00101.safetensors", + "model.layers.22.mlp.experts.84.up_proj.weight": "model-00023-of-00101.safetensors", + "model.layers.22.mlp.experts.84.down_proj.weight": "model-00023-of-00101.safetensors", + "model.layers.22.mlp.experts.85.gate_proj.weight": "model-00023-of-00101.safetensors", + "model.layers.22.mlp.experts.85.up_proj.weight": "model-00023-of-00101.safetensors", + "model.layers.22.mlp.experts.85.down_proj.weight": "model-00023-of-00101.safetensors", + "model.layers.22.mlp.experts.86.gate_proj.weight": "model-00023-of-00101.safetensors", + "model.layers.22.mlp.experts.86.up_proj.weight": "model-00023-of-00101.safetensors", + "model.layers.22.mlp.experts.86.down_proj.weight": "model-00023-of-00101.safetensors", + "model.layers.22.mlp.experts.87.gate_proj.weight": "model-00023-of-00101.safetensors", + "model.layers.22.mlp.experts.87.up_proj.weight": "model-00023-of-00101.safetensors", + "model.layers.22.mlp.experts.87.down_proj.weight": "model-00023-of-00101.safetensors", + "model.layers.22.mlp.experts.88.gate_proj.weight": "model-00023-of-00101.safetensors", + "model.layers.22.mlp.experts.88.up_proj.weight": "model-00023-of-00101.safetensors", + "model.layers.22.mlp.experts.88.down_proj.weight": "model-00023-of-00101.safetensors", + "model.layers.22.mlp.experts.89.gate_proj.weight": "model-00023-of-00101.safetensors", + "model.layers.22.mlp.experts.89.up_proj.weight": "model-00023-of-00101.safetensors", + "model.layers.22.mlp.experts.89.down_proj.weight": "model-00023-of-00101.safetensors", + "model.layers.22.mlp.experts.90.gate_proj.weight": "model-00023-of-00101.safetensors", + "model.layers.22.mlp.experts.90.up_proj.weight": "model-00023-of-00101.safetensors", + "model.layers.22.mlp.experts.90.down_proj.weight": "model-00023-of-00101.safetensors", + "model.layers.22.mlp.experts.91.gate_proj.weight": "model-00023-of-00101.safetensors", + "model.layers.22.mlp.experts.91.up_proj.weight": "model-00023-of-00101.safetensors", + "model.layers.22.mlp.experts.91.down_proj.weight": "model-00023-of-00101.safetensors", + "model.layers.22.mlp.experts.92.gate_proj.weight": "model-00023-of-00101.safetensors", + "model.layers.22.mlp.experts.92.up_proj.weight": "model-00023-of-00101.safetensors", + "model.layers.22.mlp.experts.92.down_proj.weight": "model-00023-of-00101.safetensors", + "model.layers.22.mlp.experts.93.gate_proj.weight": "model-00023-of-00101.safetensors", + "model.layers.22.mlp.experts.93.up_proj.weight": "model-00023-of-00101.safetensors", + "model.layers.22.mlp.experts.93.down_proj.weight": "model-00023-of-00101.safetensors", + "model.layers.22.mlp.experts.94.gate_proj.weight": "model-00023-of-00101.safetensors", + "model.layers.22.mlp.experts.94.up_proj.weight": "model-00023-of-00101.safetensors", + "model.layers.22.mlp.experts.94.down_proj.weight": "model-00023-of-00101.safetensors", + "model.layers.22.mlp.experts.95.gate_proj.weight": "model-00023-of-00101.safetensors", + "model.layers.22.mlp.experts.95.up_proj.weight": "model-00023-of-00101.safetensors", + "model.layers.22.mlp.experts.95.down_proj.weight": "model-00023-of-00101.safetensors", + "model.layers.22.mlp.experts.96.gate_proj.weight": "model-00023-of-00101.safetensors", + "model.layers.22.mlp.experts.96.up_proj.weight": "model-00023-of-00101.safetensors", + "model.layers.22.mlp.experts.96.down_proj.weight": "model-00023-of-00101.safetensors", + "model.layers.22.mlp.experts.97.gate_proj.weight": "model-00023-of-00101.safetensors", + "model.layers.22.mlp.experts.97.up_proj.weight": "model-00023-of-00101.safetensors", + "model.layers.22.mlp.experts.97.down_proj.weight": "model-00023-of-00101.safetensors", + "model.layers.22.mlp.experts.98.gate_proj.weight": "model-00023-of-00101.safetensors", + "model.layers.22.mlp.experts.98.up_proj.weight": "model-00023-of-00101.safetensors", + "model.layers.22.mlp.experts.98.down_proj.weight": "model-00023-of-00101.safetensors", + "model.layers.22.mlp.experts.99.gate_proj.weight": "model-00023-of-00101.safetensors", + "model.layers.22.mlp.experts.99.up_proj.weight": "model-00023-of-00101.safetensors", + "model.layers.22.mlp.experts.99.down_proj.weight": "model-00023-of-00101.safetensors", + "model.layers.22.mlp.experts.100.gate_proj.weight": "model-00023-of-00101.safetensors", + "model.layers.22.mlp.experts.100.up_proj.weight": "model-00023-of-00101.safetensors", + "model.layers.22.mlp.experts.100.down_proj.weight": "model-00023-of-00101.safetensors", + "model.layers.22.mlp.experts.101.gate_proj.weight": "model-00023-of-00101.safetensors", + "model.layers.22.mlp.experts.101.up_proj.weight": "model-00023-of-00101.safetensors", + "model.layers.22.mlp.experts.101.down_proj.weight": "model-00023-of-00101.safetensors", + "model.layers.22.mlp.experts.102.gate_proj.weight": "model-00023-of-00101.safetensors", + "model.layers.22.mlp.experts.102.up_proj.weight": "model-00023-of-00101.safetensors", + "model.layers.22.mlp.experts.102.down_proj.weight": "model-00023-of-00101.safetensors", + "model.layers.22.mlp.experts.103.gate_proj.weight": "model-00023-of-00101.safetensors", + "model.layers.22.mlp.experts.103.up_proj.weight": "model-00023-of-00101.safetensors", + "model.layers.22.mlp.experts.103.down_proj.weight": "model-00023-of-00101.safetensors", + "model.layers.22.mlp.experts.104.gate_proj.weight": "model-00023-of-00101.safetensors", + "model.layers.22.mlp.experts.104.up_proj.weight": "model-00023-of-00101.safetensors", + "model.layers.22.mlp.experts.104.down_proj.weight": "model-00023-of-00101.safetensors", + "model.layers.22.mlp.experts.105.gate_proj.weight": "model-00023-of-00101.safetensors", + "model.layers.22.mlp.experts.105.up_proj.weight": "model-00023-of-00101.safetensors", + "model.layers.22.mlp.experts.105.down_proj.weight": "model-00023-of-00101.safetensors", + "model.layers.22.mlp.experts.106.gate_proj.weight": "model-00023-of-00101.safetensors", + "model.layers.22.mlp.experts.106.up_proj.weight": "model-00023-of-00101.safetensors", + "model.layers.22.mlp.experts.106.down_proj.weight": "model-00023-of-00101.safetensors", + "model.layers.22.mlp.experts.107.gate_proj.weight": "model-00023-of-00101.safetensors", + "model.layers.22.mlp.experts.107.up_proj.weight": "model-00023-of-00101.safetensors", + "model.layers.22.mlp.experts.107.down_proj.weight": "model-00023-of-00101.safetensors", + "model.layers.22.mlp.experts.108.gate_proj.weight": "model-00023-of-00101.safetensors", + "model.layers.22.mlp.experts.108.up_proj.weight": "model-00023-of-00101.safetensors", + "model.layers.22.mlp.experts.108.down_proj.weight": "model-00023-of-00101.safetensors", + "model.layers.22.mlp.experts.109.gate_proj.weight": "model-00023-of-00101.safetensors", + "model.layers.22.mlp.experts.109.up_proj.weight": "model-00023-of-00101.safetensors", + "model.layers.22.mlp.experts.109.down_proj.weight": "model-00023-of-00101.safetensors", + "model.layers.22.mlp.experts.110.gate_proj.weight": "model-00023-of-00101.safetensors", + "model.layers.22.mlp.experts.110.up_proj.weight": "model-00023-of-00101.safetensors", + "model.layers.22.mlp.experts.110.down_proj.weight": "model-00023-of-00101.safetensors", + "model.layers.22.mlp.experts.111.gate_proj.weight": "model-00023-of-00101.safetensors", + "model.layers.22.mlp.experts.111.up_proj.weight": "model-00023-of-00101.safetensors", + "model.layers.22.mlp.experts.111.down_proj.weight": "model-00023-of-00101.safetensors", + "model.layers.22.mlp.experts.112.gate_proj.weight": "model-00023-of-00101.safetensors", + "model.layers.22.mlp.experts.112.up_proj.weight": "model-00023-of-00101.safetensors", + "model.layers.22.mlp.experts.112.down_proj.weight": "model-00023-of-00101.safetensors", + "model.layers.22.mlp.experts.113.gate_proj.weight": "model-00023-of-00101.safetensors", + "model.layers.22.mlp.experts.113.up_proj.weight": "model-00023-of-00101.safetensors", + "model.layers.22.mlp.experts.113.down_proj.weight": "model-00023-of-00101.safetensors", + "model.layers.22.mlp.experts.114.gate_proj.weight": "model-00023-of-00101.safetensors", + "model.layers.22.mlp.experts.114.up_proj.weight": "model-00023-of-00101.safetensors", + "model.layers.22.mlp.experts.114.down_proj.weight": "model-00023-of-00101.safetensors", + "model.layers.22.mlp.experts.115.gate_proj.weight": "model-00023-of-00101.safetensors", + "model.layers.22.mlp.experts.115.up_proj.weight": "model-00023-of-00101.safetensors", + "model.layers.22.mlp.experts.115.down_proj.weight": "model-00023-of-00101.safetensors", + "model.layers.22.mlp.experts.116.gate_proj.weight": "model-00023-of-00101.safetensors", + "model.layers.22.mlp.experts.116.up_proj.weight": "model-00023-of-00101.safetensors", + "model.layers.22.mlp.experts.116.down_proj.weight": "model-00023-of-00101.safetensors", + "model.layers.22.mlp.experts.117.gate_proj.weight": "model-00023-of-00101.safetensors", + "model.layers.22.mlp.experts.117.up_proj.weight": "model-00023-of-00101.safetensors", + "model.layers.22.mlp.experts.117.down_proj.weight": "model-00023-of-00101.safetensors", + "model.layers.22.mlp.experts.118.gate_proj.weight": "model-00023-of-00101.safetensors", + "model.layers.22.mlp.experts.118.up_proj.weight": "model-00023-of-00101.safetensors", + "model.layers.22.mlp.experts.118.down_proj.weight": "model-00023-of-00101.safetensors", + "model.layers.22.mlp.experts.119.gate_proj.weight": "model-00023-of-00101.safetensors", + "model.layers.22.mlp.experts.119.up_proj.weight": "model-00023-of-00101.safetensors", + "model.layers.22.mlp.experts.119.down_proj.weight": "model-00023-of-00101.safetensors", + "model.layers.22.mlp.gate.weight": "model-00023-of-00101.safetensors", + "model.layers.22.mlp.gate.e_score_correction_bias": "model-00023-of-00101.safetensors", + "model.layers.22.mlp.shared_experts.gate_proj.weight": "model-00023-of-00101.safetensors", + "model.layers.22.mlp.shared_experts.up_proj.weight": "model-00023-of-00101.safetensors", + "model.layers.22.mlp.shared_experts.down_proj.weight": "model-00023-of-00101.safetensors", + "model.layers.22.input_layernorm.weight": "model-00023-of-00101.safetensors", + "model.layers.22.post_attention_layernorm.weight": "model-00023-of-00101.safetensors", + "model.layers.23.self_attn.q_proj.weight": "model-00024-of-00101.safetensors", + "model.layers.23.self_attn.q_proj.bias": "model-00024-of-00101.safetensors", + "model.layers.23.self_attn.k_proj.weight": "model-00024-of-00101.safetensors", + "model.layers.23.self_attn.k_proj.bias": "model-00024-of-00101.safetensors", + "model.layers.23.self_attn.v_proj.weight": "model-00024-of-00101.safetensors", + "model.layers.23.self_attn.v_proj.bias": "model-00024-of-00101.safetensors", + "model.layers.23.self_attn.o_proj.weight": "model-00024-of-00101.safetensors", + "model.layers.23.self_attn.q_norm.weight": "model-00024-of-00101.safetensors", + "model.layers.23.self_attn.k_norm.weight": "model-00024-of-00101.safetensors", + "model.layers.23.mlp.experts.0.gate_proj.weight": "model-00024-of-00101.safetensors", + "model.layers.23.mlp.experts.0.up_proj.weight": "model-00024-of-00101.safetensors", + "model.layers.23.mlp.experts.0.down_proj.weight": "model-00024-of-00101.safetensors", + "model.layers.23.mlp.experts.1.gate_proj.weight": "model-00024-of-00101.safetensors", + "model.layers.23.mlp.experts.1.up_proj.weight": "model-00024-of-00101.safetensors", + "model.layers.23.mlp.experts.1.down_proj.weight": "model-00024-of-00101.safetensors", + "model.layers.23.mlp.experts.2.gate_proj.weight": "model-00024-of-00101.safetensors", + "model.layers.23.mlp.experts.2.up_proj.weight": "model-00024-of-00101.safetensors", + "model.layers.23.mlp.experts.2.down_proj.weight": "model-00024-of-00101.safetensors", + "model.layers.23.mlp.experts.3.gate_proj.weight": "model-00024-of-00101.safetensors", + "model.layers.23.mlp.experts.3.up_proj.weight": "model-00024-of-00101.safetensors", + "model.layers.23.mlp.experts.3.down_proj.weight": "model-00024-of-00101.safetensors", + "model.layers.23.mlp.experts.4.gate_proj.weight": "model-00024-of-00101.safetensors", + "model.layers.23.mlp.experts.4.up_proj.weight": "model-00024-of-00101.safetensors", + "model.layers.23.mlp.experts.4.down_proj.weight": "model-00024-of-00101.safetensors", + "model.layers.23.mlp.experts.5.gate_proj.weight": "model-00024-of-00101.safetensors", + "model.layers.23.mlp.experts.5.up_proj.weight": "model-00024-of-00101.safetensors", + "model.layers.23.mlp.experts.5.down_proj.weight": "model-00024-of-00101.safetensors", + "model.layers.23.mlp.experts.6.gate_proj.weight": "model-00024-of-00101.safetensors", + "model.layers.23.mlp.experts.6.up_proj.weight": "model-00024-of-00101.safetensors", + "model.layers.23.mlp.experts.6.down_proj.weight": "model-00024-of-00101.safetensors", + "model.layers.23.mlp.experts.7.gate_proj.weight": "model-00024-of-00101.safetensors", + "model.layers.23.mlp.experts.7.up_proj.weight": "model-00024-of-00101.safetensors", + "model.layers.23.mlp.experts.7.down_proj.weight": "model-00024-of-00101.safetensors", + "model.layers.23.mlp.experts.8.gate_proj.weight": "model-00024-of-00101.safetensors", + "model.layers.23.mlp.experts.8.up_proj.weight": "model-00024-of-00101.safetensors", + "model.layers.23.mlp.experts.8.down_proj.weight": "model-00024-of-00101.safetensors", + "model.layers.23.mlp.experts.9.gate_proj.weight": "model-00024-of-00101.safetensors", + "model.layers.23.mlp.experts.9.up_proj.weight": "model-00024-of-00101.safetensors", + "model.layers.23.mlp.experts.9.down_proj.weight": "model-00024-of-00101.safetensors", + "model.layers.23.mlp.experts.10.gate_proj.weight": "model-00024-of-00101.safetensors", + "model.layers.23.mlp.experts.10.up_proj.weight": "model-00024-of-00101.safetensors", + "model.layers.23.mlp.experts.10.down_proj.weight": "model-00024-of-00101.safetensors", + "model.layers.23.mlp.experts.11.gate_proj.weight": "model-00024-of-00101.safetensors", + "model.layers.23.mlp.experts.11.up_proj.weight": "model-00024-of-00101.safetensors", + "model.layers.23.mlp.experts.11.down_proj.weight": "model-00024-of-00101.safetensors", + "model.layers.23.mlp.experts.12.gate_proj.weight": "model-00024-of-00101.safetensors", + "model.layers.23.mlp.experts.12.up_proj.weight": "model-00024-of-00101.safetensors", + "model.layers.23.mlp.experts.12.down_proj.weight": "model-00024-of-00101.safetensors", + "model.layers.23.mlp.experts.13.gate_proj.weight": "model-00024-of-00101.safetensors", + "model.layers.23.mlp.experts.13.up_proj.weight": "model-00024-of-00101.safetensors", + "model.layers.23.mlp.experts.13.down_proj.weight": "model-00024-of-00101.safetensors", + "model.layers.23.mlp.experts.14.gate_proj.weight": "model-00024-of-00101.safetensors", + "model.layers.23.mlp.experts.14.up_proj.weight": "model-00024-of-00101.safetensors", + "model.layers.23.mlp.experts.14.down_proj.weight": "model-00024-of-00101.safetensors", + "model.layers.23.mlp.experts.15.gate_proj.weight": "model-00024-of-00101.safetensors", + "model.layers.23.mlp.experts.15.up_proj.weight": "model-00024-of-00101.safetensors", + "model.layers.23.mlp.experts.15.down_proj.weight": "model-00024-of-00101.safetensors", + "model.layers.23.mlp.experts.16.gate_proj.weight": "model-00024-of-00101.safetensors", + "model.layers.23.mlp.experts.16.up_proj.weight": "model-00024-of-00101.safetensors", + "model.layers.23.mlp.experts.16.down_proj.weight": "model-00024-of-00101.safetensors", + "model.layers.23.mlp.experts.17.gate_proj.weight": "model-00024-of-00101.safetensors", + "model.layers.23.mlp.experts.17.up_proj.weight": "model-00024-of-00101.safetensors", + "model.layers.23.mlp.experts.17.down_proj.weight": "model-00024-of-00101.safetensors", + "model.layers.23.mlp.experts.18.gate_proj.weight": "model-00024-of-00101.safetensors", + "model.layers.23.mlp.experts.18.up_proj.weight": "model-00024-of-00101.safetensors", + "model.layers.23.mlp.experts.18.down_proj.weight": "model-00024-of-00101.safetensors", + "model.layers.23.mlp.experts.19.gate_proj.weight": "model-00024-of-00101.safetensors", + "model.layers.23.mlp.experts.19.up_proj.weight": "model-00024-of-00101.safetensors", + "model.layers.23.mlp.experts.19.down_proj.weight": "model-00024-of-00101.safetensors", + "model.layers.23.mlp.experts.20.gate_proj.weight": "model-00024-of-00101.safetensors", + "model.layers.23.mlp.experts.20.up_proj.weight": "model-00024-of-00101.safetensors", + "model.layers.23.mlp.experts.20.down_proj.weight": "model-00024-of-00101.safetensors", + "model.layers.23.mlp.experts.21.gate_proj.weight": "model-00024-of-00101.safetensors", + "model.layers.23.mlp.experts.21.up_proj.weight": "model-00024-of-00101.safetensors", + "model.layers.23.mlp.experts.21.down_proj.weight": "model-00024-of-00101.safetensors", + "model.layers.23.mlp.experts.22.gate_proj.weight": "model-00024-of-00101.safetensors", + "model.layers.23.mlp.experts.22.up_proj.weight": "model-00024-of-00101.safetensors", + "model.layers.23.mlp.experts.22.down_proj.weight": "model-00024-of-00101.safetensors", + "model.layers.23.mlp.experts.23.gate_proj.weight": "model-00024-of-00101.safetensors", + "model.layers.23.mlp.experts.23.up_proj.weight": "model-00024-of-00101.safetensors", + "model.layers.23.mlp.experts.23.down_proj.weight": "model-00024-of-00101.safetensors", + "model.layers.23.mlp.experts.24.gate_proj.weight": "model-00024-of-00101.safetensors", + "model.layers.23.mlp.experts.24.up_proj.weight": "model-00024-of-00101.safetensors", + "model.layers.23.mlp.experts.24.down_proj.weight": "model-00024-of-00101.safetensors", + "model.layers.23.mlp.experts.25.gate_proj.weight": "model-00024-of-00101.safetensors", + "model.layers.23.mlp.experts.25.up_proj.weight": "model-00024-of-00101.safetensors", + "model.layers.23.mlp.experts.25.down_proj.weight": "model-00024-of-00101.safetensors", + "model.layers.23.mlp.experts.26.gate_proj.weight": "model-00024-of-00101.safetensors", + "model.layers.23.mlp.experts.26.up_proj.weight": "model-00024-of-00101.safetensors", + "model.layers.23.mlp.experts.26.down_proj.weight": "model-00024-of-00101.safetensors", + "model.layers.23.mlp.experts.27.gate_proj.weight": "model-00024-of-00101.safetensors", + "model.layers.23.mlp.experts.27.up_proj.weight": "model-00024-of-00101.safetensors", + "model.layers.23.mlp.experts.27.down_proj.weight": "model-00024-of-00101.safetensors", + "model.layers.23.mlp.experts.28.gate_proj.weight": "model-00024-of-00101.safetensors", + "model.layers.23.mlp.experts.28.up_proj.weight": "model-00024-of-00101.safetensors", + "model.layers.23.mlp.experts.28.down_proj.weight": "model-00024-of-00101.safetensors", + "model.layers.23.mlp.experts.29.gate_proj.weight": "model-00024-of-00101.safetensors", + "model.layers.23.mlp.experts.29.up_proj.weight": "model-00024-of-00101.safetensors", + "model.layers.23.mlp.experts.29.down_proj.weight": "model-00024-of-00101.safetensors", + "model.layers.23.mlp.experts.30.gate_proj.weight": "model-00024-of-00101.safetensors", + "model.layers.23.mlp.experts.30.up_proj.weight": "model-00024-of-00101.safetensors", + "model.layers.23.mlp.experts.30.down_proj.weight": "model-00024-of-00101.safetensors", + "model.layers.23.mlp.experts.31.gate_proj.weight": "model-00024-of-00101.safetensors", + "model.layers.23.mlp.experts.31.up_proj.weight": "model-00024-of-00101.safetensors", + "model.layers.23.mlp.experts.31.down_proj.weight": "model-00024-of-00101.safetensors", + "model.layers.23.mlp.experts.32.gate_proj.weight": "model-00024-of-00101.safetensors", + "model.layers.23.mlp.experts.32.up_proj.weight": "model-00024-of-00101.safetensors", + "model.layers.23.mlp.experts.32.down_proj.weight": "model-00024-of-00101.safetensors", + "model.layers.23.mlp.experts.33.gate_proj.weight": "model-00024-of-00101.safetensors", + "model.layers.23.mlp.experts.33.up_proj.weight": "model-00024-of-00101.safetensors", + "model.layers.23.mlp.experts.33.down_proj.weight": "model-00024-of-00101.safetensors", + "model.layers.23.mlp.experts.34.gate_proj.weight": "model-00024-of-00101.safetensors", + "model.layers.23.mlp.experts.34.up_proj.weight": "model-00024-of-00101.safetensors", + "model.layers.23.mlp.experts.34.down_proj.weight": "model-00024-of-00101.safetensors", + "model.layers.23.mlp.experts.35.gate_proj.weight": "model-00024-of-00101.safetensors", + "model.layers.23.mlp.experts.35.up_proj.weight": "model-00024-of-00101.safetensors", + "model.layers.23.mlp.experts.35.down_proj.weight": "model-00024-of-00101.safetensors", + "model.layers.23.mlp.experts.36.gate_proj.weight": "model-00024-of-00101.safetensors", + "model.layers.23.mlp.experts.36.up_proj.weight": "model-00024-of-00101.safetensors", + "model.layers.23.mlp.experts.36.down_proj.weight": "model-00024-of-00101.safetensors", + "model.layers.23.mlp.experts.37.gate_proj.weight": "model-00024-of-00101.safetensors", + "model.layers.23.mlp.experts.37.up_proj.weight": "model-00024-of-00101.safetensors", + "model.layers.23.mlp.experts.37.down_proj.weight": "model-00024-of-00101.safetensors", + "model.layers.23.mlp.experts.38.gate_proj.weight": "model-00024-of-00101.safetensors", + "model.layers.23.mlp.experts.38.up_proj.weight": "model-00024-of-00101.safetensors", + "model.layers.23.mlp.experts.38.down_proj.weight": "model-00024-of-00101.safetensors", + "model.layers.23.mlp.experts.39.gate_proj.weight": "model-00024-of-00101.safetensors", + "model.layers.23.mlp.experts.39.up_proj.weight": "model-00024-of-00101.safetensors", + "model.layers.23.mlp.experts.39.down_proj.weight": "model-00024-of-00101.safetensors", + "model.layers.23.mlp.experts.40.gate_proj.weight": "model-00024-of-00101.safetensors", + "model.layers.23.mlp.experts.40.up_proj.weight": "model-00024-of-00101.safetensors", + "model.layers.23.mlp.experts.40.down_proj.weight": "model-00024-of-00101.safetensors", + "model.layers.23.mlp.experts.41.gate_proj.weight": "model-00024-of-00101.safetensors", + "model.layers.23.mlp.experts.41.up_proj.weight": "model-00024-of-00101.safetensors", + "model.layers.23.mlp.experts.41.down_proj.weight": "model-00024-of-00101.safetensors", + "model.layers.23.mlp.experts.42.gate_proj.weight": "model-00024-of-00101.safetensors", + "model.layers.23.mlp.experts.42.up_proj.weight": "model-00024-of-00101.safetensors", + "model.layers.23.mlp.experts.42.down_proj.weight": "model-00024-of-00101.safetensors", + "model.layers.23.mlp.experts.43.gate_proj.weight": "model-00024-of-00101.safetensors", + "model.layers.23.mlp.experts.43.up_proj.weight": "model-00024-of-00101.safetensors", + "model.layers.23.mlp.experts.43.down_proj.weight": "model-00024-of-00101.safetensors", + "model.layers.23.mlp.experts.44.gate_proj.weight": "model-00024-of-00101.safetensors", + "model.layers.23.mlp.experts.44.up_proj.weight": "model-00024-of-00101.safetensors", + "model.layers.23.mlp.experts.44.down_proj.weight": "model-00024-of-00101.safetensors", + "model.layers.23.mlp.experts.45.gate_proj.weight": "model-00024-of-00101.safetensors", + "model.layers.23.mlp.experts.45.up_proj.weight": "model-00024-of-00101.safetensors", + "model.layers.23.mlp.experts.45.down_proj.weight": "model-00024-of-00101.safetensors", + "model.layers.23.mlp.experts.46.gate_proj.weight": "model-00024-of-00101.safetensors", + "model.layers.23.mlp.experts.46.up_proj.weight": "model-00024-of-00101.safetensors", + "model.layers.23.mlp.experts.46.down_proj.weight": "model-00024-of-00101.safetensors", + "model.layers.23.mlp.experts.47.gate_proj.weight": "model-00024-of-00101.safetensors", + "model.layers.23.mlp.experts.47.up_proj.weight": "model-00024-of-00101.safetensors", + "model.layers.23.mlp.experts.47.down_proj.weight": "model-00024-of-00101.safetensors", + "model.layers.23.mlp.experts.48.gate_proj.weight": "model-00024-of-00101.safetensors", + "model.layers.23.mlp.experts.48.up_proj.weight": "model-00024-of-00101.safetensors", + "model.layers.23.mlp.experts.48.down_proj.weight": "model-00024-of-00101.safetensors", + "model.layers.23.mlp.experts.49.gate_proj.weight": "model-00024-of-00101.safetensors", + "model.layers.23.mlp.experts.49.up_proj.weight": "model-00024-of-00101.safetensors", + "model.layers.23.mlp.experts.49.down_proj.weight": "model-00024-of-00101.safetensors", + "model.layers.23.mlp.experts.50.gate_proj.weight": "model-00024-of-00101.safetensors", + "model.layers.23.mlp.experts.50.up_proj.weight": "model-00024-of-00101.safetensors", + "model.layers.23.mlp.experts.50.down_proj.weight": "model-00024-of-00101.safetensors", + "model.layers.23.mlp.experts.51.gate_proj.weight": "model-00024-of-00101.safetensors", + "model.layers.23.mlp.experts.51.up_proj.weight": "model-00024-of-00101.safetensors", + "model.layers.23.mlp.experts.51.down_proj.weight": "model-00024-of-00101.safetensors", + "model.layers.23.mlp.experts.52.gate_proj.weight": "model-00024-of-00101.safetensors", + "model.layers.23.mlp.experts.52.up_proj.weight": "model-00024-of-00101.safetensors", + "model.layers.23.mlp.experts.52.down_proj.weight": "model-00024-of-00101.safetensors", + "model.layers.23.mlp.experts.53.gate_proj.weight": "model-00024-of-00101.safetensors", + "model.layers.23.mlp.experts.53.up_proj.weight": "model-00024-of-00101.safetensors", + "model.layers.23.mlp.experts.53.down_proj.weight": "model-00024-of-00101.safetensors", + "model.layers.23.mlp.experts.54.gate_proj.weight": "model-00024-of-00101.safetensors", + "model.layers.23.mlp.experts.54.up_proj.weight": "model-00024-of-00101.safetensors", + "model.layers.23.mlp.experts.54.down_proj.weight": "model-00024-of-00101.safetensors", + "model.layers.23.mlp.experts.55.gate_proj.weight": "model-00024-of-00101.safetensors", + "model.layers.23.mlp.experts.55.up_proj.weight": "model-00024-of-00101.safetensors", + "model.layers.23.mlp.experts.55.down_proj.weight": "model-00024-of-00101.safetensors", + "model.layers.23.mlp.experts.56.gate_proj.weight": "model-00024-of-00101.safetensors", + "model.layers.23.mlp.experts.56.up_proj.weight": "model-00024-of-00101.safetensors", + "model.layers.23.mlp.experts.56.down_proj.weight": "model-00024-of-00101.safetensors", + "model.layers.23.mlp.experts.57.gate_proj.weight": "model-00024-of-00101.safetensors", + "model.layers.23.mlp.experts.57.up_proj.weight": "model-00024-of-00101.safetensors", + "model.layers.23.mlp.experts.57.down_proj.weight": "model-00024-of-00101.safetensors", + "model.layers.23.mlp.experts.58.gate_proj.weight": "model-00024-of-00101.safetensors", + "model.layers.23.mlp.experts.58.up_proj.weight": "model-00024-of-00101.safetensors", + "model.layers.23.mlp.experts.58.down_proj.weight": "model-00024-of-00101.safetensors", + "model.layers.23.mlp.experts.59.gate_proj.weight": "model-00024-of-00101.safetensors", + "model.layers.23.mlp.experts.59.up_proj.weight": "model-00024-of-00101.safetensors", + "model.layers.23.mlp.experts.59.down_proj.weight": "model-00024-of-00101.safetensors", + "model.layers.23.mlp.experts.60.gate_proj.weight": "model-00024-of-00101.safetensors", + "model.layers.23.mlp.experts.60.up_proj.weight": "model-00024-of-00101.safetensors", + "model.layers.23.mlp.experts.60.down_proj.weight": "model-00024-of-00101.safetensors", + "model.layers.23.mlp.experts.61.gate_proj.weight": "model-00024-of-00101.safetensors", + "model.layers.23.mlp.experts.61.up_proj.weight": "model-00024-of-00101.safetensors", + "model.layers.23.mlp.experts.61.down_proj.weight": "model-00024-of-00101.safetensors", + "model.layers.23.mlp.experts.62.gate_proj.weight": "model-00024-of-00101.safetensors", + "model.layers.23.mlp.experts.62.up_proj.weight": "model-00024-of-00101.safetensors", + "model.layers.23.mlp.experts.62.down_proj.weight": "model-00024-of-00101.safetensors", + "model.layers.23.mlp.experts.63.gate_proj.weight": "model-00024-of-00101.safetensors", + "model.layers.23.mlp.experts.63.up_proj.weight": "model-00024-of-00101.safetensors", + "model.layers.23.mlp.experts.63.down_proj.weight": "model-00024-of-00101.safetensors", + "model.layers.23.mlp.experts.64.gate_proj.weight": "model-00024-of-00101.safetensors", + "model.layers.23.mlp.experts.64.up_proj.weight": "model-00024-of-00101.safetensors", + "model.layers.23.mlp.experts.64.down_proj.weight": "model-00024-of-00101.safetensors", + "model.layers.23.mlp.experts.65.gate_proj.weight": "model-00024-of-00101.safetensors", + "model.layers.23.mlp.experts.65.up_proj.weight": "model-00024-of-00101.safetensors", + "model.layers.23.mlp.experts.65.down_proj.weight": "model-00024-of-00101.safetensors", + "model.layers.23.mlp.experts.66.gate_proj.weight": "model-00024-of-00101.safetensors", + "model.layers.23.mlp.experts.66.up_proj.weight": "model-00024-of-00101.safetensors", + "model.layers.23.mlp.experts.66.down_proj.weight": "model-00024-of-00101.safetensors", + "model.layers.23.mlp.experts.67.gate_proj.weight": "model-00024-of-00101.safetensors", + "model.layers.23.mlp.experts.67.up_proj.weight": "model-00024-of-00101.safetensors", + "model.layers.23.mlp.experts.67.down_proj.weight": "model-00024-of-00101.safetensors", + "model.layers.23.mlp.experts.68.gate_proj.weight": "model-00024-of-00101.safetensors", + "model.layers.23.mlp.experts.68.up_proj.weight": "model-00024-of-00101.safetensors", + "model.layers.23.mlp.experts.68.down_proj.weight": "model-00024-of-00101.safetensors", + "model.layers.23.mlp.experts.69.gate_proj.weight": "model-00024-of-00101.safetensors", + "model.layers.23.mlp.experts.69.up_proj.weight": "model-00024-of-00101.safetensors", + "model.layers.23.mlp.experts.69.down_proj.weight": "model-00024-of-00101.safetensors", + "model.layers.23.mlp.experts.70.gate_proj.weight": "model-00024-of-00101.safetensors", + "model.layers.23.mlp.experts.70.up_proj.weight": "model-00024-of-00101.safetensors", + "model.layers.23.mlp.experts.70.down_proj.weight": "model-00024-of-00101.safetensors", + "model.layers.23.mlp.experts.71.gate_proj.weight": "model-00024-of-00101.safetensors", + "model.layers.23.mlp.experts.71.up_proj.weight": "model-00024-of-00101.safetensors", + "model.layers.23.mlp.experts.71.down_proj.weight": "model-00024-of-00101.safetensors", + "model.layers.23.mlp.experts.72.gate_proj.weight": "model-00024-of-00101.safetensors", + "model.layers.23.mlp.experts.72.up_proj.weight": "model-00024-of-00101.safetensors", + "model.layers.23.mlp.experts.72.down_proj.weight": "model-00024-of-00101.safetensors", + "model.layers.23.mlp.experts.73.gate_proj.weight": "model-00024-of-00101.safetensors", + "model.layers.23.mlp.experts.73.up_proj.weight": "model-00024-of-00101.safetensors", + "model.layers.23.mlp.experts.73.down_proj.weight": "model-00024-of-00101.safetensors", + "model.layers.23.mlp.experts.74.gate_proj.weight": "model-00024-of-00101.safetensors", + "model.layers.23.mlp.experts.74.up_proj.weight": "model-00024-of-00101.safetensors", + "model.layers.23.mlp.experts.74.down_proj.weight": "model-00024-of-00101.safetensors", + "model.layers.23.mlp.experts.75.gate_proj.weight": "model-00024-of-00101.safetensors", + "model.layers.23.mlp.experts.75.up_proj.weight": "model-00024-of-00101.safetensors", + "model.layers.23.mlp.experts.75.down_proj.weight": "model-00024-of-00101.safetensors", + "model.layers.23.mlp.experts.76.gate_proj.weight": "model-00024-of-00101.safetensors", + "model.layers.23.mlp.experts.76.up_proj.weight": "model-00024-of-00101.safetensors", + "model.layers.23.mlp.experts.76.down_proj.weight": "model-00024-of-00101.safetensors", + "model.layers.23.mlp.experts.77.gate_proj.weight": "model-00024-of-00101.safetensors", + "model.layers.23.mlp.experts.77.up_proj.weight": "model-00024-of-00101.safetensors", + "model.layers.23.mlp.experts.77.down_proj.weight": "model-00024-of-00101.safetensors", + "model.layers.23.mlp.experts.78.gate_proj.weight": "model-00024-of-00101.safetensors", + "model.layers.23.mlp.experts.78.up_proj.weight": "model-00024-of-00101.safetensors", + "model.layers.23.mlp.experts.78.down_proj.weight": "model-00024-of-00101.safetensors", + "model.layers.23.mlp.experts.79.gate_proj.weight": "model-00024-of-00101.safetensors", + "model.layers.23.mlp.experts.79.up_proj.weight": "model-00024-of-00101.safetensors", + "model.layers.23.mlp.experts.79.down_proj.weight": "model-00024-of-00101.safetensors", + "model.layers.23.mlp.experts.80.gate_proj.weight": "model-00024-of-00101.safetensors", + "model.layers.23.mlp.experts.80.up_proj.weight": "model-00024-of-00101.safetensors", + "model.layers.23.mlp.experts.80.down_proj.weight": "model-00024-of-00101.safetensors", + "model.layers.23.mlp.experts.81.gate_proj.weight": "model-00024-of-00101.safetensors", + "model.layers.23.mlp.experts.81.up_proj.weight": "model-00024-of-00101.safetensors", + "model.layers.23.mlp.experts.81.down_proj.weight": "model-00024-of-00101.safetensors", + "model.layers.23.mlp.experts.82.gate_proj.weight": "model-00024-of-00101.safetensors", + "model.layers.23.mlp.experts.82.up_proj.weight": "model-00024-of-00101.safetensors", + "model.layers.23.mlp.experts.82.down_proj.weight": "model-00024-of-00101.safetensors", + "model.layers.23.mlp.experts.83.gate_proj.weight": "model-00024-of-00101.safetensors", + "model.layers.23.mlp.experts.83.up_proj.weight": "model-00024-of-00101.safetensors", + "model.layers.23.mlp.experts.83.down_proj.weight": "model-00024-of-00101.safetensors", + "model.layers.23.mlp.experts.84.gate_proj.weight": "model-00024-of-00101.safetensors", + "model.layers.23.mlp.experts.84.up_proj.weight": "model-00024-of-00101.safetensors", + "model.layers.23.mlp.experts.84.down_proj.weight": "model-00024-of-00101.safetensors", + "model.layers.23.mlp.experts.85.gate_proj.weight": "model-00024-of-00101.safetensors", + "model.layers.23.mlp.experts.85.up_proj.weight": "model-00024-of-00101.safetensors", + "model.layers.23.mlp.experts.85.down_proj.weight": "model-00024-of-00101.safetensors", + "model.layers.23.mlp.experts.86.gate_proj.weight": "model-00024-of-00101.safetensors", + "model.layers.23.mlp.experts.86.up_proj.weight": "model-00024-of-00101.safetensors", + "model.layers.23.mlp.experts.86.down_proj.weight": "model-00024-of-00101.safetensors", + "model.layers.23.mlp.experts.87.gate_proj.weight": "model-00024-of-00101.safetensors", + "model.layers.23.mlp.experts.87.up_proj.weight": "model-00024-of-00101.safetensors", + "model.layers.23.mlp.experts.87.down_proj.weight": "model-00024-of-00101.safetensors", + "model.layers.23.mlp.experts.88.gate_proj.weight": "model-00024-of-00101.safetensors", + "model.layers.23.mlp.experts.88.up_proj.weight": "model-00024-of-00101.safetensors", + "model.layers.23.mlp.experts.88.down_proj.weight": "model-00024-of-00101.safetensors", + "model.layers.23.mlp.experts.89.gate_proj.weight": "model-00024-of-00101.safetensors", + "model.layers.23.mlp.experts.89.up_proj.weight": "model-00024-of-00101.safetensors", + "model.layers.23.mlp.experts.89.down_proj.weight": "model-00024-of-00101.safetensors", + "model.layers.23.mlp.experts.90.gate_proj.weight": "model-00024-of-00101.safetensors", + "model.layers.23.mlp.experts.90.up_proj.weight": "model-00024-of-00101.safetensors", + "model.layers.23.mlp.experts.90.down_proj.weight": "model-00024-of-00101.safetensors", + "model.layers.23.mlp.experts.91.gate_proj.weight": "model-00024-of-00101.safetensors", + "model.layers.23.mlp.experts.91.up_proj.weight": "model-00024-of-00101.safetensors", + "model.layers.23.mlp.experts.91.down_proj.weight": "model-00024-of-00101.safetensors", + "model.layers.23.mlp.experts.92.gate_proj.weight": "model-00024-of-00101.safetensors", + "model.layers.23.mlp.experts.92.up_proj.weight": "model-00024-of-00101.safetensors", + "model.layers.23.mlp.experts.92.down_proj.weight": "model-00024-of-00101.safetensors", + "model.layers.23.mlp.experts.93.gate_proj.weight": "model-00024-of-00101.safetensors", + "model.layers.23.mlp.experts.93.up_proj.weight": "model-00024-of-00101.safetensors", + "model.layers.23.mlp.experts.93.down_proj.weight": "model-00024-of-00101.safetensors", + "model.layers.23.mlp.experts.94.gate_proj.weight": "model-00024-of-00101.safetensors", + "model.layers.23.mlp.experts.94.up_proj.weight": "model-00024-of-00101.safetensors", + "model.layers.23.mlp.experts.94.down_proj.weight": "model-00024-of-00101.safetensors", + "model.layers.23.mlp.experts.95.gate_proj.weight": "model-00024-of-00101.safetensors", + "model.layers.23.mlp.experts.95.up_proj.weight": "model-00024-of-00101.safetensors", + "model.layers.23.mlp.experts.95.down_proj.weight": "model-00024-of-00101.safetensors", + "model.layers.23.mlp.experts.96.gate_proj.weight": "model-00024-of-00101.safetensors", + "model.layers.23.mlp.experts.96.up_proj.weight": "model-00024-of-00101.safetensors", + "model.layers.23.mlp.experts.96.down_proj.weight": "model-00024-of-00101.safetensors", + "model.layers.23.mlp.experts.97.gate_proj.weight": "model-00024-of-00101.safetensors", + "model.layers.23.mlp.experts.97.up_proj.weight": "model-00024-of-00101.safetensors", + "model.layers.23.mlp.experts.97.down_proj.weight": "model-00024-of-00101.safetensors", + "model.layers.23.mlp.experts.98.gate_proj.weight": "model-00024-of-00101.safetensors", + "model.layers.23.mlp.experts.98.up_proj.weight": "model-00024-of-00101.safetensors", + "model.layers.23.mlp.experts.98.down_proj.weight": "model-00024-of-00101.safetensors", + "model.layers.23.mlp.experts.99.gate_proj.weight": "model-00024-of-00101.safetensors", + "model.layers.23.mlp.experts.99.up_proj.weight": "model-00024-of-00101.safetensors", + "model.layers.23.mlp.experts.99.down_proj.weight": "model-00024-of-00101.safetensors", + "model.layers.23.mlp.experts.100.gate_proj.weight": "model-00024-of-00101.safetensors", + "model.layers.23.mlp.experts.100.up_proj.weight": "model-00024-of-00101.safetensors", + "model.layers.23.mlp.experts.100.down_proj.weight": "model-00024-of-00101.safetensors", + "model.layers.23.mlp.experts.101.gate_proj.weight": "model-00024-of-00101.safetensors", + "model.layers.23.mlp.experts.101.up_proj.weight": "model-00024-of-00101.safetensors", + "model.layers.23.mlp.experts.101.down_proj.weight": "model-00024-of-00101.safetensors", + "model.layers.23.mlp.experts.102.gate_proj.weight": "model-00024-of-00101.safetensors", + "model.layers.23.mlp.experts.102.up_proj.weight": "model-00024-of-00101.safetensors", + "model.layers.23.mlp.experts.102.down_proj.weight": "model-00024-of-00101.safetensors", + "model.layers.23.mlp.experts.103.gate_proj.weight": "model-00024-of-00101.safetensors", + "model.layers.23.mlp.experts.103.up_proj.weight": "model-00024-of-00101.safetensors", + "model.layers.23.mlp.experts.103.down_proj.weight": "model-00024-of-00101.safetensors", + "model.layers.23.mlp.experts.104.gate_proj.weight": "model-00024-of-00101.safetensors", + "model.layers.23.mlp.experts.104.up_proj.weight": "model-00024-of-00101.safetensors", + "model.layers.23.mlp.experts.104.down_proj.weight": "model-00024-of-00101.safetensors", + "model.layers.23.mlp.experts.105.gate_proj.weight": "model-00024-of-00101.safetensors", + "model.layers.23.mlp.experts.105.up_proj.weight": "model-00024-of-00101.safetensors", + "model.layers.23.mlp.experts.105.down_proj.weight": "model-00024-of-00101.safetensors", + "model.layers.23.mlp.experts.106.gate_proj.weight": "model-00024-of-00101.safetensors", + "model.layers.23.mlp.experts.106.up_proj.weight": "model-00024-of-00101.safetensors", + "model.layers.23.mlp.experts.106.down_proj.weight": "model-00024-of-00101.safetensors", + "model.layers.23.mlp.experts.107.gate_proj.weight": "model-00024-of-00101.safetensors", + "model.layers.23.mlp.experts.107.up_proj.weight": "model-00024-of-00101.safetensors", + "model.layers.23.mlp.experts.107.down_proj.weight": "model-00025-of-00101.safetensors", + "model.layers.23.mlp.experts.108.gate_proj.weight": "model-00025-of-00101.safetensors", + "model.layers.23.mlp.experts.108.up_proj.weight": "model-00025-of-00101.safetensors", + "model.layers.23.mlp.experts.108.down_proj.weight": "model-00025-of-00101.safetensors", + "model.layers.23.mlp.experts.109.gate_proj.weight": "model-00025-of-00101.safetensors", + "model.layers.23.mlp.experts.109.up_proj.weight": "model-00025-of-00101.safetensors", + "model.layers.23.mlp.experts.109.down_proj.weight": "model-00025-of-00101.safetensors", + "model.layers.23.mlp.experts.110.gate_proj.weight": "model-00025-of-00101.safetensors", + "model.layers.23.mlp.experts.110.up_proj.weight": "model-00025-of-00101.safetensors", + "model.layers.23.mlp.experts.110.down_proj.weight": "model-00025-of-00101.safetensors", + "model.layers.23.mlp.experts.111.gate_proj.weight": "model-00025-of-00101.safetensors", + "model.layers.23.mlp.experts.111.up_proj.weight": "model-00025-of-00101.safetensors", + "model.layers.23.mlp.experts.111.down_proj.weight": "model-00025-of-00101.safetensors", + "model.layers.23.mlp.experts.112.gate_proj.weight": "model-00025-of-00101.safetensors", + "model.layers.23.mlp.experts.112.up_proj.weight": "model-00025-of-00101.safetensors", + "model.layers.23.mlp.experts.112.down_proj.weight": "model-00025-of-00101.safetensors", + "model.layers.23.mlp.experts.113.gate_proj.weight": "model-00025-of-00101.safetensors", + "model.layers.23.mlp.experts.113.up_proj.weight": "model-00025-of-00101.safetensors", + "model.layers.23.mlp.experts.113.down_proj.weight": "model-00025-of-00101.safetensors", + "model.layers.23.mlp.experts.114.gate_proj.weight": "model-00025-of-00101.safetensors", + "model.layers.23.mlp.experts.114.up_proj.weight": "model-00025-of-00101.safetensors", + "model.layers.23.mlp.experts.114.down_proj.weight": "model-00025-of-00101.safetensors", + "model.layers.23.mlp.experts.115.gate_proj.weight": "model-00025-of-00101.safetensors", + "model.layers.23.mlp.experts.115.up_proj.weight": "model-00025-of-00101.safetensors", + "model.layers.23.mlp.experts.115.down_proj.weight": "model-00025-of-00101.safetensors", + "model.layers.23.mlp.experts.116.gate_proj.weight": "model-00025-of-00101.safetensors", + "model.layers.23.mlp.experts.116.up_proj.weight": "model-00025-of-00101.safetensors", + "model.layers.23.mlp.experts.116.down_proj.weight": "model-00025-of-00101.safetensors", + "model.layers.23.mlp.experts.117.gate_proj.weight": "model-00025-of-00101.safetensors", + "model.layers.23.mlp.experts.117.up_proj.weight": "model-00025-of-00101.safetensors", + "model.layers.23.mlp.experts.117.down_proj.weight": "model-00025-of-00101.safetensors", + "model.layers.23.mlp.experts.118.gate_proj.weight": "model-00025-of-00101.safetensors", + "model.layers.23.mlp.experts.118.up_proj.weight": "model-00025-of-00101.safetensors", + "model.layers.23.mlp.experts.118.down_proj.weight": "model-00025-of-00101.safetensors", + "model.layers.23.mlp.experts.119.gate_proj.weight": "model-00025-of-00101.safetensors", + "model.layers.23.mlp.experts.119.up_proj.weight": "model-00025-of-00101.safetensors", + "model.layers.23.mlp.experts.119.down_proj.weight": "model-00025-of-00101.safetensors", + "model.layers.23.mlp.gate.weight": "model-00025-of-00101.safetensors", + "model.layers.23.mlp.gate.e_score_correction_bias": "model-00025-of-00101.safetensors", + "model.layers.23.mlp.shared_experts.gate_proj.weight": "model-00025-of-00101.safetensors", + "model.layers.23.mlp.shared_experts.up_proj.weight": "model-00025-of-00101.safetensors", + "model.layers.23.mlp.shared_experts.down_proj.weight": "model-00025-of-00101.safetensors", + "model.layers.23.input_layernorm.weight": "model-00025-of-00101.safetensors", + "model.layers.23.post_attention_layernorm.weight": "model-00025-of-00101.safetensors", + "model.layers.24.self_attn.q_proj.weight": "model-00025-of-00101.safetensors", + "model.layers.24.self_attn.q_proj.bias": "model-00025-of-00101.safetensors", + "model.layers.24.self_attn.k_proj.weight": "model-00025-of-00101.safetensors", + "model.layers.24.self_attn.k_proj.bias": "model-00025-of-00101.safetensors", + "model.layers.24.self_attn.v_proj.weight": "model-00025-of-00101.safetensors", + "model.layers.24.self_attn.v_proj.bias": "model-00025-of-00101.safetensors", + "model.layers.24.self_attn.o_proj.weight": "model-00025-of-00101.safetensors", + "model.layers.24.self_attn.q_norm.weight": "model-00025-of-00101.safetensors", + "model.layers.24.self_attn.k_norm.weight": "model-00025-of-00101.safetensors", + "model.layers.24.mlp.experts.0.gate_proj.weight": "model-00025-of-00101.safetensors", + "model.layers.24.mlp.experts.0.up_proj.weight": "model-00025-of-00101.safetensors", + "model.layers.24.mlp.experts.0.down_proj.weight": "model-00025-of-00101.safetensors", + "model.layers.24.mlp.experts.1.gate_proj.weight": "model-00025-of-00101.safetensors", + "model.layers.24.mlp.experts.1.up_proj.weight": "model-00025-of-00101.safetensors", + "model.layers.24.mlp.experts.1.down_proj.weight": "model-00025-of-00101.safetensors", + "model.layers.24.mlp.experts.2.gate_proj.weight": "model-00025-of-00101.safetensors", + "model.layers.24.mlp.experts.2.up_proj.weight": "model-00025-of-00101.safetensors", + "model.layers.24.mlp.experts.2.down_proj.weight": "model-00025-of-00101.safetensors", + "model.layers.24.mlp.experts.3.gate_proj.weight": "model-00025-of-00101.safetensors", + "model.layers.24.mlp.experts.3.up_proj.weight": "model-00025-of-00101.safetensors", + "model.layers.24.mlp.experts.3.down_proj.weight": "model-00025-of-00101.safetensors", + "model.layers.24.mlp.experts.4.gate_proj.weight": "model-00025-of-00101.safetensors", + "model.layers.24.mlp.experts.4.up_proj.weight": "model-00025-of-00101.safetensors", + "model.layers.24.mlp.experts.4.down_proj.weight": "model-00025-of-00101.safetensors", + "model.layers.24.mlp.experts.5.gate_proj.weight": "model-00025-of-00101.safetensors", + "model.layers.24.mlp.experts.5.up_proj.weight": "model-00025-of-00101.safetensors", + "model.layers.24.mlp.experts.5.down_proj.weight": "model-00025-of-00101.safetensors", + "model.layers.24.mlp.experts.6.gate_proj.weight": "model-00025-of-00101.safetensors", + "model.layers.24.mlp.experts.6.up_proj.weight": "model-00025-of-00101.safetensors", + "model.layers.24.mlp.experts.6.down_proj.weight": "model-00025-of-00101.safetensors", + "model.layers.24.mlp.experts.7.gate_proj.weight": "model-00025-of-00101.safetensors", + "model.layers.24.mlp.experts.7.up_proj.weight": "model-00025-of-00101.safetensors", + "model.layers.24.mlp.experts.7.down_proj.weight": "model-00025-of-00101.safetensors", + "model.layers.24.mlp.experts.8.gate_proj.weight": "model-00025-of-00101.safetensors", + "model.layers.24.mlp.experts.8.up_proj.weight": "model-00025-of-00101.safetensors", + "model.layers.24.mlp.experts.8.down_proj.weight": "model-00025-of-00101.safetensors", + "model.layers.24.mlp.experts.9.gate_proj.weight": "model-00025-of-00101.safetensors", + "model.layers.24.mlp.experts.9.up_proj.weight": "model-00025-of-00101.safetensors", + "model.layers.24.mlp.experts.9.down_proj.weight": "model-00025-of-00101.safetensors", + "model.layers.24.mlp.experts.10.gate_proj.weight": "model-00025-of-00101.safetensors", + "model.layers.24.mlp.experts.10.up_proj.weight": "model-00025-of-00101.safetensors", + "model.layers.24.mlp.experts.10.down_proj.weight": "model-00025-of-00101.safetensors", + "model.layers.24.mlp.experts.11.gate_proj.weight": "model-00025-of-00101.safetensors", + "model.layers.24.mlp.experts.11.up_proj.weight": "model-00025-of-00101.safetensors", + "model.layers.24.mlp.experts.11.down_proj.weight": "model-00025-of-00101.safetensors", + "model.layers.24.mlp.experts.12.gate_proj.weight": "model-00025-of-00101.safetensors", + "model.layers.24.mlp.experts.12.up_proj.weight": "model-00025-of-00101.safetensors", + "model.layers.24.mlp.experts.12.down_proj.weight": "model-00025-of-00101.safetensors", + "model.layers.24.mlp.experts.13.gate_proj.weight": "model-00025-of-00101.safetensors", + "model.layers.24.mlp.experts.13.up_proj.weight": "model-00025-of-00101.safetensors", + "model.layers.24.mlp.experts.13.down_proj.weight": "model-00025-of-00101.safetensors", + "model.layers.24.mlp.experts.14.gate_proj.weight": "model-00025-of-00101.safetensors", + "model.layers.24.mlp.experts.14.up_proj.weight": "model-00025-of-00101.safetensors", + "model.layers.24.mlp.experts.14.down_proj.weight": "model-00025-of-00101.safetensors", + "model.layers.24.mlp.experts.15.gate_proj.weight": "model-00025-of-00101.safetensors", + "model.layers.24.mlp.experts.15.up_proj.weight": "model-00025-of-00101.safetensors", + "model.layers.24.mlp.experts.15.down_proj.weight": "model-00025-of-00101.safetensors", + "model.layers.24.mlp.experts.16.gate_proj.weight": "model-00025-of-00101.safetensors", + "model.layers.24.mlp.experts.16.up_proj.weight": "model-00025-of-00101.safetensors", + "model.layers.24.mlp.experts.16.down_proj.weight": "model-00025-of-00101.safetensors", + "model.layers.24.mlp.experts.17.gate_proj.weight": "model-00025-of-00101.safetensors", + "model.layers.24.mlp.experts.17.up_proj.weight": "model-00025-of-00101.safetensors", + "model.layers.24.mlp.experts.17.down_proj.weight": "model-00025-of-00101.safetensors", + "model.layers.24.mlp.experts.18.gate_proj.weight": "model-00025-of-00101.safetensors", + "model.layers.24.mlp.experts.18.up_proj.weight": "model-00025-of-00101.safetensors", + "model.layers.24.mlp.experts.18.down_proj.weight": "model-00025-of-00101.safetensors", + "model.layers.24.mlp.experts.19.gate_proj.weight": "model-00025-of-00101.safetensors", + "model.layers.24.mlp.experts.19.up_proj.weight": "model-00025-of-00101.safetensors", + "model.layers.24.mlp.experts.19.down_proj.weight": "model-00025-of-00101.safetensors", + "model.layers.24.mlp.experts.20.gate_proj.weight": "model-00025-of-00101.safetensors", + "model.layers.24.mlp.experts.20.up_proj.weight": "model-00025-of-00101.safetensors", + "model.layers.24.mlp.experts.20.down_proj.weight": "model-00025-of-00101.safetensors", + "model.layers.24.mlp.experts.21.gate_proj.weight": "model-00025-of-00101.safetensors", + "model.layers.24.mlp.experts.21.up_proj.weight": "model-00025-of-00101.safetensors", + "model.layers.24.mlp.experts.21.down_proj.weight": "model-00025-of-00101.safetensors", + "model.layers.24.mlp.experts.22.gate_proj.weight": "model-00025-of-00101.safetensors", + "model.layers.24.mlp.experts.22.up_proj.weight": "model-00025-of-00101.safetensors", + "model.layers.24.mlp.experts.22.down_proj.weight": "model-00025-of-00101.safetensors", + "model.layers.24.mlp.experts.23.gate_proj.weight": "model-00025-of-00101.safetensors", + "model.layers.24.mlp.experts.23.up_proj.weight": "model-00025-of-00101.safetensors", + "model.layers.24.mlp.experts.23.down_proj.weight": "model-00025-of-00101.safetensors", + "model.layers.24.mlp.experts.24.gate_proj.weight": "model-00025-of-00101.safetensors", + "model.layers.24.mlp.experts.24.up_proj.weight": "model-00025-of-00101.safetensors", + "model.layers.24.mlp.experts.24.down_proj.weight": "model-00025-of-00101.safetensors", + "model.layers.24.mlp.experts.25.gate_proj.weight": "model-00025-of-00101.safetensors", + "model.layers.24.mlp.experts.25.up_proj.weight": "model-00025-of-00101.safetensors", + "model.layers.24.mlp.experts.25.down_proj.weight": "model-00025-of-00101.safetensors", + "model.layers.24.mlp.experts.26.gate_proj.weight": "model-00025-of-00101.safetensors", + "model.layers.24.mlp.experts.26.up_proj.weight": "model-00025-of-00101.safetensors", + "model.layers.24.mlp.experts.26.down_proj.weight": "model-00025-of-00101.safetensors", + "model.layers.24.mlp.experts.27.gate_proj.weight": "model-00025-of-00101.safetensors", + "model.layers.24.mlp.experts.27.up_proj.weight": "model-00025-of-00101.safetensors", + "model.layers.24.mlp.experts.27.down_proj.weight": "model-00025-of-00101.safetensors", + "model.layers.24.mlp.experts.28.gate_proj.weight": "model-00025-of-00101.safetensors", + "model.layers.24.mlp.experts.28.up_proj.weight": "model-00025-of-00101.safetensors", + "model.layers.24.mlp.experts.28.down_proj.weight": "model-00025-of-00101.safetensors", + "model.layers.24.mlp.experts.29.gate_proj.weight": "model-00025-of-00101.safetensors", + "model.layers.24.mlp.experts.29.up_proj.weight": "model-00025-of-00101.safetensors", + "model.layers.24.mlp.experts.29.down_proj.weight": "model-00025-of-00101.safetensors", + "model.layers.24.mlp.experts.30.gate_proj.weight": "model-00025-of-00101.safetensors", + "model.layers.24.mlp.experts.30.up_proj.weight": "model-00025-of-00101.safetensors", + "model.layers.24.mlp.experts.30.down_proj.weight": "model-00025-of-00101.safetensors", + "model.layers.24.mlp.experts.31.gate_proj.weight": "model-00025-of-00101.safetensors", + "model.layers.24.mlp.experts.31.up_proj.weight": "model-00025-of-00101.safetensors", + "model.layers.24.mlp.experts.31.down_proj.weight": "model-00025-of-00101.safetensors", + "model.layers.24.mlp.experts.32.gate_proj.weight": "model-00025-of-00101.safetensors", + "model.layers.24.mlp.experts.32.up_proj.weight": "model-00025-of-00101.safetensors", + "model.layers.24.mlp.experts.32.down_proj.weight": "model-00025-of-00101.safetensors", + "model.layers.24.mlp.experts.33.gate_proj.weight": "model-00025-of-00101.safetensors", + "model.layers.24.mlp.experts.33.up_proj.weight": "model-00025-of-00101.safetensors", + "model.layers.24.mlp.experts.33.down_proj.weight": "model-00025-of-00101.safetensors", + "model.layers.24.mlp.experts.34.gate_proj.weight": "model-00025-of-00101.safetensors", + "model.layers.24.mlp.experts.34.up_proj.weight": "model-00025-of-00101.safetensors", + "model.layers.24.mlp.experts.34.down_proj.weight": "model-00025-of-00101.safetensors", + "model.layers.24.mlp.experts.35.gate_proj.weight": "model-00025-of-00101.safetensors", + "model.layers.24.mlp.experts.35.up_proj.weight": "model-00025-of-00101.safetensors", + "model.layers.24.mlp.experts.35.down_proj.weight": "model-00025-of-00101.safetensors", + "model.layers.24.mlp.experts.36.gate_proj.weight": "model-00025-of-00101.safetensors", + "model.layers.24.mlp.experts.36.up_proj.weight": "model-00025-of-00101.safetensors", + "model.layers.24.mlp.experts.36.down_proj.weight": "model-00025-of-00101.safetensors", + "model.layers.24.mlp.experts.37.gate_proj.weight": "model-00025-of-00101.safetensors", + "model.layers.24.mlp.experts.37.up_proj.weight": "model-00025-of-00101.safetensors", + "model.layers.24.mlp.experts.37.down_proj.weight": "model-00025-of-00101.safetensors", + "model.layers.24.mlp.experts.38.gate_proj.weight": "model-00025-of-00101.safetensors", + "model.layers.24.mlp.experts.38.up_proj.weight": "model-00025-of-00101.safetensors", + "model.layers.24.mlp.experts.38.down_proj.weight": "model-00025-of-00101.safetensors", + "model.layers.24.mlp.experts.39.gate_proj.weight": "model-00025-of-00101.safetensors", + "model.layers.24.mlp.experts.39.up_proj.weight": "model-00025-of-00101.safetensors", + "model.layers.24.mlp.experts.39.down_proj.weight": "model-00025-of-00101.safetensors", + "model.layers.24.mlp.experts.40.gate_proj.weight": "model-00025-of-00101.safetensors", + "model.layers.24.mlp.experts.40.up_proj.weight": "model-00025-of-00101.safetensors", + "model.layers.24.mlp.experts.40.down_proj.weight": "model-00025-of-00101.safetensors", + "model.layers.24.mlp.experts.41.gate_proj.weight": "model-00025-of-00101.safetensors", + "model.layers.24.mlp.experts.41.up_proj.weight": "model-00025-of-00101.safetensors", + "model.layers.24.mlp.experts.41.down_proj.weight": "model-00025-of-00101.safetensors", + "model.layers.24.mlp.experts.42.gate_proj.weight": "model-00025-of-00101.safetensors", + "model.layers.24.mlp.experts.42.up_proj.weight": "model-00025-of-00101.safetensors", + "model.layers.24.mlp.experts.42.down_proj.weight": "model-00025-of-00101.safetensors", + "model.layers.24.mlp.experts.43.gate_proj.weight": "model-00025-of-00101.safetensors", + "model.layers.24.mlp.experts.43.up_proj.weight": "model-00025-of-00101.safetensors", + "model.layers.24.mlp.experts.43.down_proj.weight": "model-00025-of-00101.safetensors", + "model.layers.24.mlp.experts.44.gate_proj.weight": "model-00025-of-00101.safetensors", + "model.layers.24.mlp.experts.44.up_proj.weight": "model-00025-of-00101.safetensors", + "model.layers.24.mlp.experts.44.down_proj.weight": "model-00025-of-00101.safetensors", + "model.layers.24.mlp.experts.45.gate_proj.weight": "model-00025-of-00101.safetensors", + "model.layers.24.mlp.experts.45.up_proj.weight": "model-00025-of-00101.safetensors", + "model.layers.24.mlp.experts.45.down_proj.weight": "model-00025-of-00101.safetensors", + "model.layers.24.mlp.experts.46.gate_proj.weight": "model-00025-of-00101.safetensors", + "model.layers.24.mlp.experts.46.up_proj.weight": "model-00025-of-00101.safetensors", + "model.layers.24.mlp.experts.46.down_proj.weight": "model-00025-of-00101.safetensors", + "model.layers.24.mlp.experts.47.gate_proj.weight": "model-00025-of-00101.safetensors", + "model.layers.24.mlp.experts.47.up_proj.weight": "model-00025-of-00101.safetensors", + "model.layers.24.mlp.experts.47.down_proj.weight": "model-00025-of-00101.safetensors", + "model.layers.24.mlp.experts.48.gate_proj.weight": "model-00025-of-00101.safetensors", + "model.layers.24.mlp.experts.48.up_proj.weight": "model-00025-of-00101.safetensors", + "model.layers.24.mlp.experts.48.down_proj.weight": "model-00025-of-00101.safetensors", + "model.layers.24.mlp.experts.49.gate_proj.weight": "model-00025-of-00101.safetensors", + "model.layers.24.mlp.experts.49.up_proj.weight": "model-00025-of-00101.safetensors", + "model.layers.24.mlp.experts.49.down_proj.weight": "model-00025-of-00101.safetensors", + "model.layers.24.mlp.experts.50.gate_proj.weight": "model-00025-of-00101.safetensors", + "model.layers.24.mlp.experts.50.up_proj.weight": "model-00025-of-00101.safetensors", + "model.layers.24.mlp.experts.50.down_proj.weight": "model-00025-of-00101.safetensors", + "model.layers.24.mlp.experts.51.gate_proj.weight": "model-00025-of-00101.safetensors", + "model.layers.24.mlp.experts.51.up_proj.weight": "model-00025-of-00101.safetensors", + "model.layers.24.mlp.experts.51.down_proj.weight": "model-00025-of-00101.safetensors", + "model.layers.24.mlp.experts.52.gate_proj.weight": "model-00025-of-00101.safetensors", + "model.layers.24.mlp.experts.52.up_proj.weight": "model-00025-of-00101.safetensors", + "model.layers.24.mlp.experts.52.down_proj.weight": "model-00025-of-00101.safetensors", + "model.layers.24.mlp.experts.53.gate_proj.weight": "model-00025-of-00101.safetensors", + "model.layers.24.mlp.experts.53.up_proj.weight": "model-00025-of-00101.safetensors", + "model.layers.24.mlp.experts.53.down_proj.weight": "model-00025-of-00101.safetensors", + "model.layers.24.mlp.experts.54.gate_proj.weight": "model-00025-of-00101.safetensors", + "model.layers.24.mlp.experts.54.up_proj.weight": "model-00025-of-00101.safetensors", + "model.layers.24.mlp.experts.54.down_proj.weight": "model-00025-of-00101.safetensors", + "model.layers.24.mlp.experts.55.gate_proj.weight": "model-00025-of-00101.safetensors", + "model.layers.24.mlp.experts.55.up_proj.weight": "model-00025-of-00101.safetensors", + "model.layers.24.mlp.experts.55.down_proj.weight": "model-00025-of-00101.safetensors", + "model.layers.24.mlp.experts.56.gate_proj.weight": "model-00025-of-00101.safetensors", + "model.layers.24.mlp.experts.56.up_proj.weight": "model-00025-of-00101.safetensors", + "model.layers.24.mlp.experts.56.down_proj.weight": "model-00025-of-00101.safetensors", + "model.layers.24.mlp.experts.57.gate_proj.weight": "model-00025-of-00101.safetensors", + "model.layers.24.mlp.experts.57.up_proj.weight": "model-00025-of-00101.safetensors", + "model.layers.24.mlp.experts.57.down_proj.weight": "model-00025-of-00101.safetensors", + "model.layers.24.mlp.experts.58.gate_proj.weight": "model-00025-of-00101.safetensors", + "model.layers.24.mlp.experts.58.up_proj.weight": "model-00025-of-00101.safetensors", + "model.layers.24.mlp.experts.58.down_proj.weight": "model-00025-of-00101.safetensors", + "model.layers.24.mlp.experts.59.gate_proj.weight": "model-00025-of-00101.safetensors", + "model.layers.24.mlp.experts.59.up_proj.weight": "model-00025-of-00101.safetensors", + "model.layers.24.mlp.experts.59.down_proj.weight": "model-00025-of-00101.safetensors", + "model.layers.24.mlp.experts.60.gate_proj.weight": "model-00025-of-00101.safetensors", + "model.layers.24.mlp.experts.60.up_proj.weight": "model-00025-of-00101.safetensors", + "model.layers.24.mlp.experts.60.down_proj.weight": "model-00025-of-00101.safetensors", + "model.layers.24.mlp.experts.61.gate_proj.weight": "model-00025-of-00101.safetensors", + "model.layers.24.mlp.experts.61.up_proj.weight": "model-00025-of-00101.safetensors", + "model.layers.24.mlp.experts.61.down_proj.weight": "model-00025-of-00101.safetensors", + "model.layers.24.mlp.experts.62.gate_proj.weight": "model-00025-of-00101.safetensors", + "model.layers.24.mlp.experts.62.up_proj.weight": "model-00025-of-00101.safetensors", + "model.layers.24.mlp.experts.62.down_proj.weight": "model-00025-of-00101.safetensors", + "model.layers.24.mlp.experts.63.gate_proj.weight": "model-00025-of-00101.safetensors", + "model.layers.24.mlp.experts.63.up_proj.weight": "model-00025-of-00101.safetensors", + "model.layers.24.mlp.experts.63.down_proj.weight": "model-00025-of-00101.safetensors", + "model.layers.24.mlp.experts.64.gate_proj.weight": "model-00025-of-00101.safetensors", + "model.layers.24.mlp.experts.64.up_proj.weight": "model-00025-of-00101.safetensors", + "model.layers.24.mlp.experts.64.down_proj.weight": "model-00025-of-00101.safetensors", + "model.layers.24.mlp.experts.65.gate_proj.weight": "model-00025-of-00101.safetensors", + "model.layers.24.mlp.experts.65.up_proj.weight": "model-00025-of-00101.safetensors", + "model.layers.24.mlp.experts.65.down_proj.weight": "model-00025-of-00101.safetensors", + "model.layers.24.mlp.experts.66.gate_proj.weight": "model-00025-of-00101.safetensors", + "model.layers.24.mlp.experts.66.up_proj.weight": "model-00025-of-00101.safetensors", + "model.layers.24.mlp.experts.66.down_proj.weight": "model-00025-of-00101.safetensors", + "model.layers.24.mlp.experts.67.gate_proj.weight": "model-00025-of-00101.safetensors", + "model.layers.24.mlp.experts.67.up_proj.weight": "model-00025-of-00101.safetensors", + "model.layers.24.mlp.experts.67.down_proj.weight": "model-00025-of-00101.safetensors", + "model.layers.24.mlp.experts.68.gate_proj.weight": "model-00025-of-00101.safetensors", + "model.layers.24.mlp.experts.68.up_proj.weight": "model-00025-of-00101.safetensors", + "model.layers.24.mlp.experts.68.down_proj.weight": "model-00025-of-00101.safetensors", + "model.layers.24.mlp.experts.69.gate_proj.weight": "model-00025-of-00101.safetensors", + "model.layers.24.mlp.experts.69.up_proj.weight": "model-00025-of-00101.safetensors", + "model.layers.24.mlp.experts.69.down_proj.weight": "model-00025-of-00101.safetensors", + "model.layers.24.mlp.experts.70.gate_proj.weight": "model-00025-of-00101.safetensors", + "model.layers.24.mlp.experts.70.up_proj.weight": "model-00025-of-00101.safetensors", + "model.layers.24.mlp.experts.70.down_proj.weight": "model-00025-of-00101.safetensors", + "model.layers.24.mlp.experts.71.gate_proj.weight": "model-00025-of-00101.safetensors", + "model.layers.24.mlp.experts.71.up_proj.weight": "model-00025-of-00101.safetensors", + "model.layers.24.mlp.experts.71.down_proj.weight": "model-00025-of-00101.safetensors", + "model.layers.24.mlp.experts.72.gate_proj.weight": "model-00025-of-00101.safetensors", + "model.layers.24.mlp.experts.72.up_proj.weight": "model-00025-of-00101.safetensors", + "model.layers.24.mlp.experts.72.down_proj.weight": "model-00025-of-00101.safetensors", + "model.layers.24.mlp.experts.73.gate_proj.weight": "model-00025-of-00101.safetensors", + "model.layers.24.mlp.experts.73.up_proj.weight": "model-00025-of-00101.safetensors", + "model.layers.24.mlp.experts.73.down_proj.weight": "model-00025-of-00101.safetensors", + "model.layers.24.mlp.experts.74.gate_proj.weight": "model-00025-of-00101.safetensors", + "model.layers.24.mlp.experts.74.up_proj.weight": "model-00025-of-00101.safetensors", + "model.layers.24.mlp.experts.74.down_proj.weight": "model-00025-of-00101.safetensors", + "model.layers.24.mlp.experts.75.gate_proj.weight": "model-00025-of-00101.safetensors", + "model.layers.24.mlp.experts.75.up_proj.weight": "model-00025-of-00101.safetensors", + "model.layers.24.mlp.experts.75.down_proj.weight": "model-00025-of-00101.safetensors", + "model.layers.24.mlp.experts.76.gate_proj.weight": "model-00025-of-00101.safetensors", + "model.layers.24.mlp.experts.76.up_proj.weight": "model-00025-of-00101.safetensors", + "model.layers.24.mlp.experts.76.down_proj.weight": "model-00025-of-00101.safetensors", + "model.layers.24.mlp.experts.77.gate_proj.weight": "model-00025-of-00101.safetensors", + "model.layers.24.mlp.experts.77.up_proj.weight": "model-00025-of-00101.safetensors", + "model.layers.24.mlp.experts.77.down_proj.weight": "model-00025-of-00101.safetensors", + "model.layers.24.mlp.experts.78.gate_proj.weight": "model-00025-of-00101.safetensors", + "model.layers.24.mlp.experts.78.up_proj.weight": "model-00025-of-00101.safetensors", + "model.layers.24.mlp.experts.78.down_proj.weight": "model-00025-of-00101.safetensors", + "model.layers.24.mlp.experts.79.gate_proj.weight": "model-00025-of-00101.safetensors", + "model.layers.24.mlp.experts.79.up_proj.weight": "model-00025-of-00101.safetensors", + "model.layers.24.mlp.experts.79.down_proj.weight": "model-00025-of-00101.safetensors", + "model.layers.24.mlp.experts.80.gate_proj.weight": "model-00025-of-00101.safetensors", + "model.layers.24.mlp.experts.80.up_proj.weight": "model-00025-of-00101.safetensors", + "model.layers.24.mlp.experts.80.down_proj.weight": "model-00025-of-00101.safetensors", + "model.layers.24.mlp.experts.81.gate_proj.weight": "model-00025-of-00101.safetensors", + "model.layers.24.mlp.experts.81.up_proj.weight": "model-00025-of-00101.safetensors", + "model.layers.24.mlp.experts.81.down_proj.weight": "model-00025-of-00101.safetensors", + "model.layers.24.mlp.experts.82.gate_proj.weight": "model-00025-of-00101.safetensors", + "model.layers.24.mlp.experts.82.up_proj.weight": "model-00025-of-00101.safetensors", + "model.layers.24.mlp.experts.82.down_proj.weight": "model-00025-of-00101.safetensors", + "model.layers.24.mlp.experts.83.gate_proj.weight": "model-00025-of-00101.safetensors", + "model.layers.24.mlp.experts.83.up_proj.weight": "model-00025-of-00101.safetensors", + "model.layers.24.mlp.experts.83.down_proj.weight": "model-00025-of-00101.safetensors", + "model.layers.24.mlp.experts.84.gate_proj.weight": "model-00025-of-00101.safetensors", + "model.layers.24.mlp.experts.84.up_proj.weight": "model-00025-of-00101.safetensors", + "model.layers.24.mlp.experts.84.down_proj.weight": "model-00025-of-00101.safetensors", + "model.layers.24.mlp.experts.85.gate_proj.weight": "model-00025-of-00101.safetensors", + "model.layers.24.mlp.experts.85.up_proj.weight": "model-00025-of-00101.safetensors", + "model.layers.24.mlp.experts.85.down_proj.weight": "model-00025-of-00101.safetensors", + "model.layers.24.mlp.experts.86.gate_proj.weight": "model-00025-of-00101.safetensors", + "model.layers.24.mlp.experts.86.up_proj.weight": "model-00025-of-00101.safetensors", + "model.layers.24.mlp.experts.86.down_proj.weight": "model-00025-of-00101.safetensors", + "model.layers.24.mlp.experts.87.gate_proj.weight": "model-00025-of-00101.safetensors", + "model.layers.24.mlp.experts.87.up_proj.weight": "model-00025-of-00101.safetensors", + "model.layers.24.mlp.experts.87.down_proj.weight": "model-00025-of-00101.safetensors", + "model.layers.24.mlp.experts.88.gate_proj.weight": "model-00025-of-00101.safetensors", + "model.layers.24.mlp.experts.88.up_proj.weight": "model-00025-of-00101.safetensors", + "model.layers.24.mlp.experts.88.down_proj.weight": "model-00025-of-00101.safetensors", + "model.layers.24.mlp.experts.89.gate_proj.weight": "model-00025-of-00101.safetensors", + "model.layers.24.mlp.experts.89.up_proj.weight": "model-00025-of-00101.safetensors", + "model.layers.24.mlp.experts.89.down_proj.weight": "model-00025-of-00101.safetensors", + "model.layers.24.mlp.experts.90.gate_proj.weight": "model-00025-of-00101.safetensors", + "model.layers.24.mlp.experts.90.up_proj.weight": "model-00025-of-00101.safetensors", + "model.layers.24.mlp.experts.90.down_proj.weight": "model-00025-of-00101.safetensors", + "model.layers.24.mlp.experts.91.gate_proj.weight": "model-00025-of-00101.safetensors", + "model.layers.24.mlp.experts.91.up_proj.weight": "model-00025-of-00101.safetensors", + "model.layers.24.mlp.experts.91.down_proj.weight": "model-00025-of-00101.safetensors", + "model.layers.24.mlp.experts.92.gate_proj.weight": "model-00025-of-00101.safetensors", + "model.layers.24.mlp.experts.92.up_proj.weight": "model-00025-of-00101.safetensors", + "model.layers.24.mlp.experts.92.down_proj.weight": "model-00025-of-00101.safetensors", + "model.layers.24.mlp.experts.93.gate_proj.weight": "model-00025-of-00101.safetensors", + "model.layers.24.mlp.experts.93.up_proj.weight": "model-00025-of-00101.safetensors", + "model.layers.24.mlp.experts.93.down_proj.weight": "model-00025-of-00101.safetensors", + "model.layers.24.mlp.experts.94.gate_proj.weight": "model-00025-of-00101.safetensors", + "model.layers.24.mlp.experts.94.up_proj.weight": "model-00026-of-00101.safetensors", + "model.layers.24.mlp.experts.94.down_proj.weight": "model-00026-of-00101.safetensors", + "model.layers.24.mlp.experts.95.gate_proj.weight": "model-00026-of-00101.safetensors", + "model.layers.24.mlp.experts.95.up_proj.weight": "model-00026-of-00101.safetensors", + "model.layers.24.mlp.experts.95.down_proj.weight": "model-00026-of-00101.safetensors", + "model.layers.24.mlp.experts.96.gate_proj.weight": "model-00026-of-00101.safetensors", + "model.layers.24.mlp.experts.96.up_proj.weight": "model-00026-of-00101.safetensors", + "model.layers.24.mlp.experts.96.down_proj.weight": "model-00026-of-00101.safetensors", + "model.layers.24.mlp.experts.97.gate_proj.weight": "model-00026-of-00101.safetensors", + "model.layers.24.mlp.experts.97.up_proj.weight": "model-00026-of-00101.safetensors", + "model.layers.24.mlp.experts.97.down_proj.weight": "model-00026-of-00101.safetensors", + "model.layers.24.mlp.experts.98.gate_proj.weight": "model-00026-of-00101.safetensors", + "model.layers.24.mlp.experts.98.up_proj.weight": "model-00026-of-00101.safetensors", + "model.layers.24.mlp.experts.98.down_proj.weight": "model-00026-of-00101.safetensors", + "model.layers.24.mlp.experts.99.gate_proj.weight": "model-00026-of-00101.safetensors", + "model.layers.24.mlp.experts.99.up_proj.weight": "model-00026-of-00101.safetensors", + "model.layers.24.mlp.experts.99.down_proj.weight": "model-00026-of-00101.safetensors", + "model.layers.24.mlp.experts.100.gate_proj.weight": "model-00026-of-00101.safetensors", + "model.layers.24.mlp.experts.100.up_proj.weight": "model-00026-of-00101.safetensors", + "model.layers.24.mlp.experts.100.down_proj.weight": "model-00026-of-00101.safetensors", + "model.layers.24.mlp.experts.101.gate_proj.weight": "model-00026-of-00101.safetensors", + "model.layers.24.mlp.experts.101.up_proj.weight": "model-00026-of-00101.safetensors", + "model.layers.24.mlp.experts.101.down_proj.weight": "model-00026-of-00101.safetensors", + "model.layers.24.mlp.experts.102.gate_proj.weight": "model-00026-of-00101.safetensors", + "model.layers.24.mlp.experts.102.up_proj.weight": "model-00026-of-00101.safetensors", + "model.layers.24.mlp.experts.102.down_proj.weight": "model-00026-of-00101.safetensors", + "model.layers.24.mlp.experts.103.gate_proj.weight": "model-00026-of-00101.safetensors", + "model.layers.24.mlp.experts.103.up_proj.weight": "model-00026-of-00101.safetensors", + "model.layers.24.mlp.experts.103.down_proj.weight": "model-00026-of-00101.safetensors", + "model.layers.24.mlp.experts.104.gate_proj.weight": "model-00026-of-00101.safetensors", + "model.layers.24.mlp.experts.104.up_proj.weight": "model-00026-of-00101.safetensors", + "model.layers.24.mlp.experts.104.down_proj.weight": "model-00026-of-00101.safetensors", + "model.layers.24.mlp.experts.105.gate_proj.weight": "model-00026-of-00101.safetensors", + "model.layers.24.mlp.experts.105.up_proj.weight": "model-00026-of-00101.safetensors", + "model.layers.24.mlp.experts.105.down_proj.weight": "model-00026-of-00101.safetensors", + "model.layers.24.mlp.experts.106.gate_proj.weight": "model-00026-of-00101.safetensors", + "model.layers.24.mlp.experts.106.up_proj.weight": "model-00026-of-00101.safetensors", + "model.layers.24.mlp.experts.106.down_proj.weight": "model-00026-of-00101.safetensors", + "model.layers.24.mlp.experts.107.gate_proj.weight": "model-00026-of-00101.safetensors", + "model.layers.24.mlp.experts.107.up_proj.weight": "model-00026-of-00101.safetensors", + "model.layers.24.mlp.experts.107.down_proj.weight": "model-00026-of-00101.safetensors", + "model.layers.24.mlp.experts.108.gate_proj.weight": "model-00026-of-00101.safetensors", + "model.layers.24.mlp.experts.108.up_proj.weight": "model-00026-of-00101.safetensors", + "model.layers.24.mlp.experts.108.down_proj.weight": "model-00026-of-00101.safetensors", + "model.layers.24.mlp.experts.109.gate_proj.weight": "model-00026-of-00101.safetensors", + "model.layers.24.mlp.experts.109.up_proj.weight": "model-00026-of-00101.safetensors", + "model.layers.24.mlp.experts.109.down_proj.weight": "model-00026-of-00101.safetensors", + "model.layers.24.mlp.experts.110.gate_proj.weight": "model-00026-of-00101.safetensors", + "model.layers.24.mlp.experts.110.up_proj.weight": "model-00026-of-00101.safetensors", + "model.layers.24.mlp.experts.110.down_proj.weight": "model-00026-of-00101.safetensors", + "model.layers.24.mlp.experts.111.gate_proj.weight": "model-00026-of-00101.safetensors", + "model.layers.24.mlp.experts.111.up_proj.weight": "model-00026-of-00101.safetensors", + "model.layers.24.mlp.experts.111.down_proj.weight": "model-00026-of-00101.safetensors", + "model.layers.24.mlp.experts.112.gate_proj.weight": "model-00026-of-00101.safetensors", + "model.layers.24.mlp.experts.112.up_proj.weight": "model-00026-of-00101.safetensors", + "model.layers.24.mlp.experts.112.down_proj.weight": "model-00026-of-00101.safetensors", + "model.layers.24.mlp.experts.113.gate_proj.weight": "model-00026-of-00101.safetensors", + "model.layers.24.mlp.experts.113.up_proj.weight": "model-00026-of-00101.safetensors", + "model.layers.24.mlp.experts.113.down_proj.weight": "model-00026-of-00101.safetensors", + "model.layers.24.mlp.experts.114.gate_proj.weight": "model-00026-of-00101.safetensors", + "model.layers.24.mlp.experts.114.up_proj.weight": "model-00026-of-00101.safetensors", + "model.layers.24.mlp.experts.114.down_proj.weight": "model-00026-of-00101.safetensors", + "model.layers.24.mlp.experts.115.gate_proj.weight": "model-00026-of-00101.safetensors", + "model.layers.24.mlp.experts.115.up_proj.weight": "model-00026-of-00101.safetensors", + "model.layers.24.mlp.experts.115.down_proj.weight": "model-00026-of-00101.safetensors", + "model.layers.24.mlp.experts.116.gate_proj.weight": "model-00026-of-00101.safetensors", + "model.layers.24.mlp.experts.116.up_proj.weight": "model-00026-of-00101.safetensors", + "model.layers.24.mlp.experts.116.down_proj.weight": "model-00026-of-00101.safetensors", + "model.layers.24.mlp.experts.117.gate_proj.weight": "model-00026-of-00101.safetensors", + "model.layers.24.mlp.experts.117.up_proj.weight": "model-00026-of-00101.safetensors", + "model.layers.24.mlp.experts.117.down_proj.weight": "model-00026-of-00101.safetensors", + "model.layers.24.mlp.experts.118.gate_proj.weight": "model-00026-of-00101.safetensors", + "model.layers.24.mlp.experts.118.up_proj.weight": "model-00026-of-00101.safetensors", + "model.layers.24.mlp.experts.118.down_proj.weight": "model-00026-of-00101.safetensors", + "model.layers.24.mlp.experts.119.gate_proj.weight": "model-00026-of-00101.safetensors", + "model.layers.24.mlp.experts.119.up_proj.weight": "model-00026-of-00101.safetensors", + "model.layers.24.mlp.experts.119.down_proj.weight": "model-00026-of-00101.safetensors", + "model.layers.24.mlp.gate.weight": "model-00026-of-00101.safetensors", + "model.layers.24.mlp.gate.e_score_correction_bias": "model-00026-of-00101.safetensors", + "model.layers.24.mlp.shared_experts.gate_proj.weight": "model-00026-of-00101.safetensors", + "model.layers.24.mlp.shared_experts.up_proj.weight": "model-00026-of-00101.safetensors", + "model.layers.24.mlp.shared_experts.down_proj.weight": "model-00026-of-00101.safetensors", + "model.layers.24.input_layernorm.weight": "model-00026-of-00101.safetensors", + "model.layers.24.post_attention_layernorm.weight": "model-00026-of-00101.safetensors", + "model.layers.25.self_attn.q_proj.weight": "model-00026-of-00101.safetensors", + "model.layers.25.self_attn.q_proj.bias": "model-00026-of-00101.safetensors", + "model.layers.25.self_attn.k_proj.weight": "model-00026-of-00101.safetensors", + "model.layers.25.self_attn.k_proj.bias": "model-00026-of-00101.safetensors", + "model.layers.25.self_attn.v_proj.weight": "model-00026-of-00101.safetensors", + "model.layers.25.self_attn.v_proj.bias": "model-00026-of-00101.safetensors", + "model.layers.25.self_attn.o_proj.weight": "model-00026-of-00101.safetensors", + "model.layers.25.self_attn.q_norm.weight": "model-00026-of-00101.safetensors", + "model.layers.25.self_attn.k_norm.weight": "model-00026-of-00101.safetensors", + "model.layers.25.mlp.experts.0.gate_proj.weight": "model-00026-of-00101.safetensors", + "model.layers.25.mlp.experts.0.up_proj.weight": "model-00026-of-00101.safetensors", + "model.layers.25.mlp.experts.0.down_proj.weight": "model-00026-of-00101.safetensors", + "model.layers.25.mlp.experts.1.gate_proj.weight": "model-00026-of-00101.safetensors", + "model.layers.25.mlp.experts.1.up_proj.weight": "model-00026-of-00101.safetensors", + "model.layers.25.mlp.experts.1.down_proj.weight": "model-00026-of-00101.safetensors", + "model.layers.25.mlp.experts.2.gate_proj.weight": "model-00026-of-00101.safetensors", + "model.layers.25.mlp.experts.2.up_proj.weight": "model-00026-of-00101.safetensors", + "model.layers.25.mlp.experts.2.down_proj.weight": "model-00026-of-00101.safetensors", + "model.layers.25.mlp.experts.3.gate_proj.weight": "model-00026-of-00101.safetensors", + "model.layers.25.mlp.experts.3.up_proj.weight": "model-00026-of-00101.safetensors", + "model.layers.25.mlp.experts.3.down_proj.weight": "model-00026-of-00101.safetensors", + "model.layers.25.mlp.experts.4.gate_proj.weight": "model-00026-of-00101.safetensors", + "model.layers.25.mlp.experts.4.up_proj.weight": "model-00026-of-00101.safetensors", + "model.layers.25.mlp.experts.4.down_proj.weight": "model-00026-of-00101.safetensors", + "model.layers.25.mlp.experts.5.gate_proj.weight": "model-00026-of-00101.safetensors", + "model.layers.25.mlp.experts.5.up_proj.weight": "model-00026-of-00101.safetensors", + "model.layers.25.mlp.experts.5.down_proj.weight": "model-00026-of-00101.safetensors", + "model.layers.25.mlp.experts.6.gate_proj.weight": "model-00026-of-00101.safetensors", + "model.layers.25.mlp.experts.6.up_proj.weight": "model-00026-of-00101.safetensors", + "model.layers.25.mlp.experts.6.down_proj.weight": "model-00026-of-00101.safetensors", + "model.layers.25.mlp.experts.7.gate_proj.weight": "model-00026-of-00101.safetensors", + "model.layers.25.mlp.experts.7.up_proj.weight": "model-00026-of-00101.safetensors", + "model.layers.25.mlp.experts.7.down_proj.weight": "model-00026-of-00101.safetensors", + "model.layers.25.mlp.experts.8.gate_proj.weight": "model-00026-of-00101.safetensors", + "model.layers.25.mlp.experts.8.up_proj.weight": "model-00026-of-00101.safetensors", + "model.layers.25.mlp.experts.8.down_proj.weight": "model-00026-of-00101.safetensors", + "model.layers.25.mlp.experts.9.gate_proj.weight": "model-00026-of-00101.safetensors", + "model.layers.25.mlp.experts.9.up_proj.weight": "model-00026-of-00101.safetensors", + "model.layers.25.mlp.experts.9.down_proj.weight": "model-00026-of-00101.safetensors", + "model.layers.25.mlp.experts.10.gate_proj.weight": "model-00026-of-00101.safetensors", + "model.layers.25.mlp.experts.10.up_proj.weight": "model-00026-of-00101.safetensors", + "model.layers.25.mlp.experts.10.down_proj.weight": "model-00026-of-00101.safetensors", + "model.layers.25.mlp.experts.11.gate_proj.weight": "model-00026-of-00101.safetensors", + "model.layers.25.mlp.experts.11.up_proj.weight": "model-00026-of-00101.safetensors", + "model.layers.25.mlp.experts.11.down_proj.weight": "model-00026-of-00101.safetensors", + "model.layers.25.mlp.experts.12.gate_proj.weight": "model-00026-of-00101.safetensors", + "model.layers.25.mlp.experts.12.up_proj.weight": "model-00026-of-00101.safetensors", + "model.layers.25.mlp.experts.12.down_proj.weight": "model-00026-of-00101.safetensors", + "model.layers.25.mlp.experts.13.gate_proj.weight": "model-00026-of-00101.safetensors", + "model.layers.25.mlp.experts.13.up_proj.weight": "model-00026-of-00101.safetensors", + "model.layers.25.mlp.experts.13.down_proj.weight": "model-00026-of-00101.safetensors", + "model.layers.25.mlp.experts.14.gate_proj.weight": "model-00026-of-00101.safetensors", + "model.layers.25.mlp.experts.14.up_proj.weight": "model-00026-of-00101.safetensors", + "model.layers.25.mlp.experts.14.down_proj.weight": "model-00026-of-00101.safetensors", + "model.layers.25.mlp.experts.15.gate_proj.weight": "model-00026-of-00101.safetensors", + "model.layers.25.mlp.experts.15.up_proj.weight": "model-00026-of-00101.safetensors", + "model.layers.25.mlp.experts.15.down_proj.weight": "model-00026-of-00101.safetensors", + "model.layers.25.mlp.experts.16.gate_proj.weight": "model-00026-of-00101.safetensors", + "model.layers.25.mlp.experts.16.up_proj.weight": "model-00026-of-00101.safetensors", + "model.layers.25.mlp.experts.16.down_proj.weight": "model-00026-of-00101.safetensors", + "model.layers.25.mlp.experts.17.gate_proj.weight": "model-00026-of-00101.safetensors", + "model.layers.25.mlp.experts.17.up_proj.weight": "model-00026-of-00101.safetensors", + "model.layers.25.mlp.experts.17.down_proj.weight": "model-00026-of-00101.safetensors", + "model.layers.25.mlp.experts.18.gate_proj.weight": "model-00026-of-00101.safetensors", + "model.layers.25.mlp.experts.18.up_proj.weight": "model-00026-of-00101.safetensors", + "model.layers.25.mlp.experts.18.down_proj.weight": "model-00026-of-00101.safetensors", + "model.layers.25.mlp.experts.19.gate_proj.weight": "model-00026-of-00101.safetensors", + "model.layers.25.mlp.experts.19.up_proj.weight": "model-00026-of-00101.safetensors", + "model.layers.25.mlp.experts.19.down_proj.weight": "model-00026-of-00101.safetensors", + "model.layers.25.mlp.experts.20.gate_proj.weight": "model-00026-of-00101.safetensors", + "model.layers.25.mlp.experts.20.up_proj.weight": "model-00026-of-00101.safetensors", + "model.layers.25.mlp.experts.20.down_proj.weight": "model-00026-of-00101.safetensors", + "model.layers.25.mlp.experts.21.gate_proj.weight": "model-00026-of-00101.safetensors", + "model.layers.25.mlp.experts.21.up_proj.weight": "model-00026-of-00101.safetensors", + "model.layers.25.mlp.experts.21.down_proj.weight": "model-00026-of-00101.safetensors", + "model.layers.25.mlp.experts.22.gate_proj.weight": "model-00026-of-00101.safetensors", + "model.layers.25.mlp.experts.22.up_proj.weight": "model-00026-of-00101.safetensors", + "model.layers.25.mlp.experts.22.down_proj.weight": "model-00026-of-00101.safetensors", + "model.layers.25.mlp.experts.23.gate_proj.weight": "model-00026-of-00101.safetensors", + "model.layers.25.mlp.experts.23.up_proj.weight": "model-00026-of-00101.safetensors", + "model.layers.25.mlp.experts.23.down_proj.weight": "model-00026-of-00101.safetensors", + "model.layers.25.mlp.experts.24.gate_proj.weight": "model-00026-of-00101.safetensors", + "model.layers.25.mlp.experts.24.up_proj.weight": "model-00026-of-00101.safetensors", + "model.layers.25.mlp.experts.24.down_proj.weight": "model-00026-of-00101.safetensors", + "model.layers.25.mlp.experts.25.gate_proj.weight": "model-00026-of-00101.safetensors", + "model.layers.25.mlp.experts.25.up_proj.weight": "model-00026-of-00101.safetensors", + "model.layers.25.mlp.experts.25.down_proj.weight": "model-00026-of-00101.safetensors", + "model.layers.25.mlp.experts.26.gate_proj.weight": "model-00026-of-00101.safetensors", + "model.layers.25.mlp.experts.26.up_proj.weight": "model-00026-of-00101.safetensors", + "model.layers.25.mlp.experts.26.down_proj.weight": "model-00026-of-00101.safetensors", + "model.layers.25.mlp.experts.27.gate_proj.weight": "model-00026-of-00101.safetensors", + "model.layers.25.mlp.experts.27.up_proj.weight": "model-00026-of-00101.safetensors", + "model.layers.25.mlp.experts.27.down_proj.weight": "model-00026-of-00101.safetensors", + "model.layers.25.mlp.experts.28.gate_proj.weight": "model-00026-of-00101.safetensors", + "model.layers.25.mlp.experts.28.up_proj.weight": "model-00026-of-00101.safetensors", + "model.layers.25.mlp.experts.28.down_proj.weight": "model-00026-of-00101.safetensors", + "model.layers.25.mlp.experts.29.gate_proj.weight": "model-00026-of-00101.safetensors", + "model.layers.25.mlp.experts.29.up_proj.weight": "model-00026-of-00101.safetensors", + "model.layers.25.mlp.experts.29.down_proj.weight": "model-00026-of-00101.safetensors", + "model.layers.25.mlp.experts.30.gate_proj.weight": "model-00026-of-00101.safetensors", + "model.layers.25.mlp.experts.30.up_proj.weight": "model-00026-of-00101.safetensors", + "model.layers.25.mlp.experts.30.down_proj.weight": "model-00026-of-00101.safetensors", + "model.layers.25.mlp.experts.31.gate_proj.weight": "model-00026-of-00101.safetensors", + "model.layers.25.mlp.experts.31.up_proj.weight": "model-00026-of-00101.safetensors", + "model.layers.25.mlp.experts.31.down_proj.weight": "model-00026-of-00101.safetensors", + "model.layers.25.mlp.experts.32.gate_proj.weight": "model-00026-of-00101.safetensors", + "model.layers.25.mlp.experts.32.up_proj.weight": "model-00026-of-00101.safetensors", + "model.layers.25.mlp.experts.32.down_proj.weight": "model-00026-of-00101.safetensors", + "model.layers.25.mlp.experts.33.gate_proj.weight": "model-00026-of-00101.safetensors", + "model.layers.25.mlp.experts.33.up_proj.weight": "model-00026-of-00101.safetensors", + "model.layers.25.mlp.experts.33.down_proj.weight": "model-00026-of-00101.safetensors", + "model.layers.25.mlp.experts.34.gate_proj.weight": "model-00026-of-00101.safetensors", + "model.layers.25.mlp.experts.34.up_proj.weight": "model-00026-of-00101.safetensors", + "model.layers.25.mlp.experts.34.down_proj.weight": "model-00026-of-00101.safetensors", + "model.layers.25.mlp.experts.35.gate_proj.weight": "model-00026-of-00101.safetensors", + "model.layers.25.mlp.experts.35.up_proj.weight": "model-00026-of-00101.safetensors", + "model.layers.25.mlp.experts.35.down_proj.weight": "model-00026-of-00101.safetensors", + "model.layers.25.mlp.experts.36.gate_proj.weight": "model-00026-of-00101.safetensors", + "model.layers.25.mlp.experts.36.up_proj.weight": "model-00026-of-00101.safetensors", + "model.layers.25.mlp.experts.36.down_proj.weight": "model-00026-of-00101.safetensors", + "model.layers.25.mlp.experts.37.gate_proj.weight": "model-00026-of-00101.safetensors", + "model.layers.25.mlp.experts.37.up_proj.weight": "model-00026-of-00101.safetensors", + "model.layers.25.mlp.experts.37.down_proj.weight": "model-00026-of-00101.safetensors", + "model.layers.25.mlp.experts.38.gate_proj.weight": "model-00026-of-00101.safetensors", + "model.layers.25.mlp.experts.38.up_proj.weight": "model-00026-of-00101.safetensors", + "model.layers.25.mlp.experts.38.down_proj.weight": "model-00026-of-00101.safetensors", + "model.layers.25.mlp.experts.39.gate_proj.weight": "model-00026-of-00101.safetensors", + "model.layers.25.mlp.experts.39.up_proj.weight": "model-00026-of-00101.safetensors", + "model.layers.25.mlp.experts.39.down_proj.weight": "model-00026-of-00101.safetensors", + "model.layers.25.mlp.experts.40.gate_proj.weight": "model-00026-of-00101.safetensors", + "model.layers.25.mlp.experts.40.up_proj.weight": "model-00026-of-00101.safetensors", + "model.layers.25.mlp.experts.40.down_proj.weight": "model-00026-of-00101.safetensors", + "model.layers.25.mlp.experts.41.gate_proj.weight": "model-00026-of-00101.safetensors", + "model.layers.25.mlp.experts.41.up_proj.weight": "model-00026-of-00101.safetensors", + "model.layers.25.mlp.experts.41.down_proj.weight": "model-00026-of-00101.safetensors", + "model.layers.25.mlp.experts.42.gate_proj.weight": "model-00026-of-00101.safetensors", + "model.layers.25.mlp.experts.42.up_proj.weight": "model-00026-of-00101.safetensors", + "model.layers.25.mlp.experts.42.down_proj.weight": "model-00026-of-00101.safetensors", + "model.layers.25.mlp.experts.43.gate_proj.weight": "model-00026-of-00101.safetensors", + "model.layers.25.mlp.experts.43.up_proj.weight": "model-00026-of-00101.safetensors", + "model.layers.25.mlp.experts.43.down_proj.weight": "model-00026-of-00101.safetensors", + "model.layers.25.mlp.experts.44.gate_proj.weight": "model-00026-of-00101.safetensors", + "model.layers.25.mlp.experts.44.up_proj.weight": "model-00026-of-00101.safetensors", + "model.layers.25.mlp.experts.44.down_proj.weight": "model-00026-of-00101.safetensors", + "model.layers.25.mlp.experts.45.gate_proj.weight": "model-00026-of-00101.safetensors", + "model.layers.25.mlp.experts.45.up_proj.weight": "model-00026-of-00101.safetensors", + "model.layers.25.mlp.experts.45.down_proj.weight": "model-00026-of-00101.safetensors", + "model.layers.25.mlp.experts.46.gate_proj.weight": "model-00026-of-00101.safetensors", + "model.layers.25.mlp.experts.46.up_proj.weight": "model-00026-of-00101.safetensors", + "model.layers.25.mlp.experts.46.down_proj.weight": "model-00026-of-00101.safetensors", + "model.layers.25.mlp.experts.47.gate_proj.weight": "model-00026-of-00101.safetensors", + "model.layers.25.mlp.experts.47.up_proj.weight": "model-00026-of-00101.safetensors", + "model.layers.25.mlp.experts.47.down_proj.weight": "model-00026-of-00101.safetensors", + "model.layers.25.mlp.experts.48.gate_proj.weight": "model-00026-of-00101.safetensors", + "model.layers.25.mlp.experts.48.up_proj.weight": "model-00026-of-00101.safetensors", + "model.layers.25.mlp.experts.48.down_proj.weight": "model-00026-of-00101.safetensors", + "model.layers.25.mlp.experts.49.gate_proj.weight": "model-00026-of-00101.safetensors", + "model.layers.25.mlp.experts.49.up_proj.weight": "model-00026-of-00101.safetensors", + "model.layers.25.mlp.experts.49.down_proj.weight": "model-00026-of-00101.safetensors", + "model.layers.25.mlp.experts.50.gate_proj.weight": "model-00026-of-00101.safetensors", + "model.layers.25.mlp.experts.50.up_proj.weight": "model-00026-of-00101.safetensors", + "model.layers.25.mlp.experts.50.down_proj.weight": "model-00026-of-00101.safetensors", + "model.layers.25.mlp.experts.51.gate_proj.weight": "model-00026-of-00101.safetensors", + "model.layers.25.mlp.experts.51.up_proj.weight": "model-00026-of-00101.safetensors", + "model.layers.25.mlp.experts.51.down_proj.weight": "model-00026-of-00101.safetensors", + "model.layers.25.mlp.experts.52.gate_proj.weight": "model-00026-of-00101.safetensors", + "model.layers.25.mlp.experts.52.up_proj.weight": "model-00026-of-00101.safetensors", + "model.layers.25.mlp.experts.52.down_proj.weight": "model-00026-of-00101.safetensors", + "model.layers.25.mlp.experts.53.gate_proj.weight": "model-00026-of-00101.safetensors", + "model.layers.25.mlp.experts.53.up_proj.weight": "model-00026-of-00101.safetensors", + "model.layers.25.mlp.experts.53.down_proj.weight": "model-00026-of-00101.safetensors", + "model.layers.25.mlp.experts.54.gate_proj.weight": "model-00026-of-00101.safetensors", + "model.layers.25.mlp.experts.54.up_proj.weight": "model-00026-of-00101.safetensors", + "model.layers.25.mlp.experts.54.down_proj.weight": "model-00026-of-00101.safetensors", + "model.layers.25.mlp.experts.55.gate_proj.weight": "model-00026-of-00101.safetensors", + "model.layers.25.mlp.experts.55.up_proj.weight": "model-00026-of-00101.safetensors", + "model.layers.25.mlp.experts.55.down_proj.weight": "model-00026-of-00101.safetensors", + "model.layers.25.mlp.experts.56.gate_proj.weight": "model-00026-of-00101.safetensors", + "model.layers.25.mlp.experts.56.up_proj.weight": "model-00026-of-00101.safetensors", + "model.layers.25.mlp.experts.56.down_proj.weight": "model-00026-of-00101.safetensors", + "model.layers.25.mlp.experts.57.gate_proj.weight": "model-00026-of-00101.safetensors", + "model.layers.25.mlp.experts.57.up_proj.weight": "model-00026-of-00101.safetensors", + "model.layers.25.mlp.experts.57.down_proj.weight": "model-00026-of-00101.safetensors", + "model.layers.25.mlp.experts.58.gate_proj.weight": "model-00026-of-00101.safetensors", + "model.layers.25.mlp.experts.58.up_proj.weight": "model-00026-of-00101.safetensors", + "model.layers.25.mlp.experts.58.down_proj.weight": "model-00026-of-00101.safetensors", + "model.layers.25.mlp.experts.59.gate_proj.weight": "model-00026-of-00101.safetensors", + "model.layers.25.mlp.experts.59.up_proj.weight": "model-00026-of-00101.safetensors", + "model.layers.25.mlp.experts.59.down_proj.weight": "model-00026-of-00101.safetensors", + "model.layers.25.mlp.experts.60.gate_proj.weight": "model-00026-of-00101.safetensors", + "model.layers.25.mlp.experts.60.up_proj.weight": "model-00026-of-00101.safetensors", + "model.layers.25.mlp.experts.60.down_proj.weight": "model-00026-of-00101.safetensors", + "model.layers.25.mlp.experts.61.gate_proj.weight": "model-00026-of-00101.safetensors", + "model.layers.25.mlp.experts.61.up_proj.weight": "model-00026-of-00101.safetensors", + "model.layers.25.mlp.experts.61.down_proj.weight": "model-00026-of-00101.safetensors", + "model.layers.25.mlp.experts.62.gate_proj.weight": "model-00026-of-00101.safetensors", + "model.layers.25.mlp.experts.62.up_proj.weight": "model-00026-of-00101.safetensors", + "model.layers.25.mlp.experts.62.down_proj.weight": "model-00026-of-00101.safetensors", + "model.layers.25.mlp.experts.63.gate_proj.weight": "model-00026-of-00101.safetensors", + "model.layers.25.mlp.experts.63.up_proj.weight": "model-00026-of-00101.safetensors", + "model.layers.25.mlp.experts.63.down_proj.weight": "model-00026-of-00101.safetensors", + "model.layers.25.mlp.experts.64.gate_proj.weight": "model-00026-of-00101.safetensors", + "model.layers.25.mlp.experts.64.up_proj.weight": "model-00026-of-00101.safetensors", + "model.layers.25.mlp.experts.64.down_proj.weight": "model-00026-of-00101.safetensors", + "model.layers.25.mlp.experts.65.gate_proj.weight": "model-00026-of-00101.safetensors", + "model.layers.25.mlp.experts.65.up_proj.weight": "model-00026-of-00101.safetensors", + "model.layers.25.mlp.experts.65.down_proj.weight": "model-00026-of-00101.safetensors", + "model.layers.25.mlp.experts.66.gate_proj.weight": "model-00026-of-00101.safetensors", + "model.layers.25.mlp.experts.66.up_proj.weight": "model-00026-of-00101.safetensors", + "model.layers.25.mlp.experts.66.down_proj.weight": "model-00026-of-00101.safetensors", + "model.layers.25.mlp.experts.67.gate_proj.weight": "model-00026-of-00101.safetensors", + "model.layers.25.mlp.experts.67.up_proj.weight": "model-00026-of-00101.safetensors", + "model.layers.25.mlp.experts.67.down_proj.weight": "model-00026-of-00101.safetensors", + "model.layers.25.mlp.experts.68.gate_proj.weight": "model-00026-of-00101.safetensors", + "model.layers.25.mlp.experts.68.up_proj.weight": "model-00026-of-00101.safetensors", + "model.layers.25.mlp.experts.68.down_proj.weight": "model-00026-of-00101.safetensors", + "model.layers.25.mlp.experts.69.gate_proj.weight": "model-00026-of-00101.safetensors", + "model.layers.25.mlp.experts.69.up_proj.weight": "model-00026-of-00101.safetensors", + "model.layers.25.mlp.experts.69.down_proj.weight": "model-00026-of-00101.safetensors", + "model.layers.25.mlp.experts.70.gate_proj.weight": "model-00026-of-00101.safetensors", + "model.layers.25.mlp.experts.70.up_proj.weight": "model-00026-of-00101.safetensors", + "model.layers.25.mlp.experts.70.down_proj.weight": "model-00026-of-00101.safetensors", + "model.layers.25.mlp.experts.71.gate_proj.weight": "model-00026-of-00101.safetensors", + "model.layers.25.mlp.experts.71.up_proj.weight": "model-00026-of-00101.safetensors", + "model.layers.25.mlp.experts.71.down_proj.weight": "model-00026-of-00101.safetensors", + "model.layers.25.mlp.experts.72.gate_proj.weight": "model-00026-of-00101.safetensors", + "model.layers.25.mlp.experts.72.up_proj.weight": "model-00026-of-00101.safetensors", + "model.layers.25.mlp.experts.72.down_proj.weight": "model-00026-of-00101.safetensors", + "model.layers.25.mlp.experts.73.gate_proj.weight": "model-00026-of-00101.safetensors", + "model.layers.25.mlp.experts.73.up_proj.weight": "model-00026-of-00101.safetensors", + "model.layers.25.mlp.experts.73.down_proj.weight": "model-00026-of-00101.safetensors", + "model.layers.25.mlp.experts.74.gate_proj.weight": "model-00026-of-00101.safetensors", + "model.layers.25.mlp.experts.74.up_proj.weight": "model-00026-of-00101.safetensors", + "model.layers.25.mlp.experts.74.down_proj.weight": "model-00026-of-00101.safetensors", + "model.layers.25.mlp.experts.75.gate_proj.weight": "model-00026-of-00101.safetensors", + "model.layers.25.mlp.experts.75.up_proj.weight": "model-00026-of-00101.safetensors", + "model.layers.25.mlp.experts.75.down_proj.weight": "model-00026-of-00101.safetensors", + "model.layers.25.mlp.experts.76.gate_proj.weight": "model-00026-of-00101.safetensors", + "model.layers.25.mlp.experts.76.up_proj.weight": "model-00026-of-00101.safetensors", + "model.layers.25.mlp.experts.76.down_proj.weight": "model-00026-of-00101.safetensors", + "model.layers.25.mlp.experts.77.gate_proj.weight": "model-00026-of-00101.safetensors", + "model.layers.25.mlp.experts.77.up_proj.weight": "model-00026-of-00101.safetensors", + "model.layers.25.mlp.experts.77.down_proj.weight": "model-00026-of-00101.safetensors", + "model.layers.25.mlp.experts.78.gate_proj.weight": "model-00026-of-00101.safetensors", + "model.layers.25.mlp.experts.78.up_proj.weight": "model-00026-of-00101.safetensors", + "model.layers.25.mlp.experts.78.down_proj.weight": "model-00026-of-00101.safetensors", + "model.layers.25.mlp.experts.79.gate_proj.weight": "model-00026-of-00101.safetensors", + "model.layers.25.mlp.experts.79.up_proj.weight": "model-00026-of-00101.safetensors", + "model.layers.25.mlp.experts.79.down_proj.weight": "model-00026-of-00101.safetensors", + "model.layers.25.mlp.experts.80.gate_proj.weight": "model-00026-of-00101.safetensors", + "model.layers.25.mlp.experts.80.up_proj.weight": "model-00026-of-00101.safetensors", + "model.layers.25.mlp.experts.80.down_proj.weight": "model-00026-of-00101.safetensors", + "model.layers.25.mlp.experts.81.gate_proj.weight": "model-00027-of-00101.safetensors", + "model.layers.25.mlp.experts.81.up_proj.weight": "model-00027-of-00101.safetensors", + "model.layers.25.mlp.experts.81.down_proj.weight": "model-00027-of-00101.safetensors", + "model.layers.25.mlp.experts.82.gate_proj.weight": "model-00027-of-00101.safetensors", + "model.layers.25.mlp.experts.82.up_proj.weight": "model-00027-of-00101.safetensors", + "model.layers.25.mlp.experts.82.down_proj.weight": "model-00027-of-00101.safetensors", + "model.layers.25.mlp.experts.83.gate_proj.weight": "model-00027-of-00101.safetensors", + "model.layers.25.mlp.experts.83.up_proj.weight": "model-00027-of-00101.safetensors", + "model.layers.25.mlp.experts.83.down_proj.weight": "model-00027-of-00101.safetensors", + "model.layers.25.mlp.experts.84.gate_proj.weight": "model-00027-of-00101.safetensors", + "model.layers.25.mlp.experts.84.up_proj.weight": "model-00027-of-00101.safetensors", + "model.layers.25.mlp.experts.84.down_proj.weight": "model-00027-of-00101.safetensors", + "model.layers.25.mlp.experts.85.gate_proj.weight": "model-00027-of-00101.safetensors", + "model.layers.25.mlp.experts.85.up_proj.weight": "model-00027-of-00101.safetensors", + "model.layers.25.mlp.experts.85.down_proj.weight": "model-00027-of-00101.safetensors", + "model.layers.25.mlp.experts.86.gate_proj.weight": "model-00027-of-00101.safetensors", + "model.layers.25.mlp.experts.86.up_proj.weight": "model-00027-of-00101.safetensors", + "model.layers.25.mlp.experts.86.down_proj.weight": "model-00027-of-00101.safetensors", + "model.layers.25.mlp.experts.87.gate_proj.weight": "model-00027-of-00101.safetensors", + "model.layers.25.mlp.experts.87.up_proj.weight": "model-00027-of-00101.safetensors", + "model.layers.25.mlp.experts.87.down_proj.weight": "model-00027-of-00101.safetensors", + "model.layers.25.mlp.experts.88.gate_proj.weight": "model-00027-of-00101.safetensors", + "model.layers.25.mlp.experts.88.up_proj.weight": "model-00027-of-00101.safetensors", + "model.layers.25.mlp.experts.88.down_proj.weight": "model-00027-of-00101.safetensors", + "model.layers.25.mlp.experts.89.gate_proj.weight": "model-00027-of-00101.safetensors", + "model.layers.25.mlp.experts.89.up_proj.weight": "model-00027-of-00101.safetensors", + "model.layers.25.mlp.experts.89.down_proj.weight": "model-00027-of-00101.safetensors", + "model.layers.25.mlp.experts.90.gate_proj.weight": "model-00027-of-00101.safetensors", + "model.layers.25.mlp.experts.90.up_proj.weight": "model-00027-of-00101.safetensors", + "model.layers.25.mlp.experts.90.down_proj.weight": "model-00027-of-00101.safetensors", + "model.layers.25.mlp.experts.91.gate_proj.weight": "model-00027-of-00101.safetensors", + "model.layers.25.mlp.experts.91.up_proj.weight": "model-00027-of-00101.safetensors", + "model.layers.25.mlp.experts.91.down_proj.weight": "model-00027-of-00101.safetensors", + "model.layers.25.mlp.experts.92.gate_proj.weight": "model-00027-of-00101.safetensors", + "model.layers.25.mlp.experts.92.up_proj.weight": "model-00027-of-00101.safetensors", + "model.layers.25.mlp.experts.92.down_proj.weight": "model-00027-of-00101.safetensors", + "model.layers.25.mlp.experts.93.gate_proj.weight": "model-00027-of-00101.safetensors", + "model.layers.25.mlp.experts.93.up_proj.weight": "model-00027-of-00101.safetensors", + "model.layers.25.mlp.experts.93.down_proj.weight": "model-00027-of-00101.safetensors", + "model.layers.25.mlp.experts.94.gate_proj.weight": "model-00027-of-00101.safetensors", + "model.layers.25.mlp.experts.94.up_proj.weight": "model-00027-of-00101.safetensors", + "model.layers.25.mlp.experts.94.down_proj.weight": "model-00027-of-00101.safetensors", + "model.layers.25.mlp.experts.95.gate_proj.weight": "model-00027-of-00101.safetensors", + "model.layers.25.mlp.experts.95.up_proj.weight": "model-00027-of-00101.safetensors", + "model.layers.25.mlp.experts.95.down_proj.weight": "model-00027-of-00101.safetensors", + "model.layers.25.mlp.experts.96.gate_proj.weight": "model-00027-of-00101.safetensors", + "model.layers.25.mlp.experts.96.up_proj.weight": "model-00027-of-00101.safetensors", + "model.layers.25.mlp.experts.96.down_proj.weight": "model-00027-of-00101.safetensors", + "model.layers.25.mlp.experts.97.gate_proj.weight": "model-00027-of-00101.safetensors", + "model.layers.25.mlp.experts.97.up_proj.weight": "model-00027-of-00101.safetensors", + "model.layers.25.mlp.experts.97.down_proj.weight": "model-00027-of-00101.safetensors", + "model.layers.25.mlp.experts.98.gate_proj.weight": "model-00027-of-00101.safetensors", + "model.layers.25.mlp.experts.98.up_proj.weight": "model-00027-of-00101.safetensors", + "model.layers.25.mlp.experts.98.down_proj.weight": "model-00027-of-00101.safetensors", + "model.layers.25.mlp.experts.99.gate_proj.weight": "model-00027-of-00101.safetensors", + "model.layers.25.mlp.experts.99.up_proj.weight": "model-00027-of-00101.safetensors", + "model.layers.25.mlp.experts.99.down_proj.weight": "model-00027-of-00101.safetensors", + "model.layers.25.mlp.experts.100.gate_proj.weight": "model-00027-of-00101.safetensors", + "model.layers.25.mlp.experts.100.up_proj.weight": "model-00027-of-00101.safetensors", + "model.layers.25.mlp.experts.100.down_proj.weight": "model-00027-of-00101.safetensors", + "model.layers.25.mlp.experts.101.gate_proj.weight": "model-00027-of-00101.safetensors", + "model.layers.25.mlp.experts.101.up_proj.weight": "model-00027-of-00101.safetensors", + "model.layers.25.mlp.experts.101.down_proj.weight": "model-00027-of-00101.safetensors", + "model.layers.25.mlp.experts.102.gate_proj.weight": "model-00027-of-00101.safetensors", + "model.layers.25.mlp.experts.102.up_proj.weight": "model-00027-of-00101.safetensors", + "model.layers.25.mlp.experts.102.down_proj.weight": "model-00027-of-00101.safetensors", + "model.layers.25.mlp.experts.103.gate_proj.weight": "model-00027-of-00101.safetensors", + "model.layers.25.mlp.experts.103.up_proj.weight": "model-00027-of-00101.safetensors", + "model.layers.25.mlp.experts.103.down_proj.weight": "model-00027-of-00101.safetensors", + "model.layers.25.mlp.experts.104.gate_proj.weight": "model-00027-of-00101.safetensors", + "model.layers.25.mlp.experts.104.up_proj.weight": "model-00027-of-00101.safetensors", + "model.layers.25.mlp.experts.104.down_proj.weight": "model-00027-of-00101.safetensors", + "model.layers.25.mlp.experts.105.gate_proj.weight": "model-00027-of-00101.safetensors", + "model.layers.25.mlp.experts.105.up_proj.weight": "model-00027-of-00101.safetensors", + "model.layers.25.mlp.experts.105.down_proj.weight": "model-00027-of-00101.safetensors", + "model.layers.25.mlp.experts.106.gate_proj.weight": "model-00027-of-00101.safetensors", + "model.layers.25.mlp.experts.106.up_proj.weight": "model-00027-of-00101.safetensors", + "model.layers.25.mlp.experts.106.down_proj.weight": "model-00027-of-00101.safetensors", + "model.layers.25.mlp.experts.107.gate_proj.weight": "model-00027-of-00101.safetensors", + "model.layers.25.mlp.experts.107.up_proj.weight": "model-00027-of-00101.safetensors", + "model.layers.25.mlp.experts.107.down_proj.weight": "model-00027-of-00101.safetensors", + "model.layers.25.mlp.experts.108.gate_proj.weight": "model-00027-of-00101.safetensors", + "model.layers.25.mlp.experts.108.up_proj.weight": "model-00027-of-00101.safetensors", + "model.layers.25.mlp.experts.108.down_proj.weight": "model-00027-of-00101.safetensors", + "model.layers.25.mlp.experts.109.gate_proj.weight": "model-00027-of-00101.safetensors", + "model.layers.25.mlp.experts.109.up_proj.weight": "model-00027-of-00101.safetensors", + "model.layers.25.mlp.experts.109.down_proj.weight": "model-00027-of-00101.safetensors", + "model.layers.25.mlp.experts.110.gate_proj.weight": "model-00027-of-00101.safetensors", + "model.layers.25.mlp.experts.110.up_proj.weight": "model-00027-of-00101.safetensors", + "model.layers.25.mlp.experts.110.down_proj.weight": "model-00027-of-00101.safetensors", + "model.layers.25.mlp.experts.111.gate_proj.weight": "model-00027-of-00101.safetensors", + "model.layers.25.mlp.experts.111.up_proj.weight": "model-00027-of-00101.safetensors", + "model.layers.25.mlp.experts.111.down_proj.weight": "model-00027-of-00101.safetensors", + "model.layers.25.mlp.experts.112.gate_proj.weight": "model-00027-of-00101.safetensors", + "model.layers.25.mlp.experts.112.up_proj.weight": "model-00027-of-00101.safetensors", + "model.layers.25.mlp.experts.112.down_proj.weight": "model-00027-of-00101.safetensors", + "model.layers.25.mlp.experts.113.gate_proj.weight": "model-00027-of-00101.safetensors", + "model.layers.25.mlp.experts.113.up_proj.weight": "model-00027-of-00101.safetensors", + "model.layers.25.mlp.experts.113.down_proj.weight": "model-00027-of-00101.safetensors", + "model.layers.25.mlp.experts.114.gate_proj.weight": "model-00027-of-00101.safetensors", + "model.layers.25.mlp.experts.114.up_proj.weight": "model-00027-of-00101.safetensors", + "model.layers.25.mlp.experts.114.down_proj.weight": "model-00027-of-00101.safetensors", + "model.layers.25.mlp.experts.115.gate_proj.weight": "model-00027-of-00101.safetensors", + "model.layers.25.mlp.experts.115.up_proj.weight": "model-00027-of-00101.safetensors", + "model.layers.25.mlp.experts.115.down_proj.weight": "model-00027-of-00101.safetensors", + "model.layers.25.mlp.experts.116.gate_proj.weight": "model-00027-of-00101.safetensors", + "model.layers.25.mlp.experts.116.up_proj.weight": "model-00027-of-00101.safetensors", + "model.layers.25.mlp.experts.116.down_proj.weight": "model-00027-of-00101.safetensors", + "model.layers.25.mlp.experts.117.gate_proj.weight": "model-00027-of-00101.safetensors", + "model.layers.25.mlp.experts.117.up_proj.weight": "model-00027-of-00101.safetensors", + "model.layers.25.mlp.experts.117.down_proj.weight": "model-00027-of-00101.safetensors", + "model.layers.25.mlp.experts.118.gate_proj.weight": "model-00027-of-00101.safetensors", + "model.layers.25.mlp.experts.118.up_proj.weight": "model-00027-of-00101.safetensors", + "model.layers.25.mlp.experts.118.down_proj.weight": "model-00027-of-00101.safetensors", + "model.layers.25.mlp.experts.119.gate_proj.weight": "model-00027-of-00101.safetensors", + "model.layers.25.mlp.experts.119.up_proj.weight": "model-00027-of-00101.safetensors", + "model.layers.25.mlp.experts.119.down_proj.weight": "model-00027-of-00101.safetensors", + "model.layers.25.mlp.gate.weight": "model-00027-of-00101.safetensors", + "model.layers.25.mlp.gate.e_score_correction_bias": "model-00027-of-00101.safetensors", + "model.layers.25.mlp.shared_experts.gate_proj.weight": "model-00027-of-00101.safetensors", + "model.layers.25.mlp.shared_experts.up_proj.weight": "model-00027-of-00101.safetensors", + "model.layers.25.mlp.shared_experts.down_proj.weight": "model-00027-of-00101.safetensors", + "model.layers.25.input_layernorm.weight": "model-00027-of-00101.safetensors", + "model.layers.25.post_attention_layernorm.weight": "model-00027-of-00101.safetensors", + "model.layers.26.self_attn.q_proj.weight": "model-00027-of-00101.safetensors", + "model.layers.26.self_attn.q_proj.bias": "model-00027-of-00101.safetensors", + "model.layers.26.self_attn.k_proj.weight": "model-00027-of-00101.safetensors", + "model.layers.26.self_attn.k_proj.bias": "model-00027-of-00101.safetensors", + "model.layers.26.self_attn.v_proj.weight": "model-00027-of-00101.safetensors", + "model.layers.26.self_attn.v_proj.bias": "model-00027-of-00101.safetensors", + "model.layers.26.self_attn.o_proj.weight": "model-00027-of-00101.safetensors", + "model.layers.26.self_attn.q_norm.weight": "model-00027-of-00101.safetensors", + "model.layers.26.self_attn.k_norm.weight": "model-00027-of-00101.safetensors", + "model.layers.26.mlp.experts.0.gate_proj.weight": "model-00027-of-00101.safetensors", + "model.layers.26.mlp.experts.0.up_proj.weight": "model-00027-of-00101.safetensors", + "model.layers.26.mlp.experts.0.down_proj.weight": "model-00027-of-00101.safetensors", + "model.layers.26.mlp.experts.1.gate_proj.weight": "model-00027-of-00101.safetensors", + "model.layers.26.mlp.experts.1.up_proj.weight": "model-00027-of-00101.safetensors", + "model.layers.26.mlp.experts.1.down_proj.weight": "model-00027-of-00101.safetensors", + "model.layers.26.mlp.experts.2.gate_proj.weight": "model-00027-of-00101.safetensors", + "model.layers.26.mlp.experts.2.up_proj.weight": "model-00027-of-00101.safetensors", + "model.layers.26.mlp.experts.2.down_proj.weight": "model-00027-of-00101.safetensors", + "model.layers.26.mlp.experts.3.gate_proj.weight": "model-00027-of-00101.safetensors", + "model.layers.26.mlp.experts.3.up_proj.weight": "model-00027-of-00101.safetensors", + "model.layers.26.mlp.experts.3.down_proj.weight": "model-00027-of-00101.safetensors", + "model.layers.26.mlp.experts.4.gate_proj.weight": "model-00027-of-00101.safetensors", + "model.layers.26.mlp.experts.4.up_proj.weight": "model-00027-of-00101.safetensors", + "model.layers.26.mlp.experts.4.down_proj.weight": "model-00027-of-00101.safetensors", + "model.layers.26.mlp.experts.5.gate_proj.weight": "model-00027-of-00101.safetensors", + "model.layers.26.mlp.experts.5.up_proj.weight": "model-00027-of-00101.safetensors", + "model.layers.26.mlp.experts.5.down_proj.weight": "model-00027-of-00101.safetensors", + "model.layers.26.mlp.experts.6.gate_proj.weight": "model-00027-of-00101.safetensors", + "model.layers.26.mlp.experts.6.up_proj.weight": "model-00027-of-00101.safetensors", + "model.layers.26.mlp.experts.6.down_proj.weight": "model-00027-of-00101.safetensors", + "model.layers.26.mlp.experts.7.gate_proj.weight": "model-00027-of-00101.safetensors", + "model.layers.26.mlp.experts.7.up_proj.weight": "model-00027-of-00101.safetensors", + "model.layers.26.mlp.experts.7.down_proj.weight": "model-00027-of-00101.safetensors", + "model.layers.26.mlp.experts.8.gate_proj.weight": "model-00027-of-00101.safetensors", + "model.layers.26.mlp.experts.8.up_proj.weight": "model-00027-of-00101.safetensors", + "model.layers.26.mlp.experts.8.down_proj.weight": "model-00027-of-00101.safetensors", + "model.layers.26.mlp.experts.9.gate_proj.weight": "model-00027-of-00101.safetensors", + "model.layers.26.mlp.experts.9.up_proj.weight": "model-00027-of-00101.safetensors", + "model.layers.26.mlp.experts.9.down_proj.weight": "model-00027-of-00101.safetensors", + "model.layers.26.mlp.experts.10.gate_proj.weight": "model-00027-of-00101.safetensors", + "model.layers.26.mlp.experts.10.up_proj.weight": "model-00027-of-00101.safetensors", + "model.layers.26.mlp.experts.10.down_proj.weight": "model-00027-of-00101.safetensors", + "model.layers.26.mlp.experts.11.gate_proj.weight": "model-00027-of-00101.safetensors", + "model.layers.26.mlp.experts.11.up_proj.weight": "model-00027-of-00101.safetensors", + "model.layers.26.mlp.experts.11.down_proj.weight": "model-00027-of-00101.safetensors", + "model.layers.26.mlp.experts.12.gate_proj.weight": "model-00027-of-00101.safetensors", + "model.layers.26.mlp.experts.12.up_proj.weight": "model-00027-of-00101.safetensors", + "model.layers.26.mlp.experts.12.down_proj.weight": "model-00027-of-00101.safetensors", + "model.layers.26.mlp.experts.13.gate_proj.weight": "model-00027-of-00101.safetensors", + "model.layers.26.mlp.experts.13.up_proj.weight": "model-00027-of-00101.safetensors", + "model.layers.26.mlp.experts.13.down_proj.weight": "model-00027-of-00101.safetensors", + "model.layers.26.mlp.experts.14.gate_proj.weight": "model-00027-of-00101.safetensors", + "model.layers.26.mlp.experts.14.up_proj.weight": "model-00027-of-00101.safetensors", + "model.layers.26.mlp.experts.14.down_proj.weight": "model-00027-of-00101.safetensors", + "model.layers.26.mlp.experts.15.gate_proj.weight": "model-00027-of-00101.safetensors", + "model.layers.26.mlp.experts.15.up_proj.weight": "model-00027-of-00101.safetensors", + "model.layers.26.mlp.experts.15.down_proj.weight": "model-00027-of-00101.safetensors", + "model.layers.26.mlp.experts.16.gate_proj.weight": "model-00027-of-00101.safetensors", + "model.layers.26.mlp.experts.16.up_proj.weight": "model-00027-of-00101.safetensors", + "model.layers.26.mlp.experts.16.down_proj.weight": "model-00027-of-00101.safetensors", + "model.layers.26.mlp.experts.17.gate_proj.weight": "model-00027-of-00101.safetensors", + "model.layers.26.mlp.experts.17.up_proj.weight": "model-00027-of-00101.safetensors", + "model.layers.26.mlp.experts.17.down_proj.weight": "model-00027-of-00101.safetensors", + "model.layers.26.mlp.experts.18.gate_proj.weight": "model-00027-of-00101.safetensors", + "model.layers.26.mlp.experts.18.up_proj.weight": "model-00027-of-00101.safetensors", + "model.layers.26.mlp.experts.18.down_proj.weight": "model-00027-of-00101.safetensors", + "model.layers.26.mlp.experts.19.gate_proj.weight": "model-00027-of-00101.safetensors", + "model.layers.26.mlp.experts.19.up_proj.weight": "model-00027-of-00101.safetensors", + "model.layers.26.mlp.experts.19.down_proj.weight": "model-00027-of-00101.safetensors", + "model.layers.26.mlp.experts.20.gate_proj.weight": "model-00027-of-00101.safetensors", + "model.layers.26.mlp.experts.20.up_proj.weight": "model-00027-of-00101.safetensors", + "model.layers.26.mlp.experts.20.down_proj.weight": "model-00027-of-00101.safetensors", + "model.layers.26.mlp.experts.21.gate_proj.weight": "model-00027-of-00101.safetensors", + "model.layers.26.mlp.experts.21.up_proj.weight": "model-00027-of-00101.safetensors", + "model.layers.26.mlp.experts.21.down_proj.weight": "model-00027-of-00101.safetensors", + "model.layers.26.mlp.experts.22.gate_proj.weight": "model-00027-of-00101.safetensors", + "model.layers.26.mlp.experts.22.up_proj.weight": "model-00027-of-00101.safetensors", + "model.layers.26.mlp.experts.22.down_proj.weight": "model-00027-of-00101.safetensors", + "model.layers.26.mlp.experts.23.gate_proj.weight": "model-00027-of-00101.safetensors", + "model.layers.26.mlp.experts.23.up_proj.weight": "model-00027-of-00101.safetensors", + "model.layers.26.mlp.experts.23.down_proj.weight": "model-00027-of-00101.safetensors", + "model.layers.26.mlp.experts.24.gate_proj.weight": "model-00027-of-00101.safetensors", + "model.layers.26.mlp.experts.24.up_proj.weight": "model-00027-of-00101.safetensors", + "model.layers.26.mlp.experts.24.down_proj.weight": "model-00027-of-00101.safetensors", + "model.layers.26.mlp.experts.25.gate_proj.weight": "model-00027-of-00101.safetensors", + "model.layers.26.mlp.experts.25.up_proj.weight": "model-00027-of-00101.safetensors", + "model.layers.26.mlp.experts.25.down_proj.weight": "model-00027-of-00101.safetensors", + "model.layers.26.mlp.experts.26.gate_proj.weight": "model-00027-of-00101.safetensors", + "model.layers.26.mlp.experts.26.up_proj.weight": "model-00027-of-00101.safetensors", + "model.layers.26.mlp.experts.26.down_proj.weight": "model-00027-of-00101.safetensors", + "model.layers.26.mlp.experts.27.gate_proj.weight": "model-00027-of-00101.safetensors", + "model.layers.26.mlp.experts.27.up_proj.weight": "model-00027-of-00101.safetensors", + "model.layers.26.mlp.experts.27.down_proj.weight": "model-00027-of-00101.safetensors", + "model.layers.26.mlp.experts.28.gate_proj.weight": "model-00027-of-00101.safetensors", + "model.layers.26.mlp.experts.28.up_proj.weight": "model-00027-of-00101.safetensors", + "model.layers.26.mlp.experts.28.down_proj.weight": "model-00027-of-00101.safetensors", + "model.layers.26.mlp.experts.29.gate_proj.weight": "model-00027-of-00101.safetensors", + "model.layers.26.mlp.experts.29.up_proj.weight": "model-00027-of-00101.safetensors", + "model.layers.26.mlp.experts.29.down_proj.weight": "model-00027-of-00101.safetensors", + "model.layers.26.mlp.experts.30.gate_proj.weight": "model-00027-of-00101.safetensors", + "model.layers.26.mlp.experts.30.up_proj.weight": "model-00027-of-00101.safetensors", + "model.layers.26.mlp.experts.30.down_proj.weight": "model-00027-of-00101.safetensors", + "model.layers.26.mlp.experts.31.gate_proj.weight": "model-00027-of-00101.safetensors", + "model.layers.26.mlp.experts.31.up_proj.weight": "model-00027-of-00101.safetensors", + "model.layers.26.mlp.experts.31.down_proj.weight": "model-00027-of-00101.safetensors", + "model.layers.26.mlp.experts.32.gate_proj.weight": "model-00027-of-00101.safetensors", + "model.layers.26.mlp.experts.32.up_proj.weight": "model-00027-of-00101.safetensors", + "model.layers.26.mlp.experts.32.down_proj.weight": "model-00027-of-00101.safetensors", + "model.layers.26.mlp.experts.33.gate_proj.weight": "model-00027-of-00101.safetensors", + "model.layers.26.mlp.experts.33.up_proj.weight": "model-00027-of-00101.safetensors", + "model.layers.26.mlp.experts.33.down_proj.weight": "model-00027-of-00101.safetensors", + "model.layers.26.mlp.experts.34.gate_proj.weight": "model-00027-of-00101.safetensors", + "model.layers.26.mlp.experts.34.up_proj.weight": "model-00027-of-00101.safetensors", + "model.layers.26.mlp.experts.34.down_proj.weight": "model-00027-of-00101.safetensors", + "model.layers.26.mlp.experts.35.gate_proj.weight": "model-00027-of-00101.safetensors", + "model.layers.26.mlp.experts.35.up_proj.weight": "model-00027-of-00101.safetensors", + "model.layers.26.mlp.experts.35.down_proj.weight": "model-00027-of-00101.safetensors", + "model.layers.26.mlp.experts.36.gate_proj.weight": "model-00027-of-00101.safetensors", + "model.layers.26.mlp.experts.36.up_proj.weight": "model-00027-of-00101.safetensors", + "model.layers.26.mlp.experts.36.down_proj.weight": "model-00027-of-00101.safetensors", + "model.layers.26.mlp.experts.37.gate_proj.weight": "model-00027-of-00101.safetensors", + "model.layers.26.mlp.experts.37.up_proj.weight": "model-00027-of-00101.safetensors", + "model.layers.26.mlp.experts.37.down_proj.weight": "model-00027-of-00101.safetensors", + "model.layers.26.mlp.experts.38.gate_proj.weight": "model-00027-of-00101.safetensors", + "model.layers.26.mlp.experts.38.up_proj.weight": "model-00027-of-00101.safetensors", + "model.layers.26.mlp.experts.38.down_proj.weight": "model-00027-of-00101.safetensors", + "model.layers.26.mlp.experts.39.gate_proj.weight": "model-00027-of-00101.safetensors", + "model.layers.26.mlp.experts.39.up_proj.weight": "model-00027-of-00101.safetensors", + "model.layers.26.mlp.experts.39.down_proj.weight": "model-00027-of-00101.safetensors", + "model.layers.26.mlp.experts.40.gate_proj.weight": "model-00027-of-00101.safetensors", + "model.layers.26.mlp.experts.40.up_proj.weight": "model-00027-of-00101.safetensors", + "model.layers.26.mlp.experts.40.down_proj.weight": "model-00027-of-00101.safetensors", + "model.layers.26.mlp.experts.41.gate_proj.weight": "model-00027-of-00101.safetensors", + "model.layers.26.mlp.experts.41.up_proj.weight": "model-00027-of-00101.safetensors", + "model.layers.26.mlp.experts.41.down_proj.weight": "model-00027-of-00101.safetensors", + "model.layers.26.mlp.experts.42.gate_proj.weight": "model-00027-of-00101.safetensors", + "model.layers.26.mlp.experts.42.up_proj.weight": "model-00027-of-00101.safetensors", + "model.layers.26.mlp.experts.42.down_proj.weight": "model-00027-of-00101.safetensors", + "model.layers.26.mlp.experts.43.gate_proj.weight": "model-00027-of-00101.safetensors", + "model.layers.26.mlp.experts.43.up_proj.weight": "model-00027-of-00101.safetensors", + "model.layers.26.mlp.experts.43.down_proj.weight": "model-00027-of-00101.safetensors", + "model.layers.26.mlp.experts.44.gate_proj.weight": "model-00027-of-00101.safetensors", + "model.layers.26.mlp.experts.44.up_proj.weight": "model-00027-of-00101.safetensors", + "model.layers.26.mlp.experts.44.down_proj.weight": "model-00027-of-00101.safetensors", + "model.layers.26.mlp.experts.45.gate_proj.weight": "model-00027-of-00101.safetensors", + "model.layers.26.mlp.experts.45.up_proj.weight": "model-00027-of-00101.safetensors", + "model.layers.26.mlp.experts.45.down_proj.weight": "model-00027-of-00101.safetensors", + "model.layers.26.mlp.experts.46.gate_proj.weight": "model-00027-of-00101.safetensors", + "model.layers.26.mlp.experts.46.up_proj.weight": "model-00027-of-00101.safetensors", + "model.layers.26.mlp.experts.46.down_proj.weight": "model-00027-of-00101.safetensors", + "model.layers.26.mlp.experts.47.gate_proj.weight": "model-00027-of-00101.safetensors", + "model.layers.26.mlp.experts.47.up_proj.weight": "model-00027-of-00101.safetensors", + "model.layers.26.mlp.experts.47.down_proj.weight": "model-00027-of-00101.safetensors", + "model.layers.26.mlp.experts.48.gate_proj.weight": "model-00027-of-00101.safetensors", + "model.layers.26.mlp.experts.48.up_proj.weight": "model-00027-of-00101.safetensors", + "model.layers.26.mlp.experts.48.down_proj.weight": "model-00027-of-00101.safetensors", + "model.layers.26.mlp.experts.49.gate_proj.weight": "model-00027-of-00101.safetensors", + "model.layers.26.mlp.experts.49.up_proj.weight": "model-00027-of-00101.safetensors", + "model.layers.26.mlp.experts.49.down_proj.weight": "model-00027-of-00101.safetensors", + "model.layers.26.mlp.experts.50.gate_proj.weight": "model-00027-of-00101.safetensors", + "model.layers.26.mlp.experts.50.up_proj.weight": "model-00027-of-00101.safetensors", + "model.layers.26.mlp.experts.50.down_proj.weight": "model-00027-of-00101.safetensors", + "model.layers.26.mlp.experts.51.gate_proj.weight": "model-00027-of-00101.safetensors", + "model.layers.26.mlp.experts.51.up_proj.weight": "model-00027-of-00101.safetensors", + "model.layers.26.mlp.experts.51.down_proj.weight": "model-00027-of-00101.safetensors", + "model.layers.26.mlp.experts.52.gate_proj.weight": "model-00027-of-00101.safetensors", + "model.layers.26.mlp.experts.52.up_proj.weight": "model-00027-of-00101.safetensors", + "model.layers.26.mlp.experts.52.down_proj.weight": "model-00027-of-00101.safetensors", + "model.layers.26.mlp.experts.53.gate_proj.weight": "model-00027-of-00101.safetensors", + "model.layers.26.mlp.experts.53.up_proj.weight": "model-00027-of-00101.safetensors", + "model.layers.26.mlp.experts.53.down_proj.weight": "model-00027-of-00101.safetensors", + "model.layers.26.mlp.experts.54.gate_proj.weight": "model-00027-of-00101.safetensors", + "model.layers.26.mlp.experts.54.up_proj.weight": "model-00027-of-00101.safetensors", + "model.layers.26.mlp.experts.54.down_proj.weight": "model-00027-of-00101.safetensors", + "model.layers.26.mlp.experts.55.gate_proj.weight": "model-00027-of-00101.safetensors", + "model.layers.26.mlp.experts.55.up_proj.weight": "model-00027-of-00101.safetensors", + "model.layers.26.mlp.experts.55.down_proj.weight": "model-00027-of-00101.safetensors", + "model.layers.26.mlp.experts.56.gate_proj.weight": "model-00027-of-00101.safetensors", + "model.layers.26.mlp.experts.56.up_proj.weight": "model-00027-of-00101.safetensors", + "model.layers.26.mlp.experts.56.down_proj.weight": "model-00027-of-00101.safetensors", + "model.layers.26.mlp.experts.57.gate_proj.weight": "model-00027-of-00101.safetensors", + "model.layers.26.mlp.experts.57.up_proj.weight": "model-00027-of-00101.safetensors", + "model.layers.26.mlp.experts.57.down_proj.weight": "model-00027-of-00101.safetensors", + "model.layers.26.mlp.experts.58.gate_proj.weight": "model-00027-of-00101.safetensors", + "model.layers.26.mlp.experts.58.up_proj.weight": "model-00027-of-00101.safetensors", + "model.layers.26.mlp.experts.58.down_proj.weight": "model-00027-of-00101.safetensors", + "model.layers.26.mlp.experts.59.gate_proj.weight": "model-00027-of-00101.safetensors", + "model.layers.26.mlp.experts.59.up_proj.weight": "model-00027-of-00101.safetensors", + "model.layers.26.mlp.experts.59.down_proj.weight": "model-00027-of-00101.safetensors", + "model.layers.26.mlp.experts.60.gate_proj.weight": "model-00027-of-00101.safetensors", + "model.layers.26.mlp.experts.60.up_proj.weight": "model-00027-of-00101.safetensors", + "model.layers.26.mlp.experts.60.down_proj.weight": "model-00027-of-00101.safetensors", + "model.layers.26.mlp.experts.61.gate_proj.weight": "model-00027-of-00101.safetensors", + "model.layers.26.mlp.experts.61.up_proj.weight": "model-00027-of-00101.safetensors", + "model.layers.26.mlp.experts.61.down_proj.weight": "model-00027-of-00101.safetensors", + "model.layers.26.mlp.experts.62.gate_proj.weight": "model-00027-of-00101.safetensors", + "model.layers.26.mlp.experts.62.up_proj.weight": "model-00027-of-00101.safetensors", + "model.layers.26.mlp.experts.62.down_proj.weight": "model-00027-of-00101.safetensors", + "model.layers.26.mlp.experts.63.gate_proj.weight": "model-00027-of-00101.safetensors", + "model.layers.26.mlp.experts.63.up_proj.weight": "model-00027-of-00101.safetensors", + "model.layers.26.mlp.experts.63.down_proj.weight": "model-00027-of-00101.safetensors", + "model.layers.26.mlp.experts.64.gate_proj.weight": "model-00027-of-00101.safetensors", + "model.layers.26.mlp.experts.64.up_proj.weight": "model-00027-of-00101.safetensors", + "model.layers.26.mlp.experts.64.down_proj.weight": "model-00027-of-00101.safetensors", + "model.layers.26.mlp.experts.65.gate_proj.weight": "model-00027-of-00101.safetensors", + "model.layers.26.mlp.experts.65.up_proj.weight": "model-00027-of-00101.safetensors", + "model.layers.26.mlp.experts.65.down_proj.weight": "model-00027-of-00101.safetensors", + "model.layers.26.mlp.experts.66.gate_proj.weight": "model-00027-of-00101.safetensors", + "model.layers.26.mlp.experts.66.up_proj.weight": "model-00027-of-00101.safetensors", + "model.layers.26.mlp.experts.66.down_proj.weight": "model-00027-of-00101.safetensors", + "model.layers.26.mlp.experts.67.gate_proj.weight": "model-00027-of-00101.safetensors", + "model.layers.26.mlp.experts.67.up_proj.weight": "model-00027-of-00101.safetensors", + "model.layers.26.mlp.experts.67.down_proj.weight": "model-00028-of-00101.safetensors", + "model.layers.26.mlp.experts.68.gate_proj.weight": "model-00028-of-00101.safetensors", + "model.layers.26.mlp.experts.68.up_proj.weight": "model-00028-of-00101.safetensors", + "model.layers.26.mlp.experts.68.down_proj.weight": "model-00028-of-00101.safetensors", + "model.layers.26.mlp.experts.69.gate_proj.weight": "model-00028-of-00101.safetensors", + "model.layers.26.mlp.experts.69.up_proj.weight": "model-00028-of-00101.safetensors", + "model.layers.26.mlp.experts.69.down_proj.weight": "model-00028-of-00101.safetensors", + "model.layers.26.mlp.experts.70.gate_proj.weight": "model-00028-of-00101.safetensors", + "model.layers.26.mlp.experts.70.up_proj.weight": "model-00028-of-00101.safetensors", + "model.layers.26.mlp.experts.70.down_proj.weight": "model-00028-of-00101.safetensors", + "model.layers.26.mlp.experts.71.gate_proj.weight": "model-00028-of-00101.safetensors", + "model.layers.26.mlp.experts.71.up_proj.weight": "model-00028-of-00101.safetensors", + "model.layers.26.mlp.experts.71.down_proj.weight": "model-00028-of-00101.safetensors", + "model.layers.26.mlp.experts.72.gate_proj.weight": "model-00028-of-00101.safetensors", + "model.layers.26.mlp.experts.72.up_proj.weight": "model-00028-of-00101.safetensors", + "model.layers.26.mlp.experts.72.down_proj.weight": "model-00028-of-00101.safetensors", + "model.layers.26.mlp.experts.73.gate_proj.weight": "model-00028-of-00101.safetensors", + "model.layers.26.mlp.experts.73.up_proj.weight": "model-00028-of-00101.safetensors", + "model.layers.26.mlp.experts.73.down_proj.weight": "model-00028-of-00101.safetensors", + "model.layers.26.mlp.experts.74.gate_proj.weight": "model-00028-of-00101.safetensors", + "model.layers.26.mlp.experts.74.up_proj.weight": "model-00028-of-00101.safetensors", + "model.layers.26.mlp.experts.74.down_proj.weight": "model-00028-of-00101.safetensors", + "model.layers.26.mlp.experts.75.gate_proj.weight": "model-00028-of-00101.safetensors", + "model.layers.26.mlp.experts.75.up_proj.weight": "model-00028-of-00101.safetensors", + "model.layers.26.mlp.experts.75.down_proj.weight": "model-00028-of-00101.safetensors", + "model.layers.26.mlp.experts.76.gate_proj.weight": "model-00028-of-00101.safetensors", + "model.layers.26.mlp.experts.76.up_proj.weight": "model-00028-of-00101.safetensors", + "model.layers.26.mlp.experts.76.down_proj.weight": "model-00028-of-00101.safetensors", + "model.layers.26.mlp.experts.77.gate_proj.weight": "model-00028-of-00101.safetensors", + "model.layers.26.mlp.experts.77.up_proj.weight": "model-00028-of-00101.safetensors", + "model.layers.26.mlp.experts.77.down_proj.weight": "model-00028-of-00101.safetensors", + "model.layers.26.mlp.experts.78.gate_proj.weight": "model-00028-of-00101.safetensors", + "model.layers.26.mlp.experts.78.up_proj.weight": "model-00028-of-00101.safetensors", + "model.layers.26.mlp.experts.78.down_proj.weight": "model-00028-of-00101.safetensors", + "model.layers.26.mlp.experts.79.gate_proj.weight": "model-00028-of-00101.safetensors", + "model.layers.26.mlp.experts.79.up_proj.weight": "model-00028-of-00101.safetensors", + "model.layers.26.mlp.experts.79.down_proj.weight": "model-00028-of-00101.safetensors", + "model.layers.26.mlp.experts.80.gate_proj.weight": "model-00028-of-00101.safetensors", + "model.layers.26.mlp.experts.80.up_proj.weight": "model-00028-of-00101.safetensors", + "model.layers.26.mlp.experts.80.down_proj.weight": "model-00028-of-00101.safetensors", + "model.layers.26.mlp.experts.81.gate_proj.weight": "model-00028-of-00101.safetensors", + "model.layers.26.mlp.experts.81.up_proj.weight": "model-00028-of-00101.safetensors", + "model.layers.26.mlp.experts.81.down_proj.weight": "model-00028-of-00101.safetensors", + "model.layers.26.mlp.experts.82.gate_proj.weight": "model-00028-of-00101.safetensors", + "model.layers.26.mlp.experts.82.up_proj.weight": "model-00028-of-00101.safetensors", + "model.layers.26.mlp.experts.82.down_proj.weight": "model-00028-of-00101.safetensors", + "model.layers.26.mlp.experts.83.gate_proj.weight": "model-00028-of-00101.safetensors", + "model.layers.26.mlp.experts.83.up_proj.weight": "model-00028-of-00101.safetensors", + "model.layers.26.mlp.experts.83.down_proj.weight": "model-00028-of-00101.safetensors", + "model.layers.26.mlp.experts.84.gate_proj.weight": "model-00028-of-00101.safetensors", + "model.layers.26.mlp.experts.84.up_proj.weight": "model-00028-of-00101.safetensors", + "model.layers.26.mlp.experts.84.down_proj.weight": "model-00028-of-00101.safetensors", + "model.layers.26.mlp.experts.85.gate_proj.weight": "model-00028-of-00101.safetensors", + "model.layers.26.mlp.experts.85.up_proj.weight": "model-00028-of-00101.safetensors", + "model.layers.26.mlp.experts.85.down_proj.weight": "model-00028-of-00101.safetensors", + "model.layers.26.mlp.experts.86.gate_proj.weight": "model-00028-of-00101.safetensors", + "model.layers.26.mlp.experts.86.up_proj.weight": "model-00028-of-00101.safetensors", + "model.layers.26.mlp.experts.86.down_proj.weight": "model-00028-of-00101.safetensors", + "model.layers.26.mlp.experts.87.gate_proj.weight": "model-00028-of-00101.safetensors", + "model.layers.26.mlp.experts.87.up_proj.weight": "model-00028-of-00101.safetensors", + "model.layers.26.mlp.experts.87.down_proj.weight": "model-00028-of-00101.safetensors", + "model.layers.26.mlp.experts.88.gate_proj.weight": "model-00028-of-00101.safetensors", + "model.layers.26.mlp.experts.88.up_proj.weight": "model-00028-of-00101.safetensors", + "model.layers.26.mlp.experts.88.down_proj.weight": "model-00028-of-00101.safetensors", + "model.layers.26.mlp.experts.89.gate_proj.weight": "model-00028-of-00101.safetensors", + "model.layers.26.mlp.experts.89.up_proj.weight": "model-00028-of-00101.safetensors", + "model.layers.26.mlp.experts.89.down_proj.weight": "model-00028-of-00101.safetensors", + "model.layers.26.mlp.experts.90.gate_proj.weight": "model-00028-of-00101.safetensors", + "model.layers.26.mlp.experts.90.up_proj.weight": "model-00028-of-00101.safetensors", + "model.layers.26.mlp.experts.90.down_proj.weight": "model-00028-of-00101.safetensors", + "model.layers.26.mlp.experts.91.gate_proj.weight": "model-00028-of-00101.safetensors", + "model.layers.26.mlp.experts.91.up_proj.weight": "model-00028-of-00101.safetensors", + "model.layers.26.mlp.experts.91.down_proj.weight": "model-00028-of-00101.safetensors", + "model.layers.26.mlp.experts.92.gate_proj.weight": "model-00028-of-00101.safetensors", + "model.layers.26.mlp.experts.92.up_proj.weight": "model-00028-of-00101.safetensors", + "model.layers.26.mlp.experts.92.down_proj.weight": "model-00028-of-00101.safetensors", + "model.layers.26.mlp.experts.93.gate_proj.weight": "model-00028-of-00101.safetensors", + "model.layers.26.mlp.experts.93.up_proj.weight": "model-00028-of-00101.safetensors", + "model.layers.26.mlp.experts.93.down_proj.weight": "model-00028-of-00101.safetensors", + "model.layers.26.mlp.experts.94.gate_proj.weight": "model-00028-of-00101.safetensors", + "model.layers.26.mlp.experts.94.up_proj.weight": "model-00028-of-00101.safetensors", + "model.layers.26.mlp.experts.94.down_proj.weight": "model-00028-of-00101.safetensors", + "model.layers.26.mlp.experts.95.gate_proj.weight": "model-00028-of-00101.safetensors", + "model.layers.26.mlp.experts.95.up_proj.weight": "model-00028-of-00101.safetensors", + "model.layers.26.mlp.experts.95.down_proj.weight": "model-00028-of-00101.safetensors", + "model.layers.26.mlp.experts.96.gate_proj.weight": "model-00028-of-00101.safetensors", + "model.layers.26.mlp.experts.96.up_proj.weight": "model-00028-of-00101.safetensors", + "model.layers.26.mlp.experts.96.down_proj.weight": "model-00028-of-00101.safetensors", + "model.layers.26.mlp.experts.97.gate_proj.weight": "model-00028-of-00101.safetensors", + "model.layers.26.mlp.experts.97.up_proj.weight": "model-00028-of-00101.safetensors", + "model.layers.26.mlp.experts.97.down_proj.weight": "model-00028-of-00101.safetensors", + "model.layers.26.mlp.experts.98.gate_proj.weight": "model-00028-of-00101.safetensors", + "model.layers.26.mlp.experts.98.up_proj.weight": "model-00028-of-00101.safetensors", + "model.layers.26.mlp.experts.98.down_proj.weight": "model-00028-of-00101.safetensors", + "model.layers.26.mlp.experts.99.gate_proj.weight": "model-00028-of-00101.safetensors", + "model.layers.26.mlp.experts.99.up_proj.weight": "model-00028-of-00101.safetensors", + "model.layers.26.mlp.experts.99.down_proj.weight": "model-00028-of-00101.safetensors", + "model.layers.26.mlp.experts.100.gate_proj.weight": "model-00028-of-00101.safetensors", + "model.layers.26.mlp.experts.100.up_proj.weight": "model-00028-of-00101.safetensors", + "model.layers.26.mlp.experts.100.down_proj.weight": "model-00028-of-00101.safetensors", + "model.layers.26.mlp.experts.101.gate_proj.weight": "model-00028-of-00101.safetensors", + "model.layers.26.mlp.experts.101.up_proj.weight": "model-00028-of-00101.safetensors", + "model.layers.26.mlp.experts.101.down_proj.weight": "model-00028-of-00101.safetensors", + "model.layers.26.mlp.experts.102.gate_proj.weight": "model-00028-of-00101.safetensors", + "model.layers.26.mlp.experts.102.up_proj.weight": "model-00028-of-00101.safetensors", + "model.layers.26.mlp.experts.102.down_proj.weight": "model-00028-of-00101.safetensors", + "model.layers.26.mlp.experts.103.gate_proj.weight": "model-00028-of-00101.safetensors", + "model.layers.26.mlp.experts.103.up_proj.weight": "model-00028-of-00101.safetensors", + "model.layers.26.mlp.experts.103.down_proj.weight": "model-00028-of-00101.safetensors", + "model.layers.26.mlp.experts.104.gate_proj.weight": "model-00028-of-00101.safetensors", + "model.layers.26.mlp.experts.104.up_proj.weight": "model-00028-of-00101.safetensors", + "model.layers.26.mlp.experts.104.down_proj.weight": "model-00028-of-00101.safetensors", + "model.layers.26.mlp.experts.105.gate_proj.weight": "model-00028-of-00101.safetensors", + "model.layers.26.mlp.experts.105.up_proj.weight": "model-00028-of-00101.safetensors", + "model.layers.26.mlp.experts.105.down_proj.weight": "model-00028-of-00101.safetensors", + "model.layers.26.mlp.experts.106.gate_proj.weight": "model-00028-of-00101.safetensors", + "model.layers.26.mlp.experts.106.up_proj.weight": "model-00028-of-00101.safetensors", + "model.layers.26.mlp.experts.106.down_proj.weight": "model-00028-of-00101.safetensors", + "model.layers.26.mlp.experts.107.gate_proj.weight": "model-00028-of-00101.safetensors", + "model.layers.26.mlp.experts.107.up_proj.weight": "model-00028-of-00101.safetensors", + "model.layers.26.mlp.experts.107.down_proj.weight": "model-00028-of-00101.safetensors", + "model.layers.26.mlp.experts.108.gate_proj.weight": "model-00028-of-00101.safetensors", + "model.layers.26.mlp.experts.108.up_proj.weight": "model-00028-of-00101.safetensors", + "model.layers.26.mlp.experts.108.down_proj.weight": "model-00028-of-00101.safetensors", + "model.layers.26.mlp.experts.109.gate_proj.weight": "model-00028-of-00101.safetensors", + "model.layers.26.mlp.experts.109.up_proj.weight": "model-00028-of-00101.safetensors", + "model.layers.26.mlp.experts.109.down_proj.weight": "model-00028-of-00101.safetensors", + "model.layers.26.mlp.experts.110.gate_proj.weight": "model-00028-of-00101.safetensors", + "model.layers.26.mlp.experts.110.up_proj.weight": "model-00028-of-00101.safetensors", + "model.layers.26.mlp.experts.110.down_proj.weight": "model-00028-of-00101.safetensors", + "model.layers.26.mlp.experts.111.gate_proj.weight": "model-00028-of-00101.safetensors", + "model.layers.26.mlp.experts.111.up_proj.weight": "model-00028-of-00101.safetensors", + "model.layers.26.mlp.experts.111.down_proj.weight": "model-00028-of-00101.safetensors", + "model.layers.26.mlp.experts.112.gate_proj.weight": "model-00028-of-00101.safetensors", + "model.layers.26.mlp.experts.112.up_proj.weight": "model-00028-of-00101.safetensors", + "model.layers.26.mlp.experts.112.down_proj.weight": "model-00028-of-00101.safetensors", + "model.layers.26.mlp.experts.113.gate_proj.weight": "model-00028-of-00101.safetensors", + "model.layers.26.mlp.experts.113.up_proj.weight": "model-00028-of-00101.safetensors", + "model.layers.26.mlp.experts.113.down_proj.weight": "model-00028-of-00101.safetensors", + "model.layers.26.mlp.experts.114.gate_proj.weight": "model-00028-of-00101.safetensors", + "model.layers.26.mlp.experts.114.up_proj.weight": "model-00028-of-00101.safetensors", + "model.layers.26.mlp.experts.114.down_proj.weight": "model-00028-of-00101.safetensors", + "model.layers.26.mlp.experts.115.gate_proj.weight": "model-00028-of-00101.safetensors", + "model.layers.26.mlp.experts.115.up_proj.weight": "model-00028-of-00101.safetensors", + "model.layers.26.mlp.experts.115.down_proj.weight": "model-00028-of-00101.safetensors", + "model.layers.26.mlp.experts.116.gate_proj.weight": "model-00028-of-00101.safetensors", + "model.layers.26.mlp.experts.116.up_proj.weight": "model-00028-of-00101.safetensors", + "model.layers.26.mlp.experts.116.down_proj.weight": "model-00028-of-00101.safetensors", + "model.layers.26.mlp.experts.117.gate_proj.weight": "model-00028-of-00101.safetensors", + "model.layers.26.mlp.experts.117.up_proj.weight": "model-00028-of-00101.safetensors", + "model.layers.26.mlp.experts.117.down_proj.weight": "model-00028-of-00101.safetensors", + "model.layers.26.mlp.experts.118.gate_proj.weight": "model-00028-of-00101.safetensors", + "model.layers.26.mlp.experts.118.up_proj.weight": "model-00028-of-00101.safetensors", + "model.layers.26.mlp.experts.118.down_proj.weight": "model-00028-of-00101.safetensors", + "model.layers.26.mlp.experts.119.gate_proj.weight": "model-00028-of-00101.safetensors", + "model.layers.26.mlp.experts.119.up_proj.weight": "model-00028-of-00101.safetensors", + "model.layers.26.mlp.experts.119.down_proj.weight": "model-00028-of-00101.safetensors", + "model.layers.26.mlp.gate.weight": "model-00028-of-00101.safetensors", + "model.layers.26.mlp.gate.e_score_correction_bias": "model-00028-of-00101.safetensors", + "model.layers.26.mlp.shared_experts.gate_proj.weight": "model-00028-of-00101.safetensors", + "model.layers.26.mlp.shared_experts.up_proj.weight": "model-00028-of-00101.safetensors", + "model.layers.26.mlp.shared_experts.down_proj.weight": "model-00028-of-00101.safetensors", + "model.layers.26.input_layernorm.weight": "model-00028-of-00101.safetensors", + "model.layers.26.post_attention_layernorm.weight": "model-00028-of-00101.safetensors", + "model.layers.27.self_attn.q_proj.weight": "model-00028-of-00101.safetensors", + "model.layers.27.self_attn.q_proj.bias": "model-00028-of-00101.safetensors", + "model.layers.27.self_attn.k_proj.weight": "model-00028-of-00101.safetensors", + "model.layers.27.self_attn.k_proj.bias": "model-00028-of-00101.safetensors", + "model.layers.27.self_attn.v_proj.weight": "model-00028-of-00101.safetensors", + "model.layers.27.self_attn.v_proj.bias": "model-00028-of-00101.safetensors", + "model.layers.27.self_attn.o_proj.weight": "model-00028-of-00101.safetensors", + "model.layers.27.self_attn.q_norm.weight": "model-00028-of-00101.safetensors", + "model.layers.27.self_attn.k_norm.weight": "model-00028-of-00101.safetensors", + "model.layers.27.mlp.experts.0.gate_proj.weight": "model-00028-of-00101.safetensors", + "model.layers.27.mlp.experts.0.up_proj.weight": "model-00028-of-00101.safetensors", + "model.layers.27.mlp.experts.0.down_proj.weight": "model-00028-of-00101.safetensors", + "model.layers.27.mlp.experts.1.gate_proj.weight": "model-00028-of-00101.safetensors", + "model.layers.27.mlp.experts.1.up_proj.weight": "model-00028-of-00101.safetensors", + "model.layers.27.mlp.experts.1.down_proj.weight": "model-00028-of-00101.safetensors", + "model.layers.27.mlp.experts.2.gate_proj.weight": "model-00028-of-00101.safetensors", + "model.layers.27.mlp.experts.2.up_proj.weight": "model-00028-of-00101.safetensors", + "model.layers.27.mlp.experts.2.down_proj.weight": "model-00028-of-00101.safetensors", + "model.layers.27.mlp.experts.3.gate_proj.weight": "model-00028-of-00101.safetensors", + "model.layers.27.mlp.experts.3.up_proj.weight": "model-00028-of-00101.safetensors", + "model.layers.27.mlp.experts.3.down_proj.weight": "model-00028-of-00101.safetensors", + "model.layers.27.mlp.experts.4.gate_proj.weight": "model-00028-of-00101.safetensors", + "model.layers.27.mlp.experts.4.up_proj.weight": "model-00028-of-00101.safetensors", + "model.layers.27.mlp.experts.4.down_proj.weight": "model-00028-of-00101.safetensors", + "model.layers.27.mlp.experts.5.gate_proj.weight": "model-00028-of-00101.safetensors", + "model.layers.27.mlp.experts.5.up_proj.weight": "model-00028-of-00101.safetensors", + "model.layers.27.mlp.experts.5.down_proj.weight": "model-00028-of-00101.safetensors", + "model.layers.27.mlp.experts.6.gate_proj.weight": "model-00028-of-00101.safetensors", + "model.layers.27.mlp.experts.6.up_proj.weight": "model-00028-of-00101.safetensors", + "model.layers.27.mlp.experts.6.down_proj.weight": "model-00028-of-00101.safetensors", + "model.layers.27.mlp.experts.7.gate_proj.weight": "model-00028-of-00101.safetensors", + "model.layers.27.mlp.experts.7.up_proj.weight": "model-00028-of-00101.safetensors", + "model.layers.27.mlp.experts.7.down_proj.weight": "model-00028-of-00101.safetensors", + "model.layers.27.mlp.experts.8.gate_proj.weight": "model-00028-of-00101.safetensors", + "model.layers.27.mlp.experts.8.up_proj.weight": "model-00028-of-00101.safetensors", + "model.layers.27.mlp.experts.8.down_proj.weight": "model-00028-of-00101.safetensors", + "model.layers.27.mlp.experts.9.gate_proj.weight": "model-00028-of-00101.safetensors", + "model.layers.27.mlp.experts.9.up_proj.weight": "model-00028-of-00101.safetensors", + "model.layers.27.mlp.experts.9.down_proj.weight": "model-00028-of-00101.safetensors", + "model.layers.27.mlp.experts.10.gate_proj.weight": "model-00028-of-00101.safetensors", + "model.layers.27.mlp.experts.10.up_proj.weight": "model-00028-of-00101.safetensors", + "model.layers.27.mlp.experts.10.down_proj.weight": "model-00028-of-00101.safetensors", + "model.layers.27.mlp.experts.11.gate_proj.weight": "model-00028-of-00101.safetensors", + "model.layers.27.mlp.experts.11.up_proj.weight": "model-00028-of-00101.safetensors", + "model.layers.27.mlp.experts.11.down_proj.weight": "model-00028-of-00101.safetensors", + "model.layers.27.mlp.experts.12.gate_proj.weight": "model-00028-of-00101.safetensors", + "model.layers.27.mlp.experts.12.up_proj.weight": "model-00028-of-00101.safetensors", + "model.layers.27.mlp.experts.12.down_proj.weight": "model-00028-of-00101.safetensors", + "model.layers.27.mlp.experts.13.gate_proj.weight": "model-00028-of-00101.safetensors", + "model.layers.27.mlp.experts.13.up_proj.weight": "model-00028-of-00101.safetensors", + "model.layers.27.mlp.experts.13.down_proj.weight": "model-00028-of-00101.safetensors", + "model.layers.27.mlp.experts.14.gate_proj.weight": "model-00028-of-00101.safetensors", + "model.layers.27.mlp.experts.14.up_proj.weight": "model-00028-of-00101.safetensors", + "model.layers.27.mlp.experts.14.down_proj.weight": "model-00028-of-00101.safetensors", + "model.layers.27.mlp.experts.15.gate_proj.weight": "model-00028-of-00101.safetensors", + "model.layers.27.mlp.experts.15.up_proj.weight": "model-00028-of-00101.safetensors", + "model.layers.27.mlp.experts.15.down_proj.weight": "model-00028-of-00101.safetensors", + "model.layers.27.mlp.experts.16.gate_proj.weight": "model-00028-of-00101.safetensors", + "model.layers.27.mlp.experts.16.up_proj.weight": "model-00028-of-00101.safetensors", + "model.layers.27.mlp.experts.16.down_proj.weight": "model-00028-of-00101.safetensors", + "model.layers.27.mlp.experts.17.gate_proj.weight": "model-00028-of-00101.safetensors", + "model.layers.27.mlp.experts.17.up_proj.weight": "model-00028-of-00101.safetensors", + "model.layers.27.mlp.experts.17.down_proj.weight": "model-00028-of-00101.safetensors", + "model.layers.27.mlp.experts.18.gate_proj.weight": "model-00028-of-00101.safetensors", + "model.layers.27.mlp.experts.18.up_proj.weight": "model-00028-of-00101.safetensors", + "model.layers.27.mlp.experts.18.down_proj.weight": "model-00028-of-00101.safetensors", + "model.layers.27.mlp.experts.19.gate_proj.weight": "model-00028-of-00101.safetensors", + "model.layers.27.mlp.experts.19.up_proj.weight": "model-00028-of-00101.safetensors", + "model.layers.27.mlp.experts.19.down_proj.weight": "model-00028-of-00101.safetensors", + "model.layers.27.mlp.experts.20.gate_proj.weight": "model-00028-of-00101.safetensors", + "model.layers.27.mlp.experts.20.up_proj.weight": "model-00028-of-00101.safetensors", + "model.layers.27.mlp.experts.20.down_proj.weight": "model-00028-of-00101.safetensors", + "model.layers.27.mlp.experts.21.gate_proj.weight": "model-00028-of-00101.safetensors", + "model.layers.27.mlp.experts.21.up_proj.weight": "model-00028-of-00101.safetensors", + "model.layers.27.mlp.experts.21.down_proj.weight": "model-00028-of-00101.safetensors", + "model.layers.27.mlp.experts.22.gate_proj.weight": "model-00028-of-00101.safetensors", + "model.layers.27.mlp.experts.22.up_proj.weight": "model-00028-of-00101.safetensors", + "model.layers.27.mlp.experts.22.down_proj.weight": "model-00028-of-00101.safetensors", + "model.layers.27.mlp.experts.23.gate_proj.weight": "model-00028-of-00101.safetensors", + "model.layers.27.mlp.experts.23.up_proj.weight": "model-00028-of-00101.safetensors", + "model.layers.27.mlp.experts.23.down_proj.weight": "model-00028-of-00101.safetensors", + "model.layers.27.mlp.experts.24.gate_proj.weight": "model-00028-of-00101.safetensors", + "model.layers.27.mlp.experts.24.up_proj.weight": "model-00028-of-00101.safetensors", + "model.layers.27.mlp.experts.24.down_proj.weight": "model-00028-of-00101.safetensors", + "model.layers.27.mlp.experts.25.gate_proj.weight": "model-00028-of-00101.safetensors", + "model.layers.27.mlp.experts.25.up_proj.weight": "model-00028-of-00101.safetensors", + "model.layers.27.mlp.experts.25.down_proj.weight": "model-00028-of-00101.safetensors", + "model.layers.27.mlp.experts.26.gate_proj.weight": "model-00028-of-00101.safetensors", + "model.layers.27.mlp.experts.26.up_proj.weight": "model-00028-of-00101.safetensors", + "model.layers.27.mlp.experts.26.down_proj.weight": "model-00028-of-00101.safetensors", + "model.layers.27.mlp.experts.27.gate_proj.weight": "model-00028-of-00101.safetensors", + "model.layers.27.mlp.experts.27.up_proj.weight": "model-00028-of-00101.safetensors", + "model.layers.27.mlp.experts.27.down_proj.weight": "model-00028-of-00101.safetensors", + "model.layers.27.mlp.experts.28.gate_proj.weight": "model-00028-of-00101.safetensors", + "model.layers.27.mlp.experts.28.up_proj.weight": "model-00028-of-00101.safetensors", + "model.layers.27.mlp.experts.28.down_proj.weight": "model-00028-of-00101.safetensors", + "model.layers.27.mlp.experts.29.gate_proj.weight": "model-00028-of-00101.safetensors", + "model.layers.27.mlp.experts.29.up_proj.weight": "model-00028-of-00101.safetensors", + "model.layers.27.mlp.experts.29.down_proj.weight": "model-00028-of-00101.safetensors", + "model.layers.27.mlp.experts.30.gate_proj.weight": "model-00028-of-00101.safetensors", + "model.layers.27.mlp.experts.30.up_proj.weight": "model-00028-of-00101.safetensors", + "model.layers.27.mlp.experts.30.down_proj.weight": "model-00028-of-00101.safetensors", + "model.layers.27.mlp.experts.31.gate_proj.weight": "model-00028-of-00101.safetensors", + "model.layers.27.mlp.experts.31.up_proj.weight": "model-00028-of-00101.safetensors", + "model.layers.27.mlp.experts.31.down_proj.weight": "model-00028-of-00101.safetensors", + "model.layers.27.mlp.experts.32.gate_proj.weight": "model-00028-of-00101.safetensors", + "model.layers.27.mlp.experts.32.up_proj.weight": "model-00028-of-00101.safetensors", + "model.layers.27.mlp.experts.32.down_proj.weight": "model-00028-of-00101.safetensors", + "model.layers.27.mlp.experts.33.gate_proj.weight": "model-00028-of-00101.safetensors", + "model.layers.27.mlp.experts.33.up_proj.weight": "model-00028-of-00101.safetensors", + "model.layers.27.mlp.experts.33.down_proj.weight": "model-00028-of-00101.safetensors", + "model.layers.27.mlp.experts.34.gate_proj.weight": "model-00028-of-00101.safetensors", + "model.layers.27.mlp.experts.34.up_proj.weight": "model-00028-of-00101.safetensors", + "model.layers.27.mlp.experts.34.down_proj.weight": "model-00028-of-00101.safetensors", + "model.layers.27.mlp.experts.35.gate_proj.weight": "model-00028-of-00101.safetensors", + "model.layers.27.mlp.experts.35.up_proj.weight": "model-00028-of-00101.safetensors", + "model.layers.27.mlp.experts.35.down_proj.weight": "model-00028-of-00101.safetensors", + "model.layers.27.mlp.experts.36.gate_proj.weight": "model-00028-of-00101.safetensors", + "model.layers.27.mlp.experts.36.up_proj.weight": "model-00028-of-00101.safetensors", + "model.layers.27.mlp.experts.36.down_proj.weight": "model-00028-of-00101.safetensors", + "model.layers.27.mlp.experts.37.gate_proj.weight": "model-00028-of-00101.safetensors", + "model.layers.27.mlp.experts.37.up_proj.weight": "model-00028-of-00101.safetensors", + "model.layers.27.mlp.experts.37.down_proj.weight": "model-00028-of-00101.safetensors", + "model.layers.27.mlp.experts.38.gate_proj.weight": "model-00028-of-00101.safetensors", + "model.layers.27.mlp.experts.38.up_proj.weight": "model-00028-of-00101.safetensors", + "model.layers.27.mlp.experts.38.down_proj.weight": "model-00028-of-00101.safetensors", + "model.layers.27.mlp.experts.39.gate_proj.weight": "model-00028-of-00101.safetensors", + "model.layers.27.mlp.experts.39.up_proj.weight": "model-00028-of-00101.safetensors", + "model.layers.27.mlp.experts.39.down_proj.weight": "model-00028-of-00101.safetensors", + "model.layers.27.mlp.experts.40.gate_proj.weight": "model-00028-of-00101.safetensors", + "model.layers.27.mlp.experts.40.up_proj.weight": "model-00028-of-00101.safetensors", + "model.layers.27.mlp.experts.40.down_proj.weight": "model-00028-of-00101.safetensors", + "model.layers.27.mlp.experts.41.gate_proj.weight": "model-00028-of-00101.safetensors", + "model.layers.27.mlp.experts.41.up_proj.weight": "model-00028-of-00101.safetensors", + "model.layers.27.mlp.experts.41.down_proj.weight": "model-00028-of-00101.safetensors", + "model.layers.27.mlp.experts.42.gate_proj.weight": "model-00028-of-00101.safetensors", + "model.layers.27.mlp.experts.42.up_proj.weight": "model-00028-of-00101.safetensors", + "model.layers.27.mlp.experts.42.down_proj.weight": "model-00028-of-00101.safetensors", + "model.layers.27.mlp.experts.43.gate_proj.weight": "model-00028-of-00101.safetensors", + "model.layers.27.mlp.experts.43.up_proj.weight": "model-00028-of-00101.safetensors", + "model.layers.27.mlp.experts.43.down_proj.weight": "model-00028-of-00101.safetensors", + "model.layers.27.mlp.experts.44.gate_proj.weight": "model-00028-of-00101.safetensors", + "model.layers.27.mlp.experts.44.up_proj.weight": "model-00028-of-00101.safetensors", + "model.layers.27.mlp.experts.44.down_proj.weight": "model-00028-of-00101.safetensors", + "model.layers.27.mlp.experts.45.gate_proj.weight": "model-00028-of-00101.safetensors", + "model.layers.27.mlp.experts.45.up_proj.weight": "model-00028-of-00101.safetensors", + "model.layers.27.mlp.experts.45.down_proj.weight": "model-00028-of-00101.safetensors", + "model.layers.27.mlp.experts.46.gate_proj.weight": "model-00028-of-00101.safetensors", + "model.layers.27.mlp.experts.46.up_proj.weight": "model-00028-of-00101.safetensors", + "model.layers.27.mlp.experts.46.down_proj.weight": "model-00028-of-00101.safetensors", + "model.layers.27.mlp.experts.47.gate_proj.weight": "model-00028-of-00101.safetensors", + "model.layers.27.mlp.experts.47.up_proj.weight": "model-00028-of-00101.safetensors", + "model.layers.27.mlp.experts.47.down_proj.weight": "model-00028-of-00101.safetensors", + "model.layers.27.mlp.experts.48.gate_proj.weight": "model-00028-of-00101.safetensors", + "model.layers.27.mlp.experts.48.up_proj.weight": "model-00028-of-00101.safetensors", + "model.layers.27.mlp.experts.48.down_proj.weight": "model-00028-of-00101.safetensors", + "model.layers.27.mlp.experts.49.gate_proj.weight": "model-00028-of-00101.safetensors", + "model.layers.27.mlp.experts.49.up_proj.weight": "model-00028-of-00101.safetensors", + "model.layers.27.mlp.experts.49.down_proj.weight": "model-00028-of-00101.safetensors", + "model.layers.27.mlp.experts.50.gate_proj.weight": "model-00028-of-00101.safetensors", + "model.layers.27.mlp.experts.50.up_proj.weight": "model-00028-of-00101.safetensors", + "model.layers.27.mlp.experts.50.down_proj.weight": "model-00028-of-00101.safetensors", + "model.layers.27.mlp.experts.51.gate_proj.weight": "model-00028-of-00101.safetensors", + "model.layers.27.mlp.experts.51.up_proj.weight": "model-00028-of-00101.safetensors", + "model.layers.27.mlp.experts.51.down_proj.weight": "model-00028-of-00101.safetensors", + "model.layers.27.mlp.experts.52.gate_proj.weight": "model-00028-of-00101.safetensors", + "model.layers.27.mlp.experts.52.up_proj.weight": "model-00028-of-00101.safetensors", + "model.layers.27.mlp.experts.52.down_proj.weight": "model-00028-of-00101.safetensors", + "model.layers.27.mlp.experts.53.gate_proj.weight": "model-00028-of-00101.safetensors", + "model.layers.27.mlp.experts.53.up_proj.weight": "model-00028-of-00101.safetensors", + "model.layers.27.mlp.experts.53.down_proj.weight": "model-00028-of-00101.safetensors", + "model.layers.27.mlp.experts.54.gate_proj.weight": "model-00028-of-00101.safetensors", + "model.layers.27.mlp.experts.54.up_proj.weight": "model-00029-of-00101.safetensors", + "model.layers.27.mlp.experts.54.down_proj.weight": "model-00029-of-00101.safetensors", + "model.layers.27.mlp.experts.55.gate_proj.weight": "model-00029-of-00101.safetensors", + "model.layers.27.mlp.experts.55.up_proj.weight": "model-00029-of-00101.safetensors", + "model.layers.27.mlp.experts.55.down_proj.weight": "model-00029-of-00101.safetensors", + "model.layers.27.mlp.experts.56.gate_proj.weight": "model-00029-of-00101.safetensors", + "model.layers.27.mlp.experts.56.up_proj.weight": "model-00029-of-00101.safetensors", + "model.layers.27.mlp.experts.56.down_proj.weight": "model-00029-of-00101.safetensors", + "model.layers.27.mlp.experts.57.gate_proj.weight": "model-00029-of-00101.safetensors", + "model.layers.27.mlp.experts.57.up_proj.weight": "model-00029-of-00101.safetensors", + "model.layers.27.mlp.experts.57.down_proj.weight": "model-00029-of-00101.safetensors", + "model.layers.27.mlp.experts.58.gate_proj.weight": "model-00029-of-00101.safetensors", + "model.layers.27.mlp.experts.58.up_proj.weight": "model-00029-of-00101.safetensors", + "model.layers.27.mlp.experts.58.down_proj.weight": "model-00029-of-00101.safetensors", + "model.layers.27.mlp.experts.59.gate_proj.weight": "model-00029-of-00101.safetensors", + "model.layers.27.mlp.experts.59.up_proj.weight": "model-00029-of-00101.safetensors", + "model.layers.27.mlp.experts.59.down_proj.weight": "model-00029-of-00101.safetensors", + "model.layers.27.mlp.experts.60.gate_proj.weight": "model-00029-of-00101.safetensors", + "model.layers.27.mlp.experts.60.up_proj.weight": "model-00029-of-00101.safetensors", + "model.layers.27.mlp.experts.60.down_proj.weight": "model-00029-of-00101.safetensors", + "model.layers.27.mlp.experts.61.gate_proj.weight": "model-00029-of-00101.safetensors", + "model.layers.27.mlp.experts.61.up_proj.weight": "model-00029-of-00101.safetensors", + "model.layers.27.mlp.experts.61.down_proj.weight": "model-00029-of-00101.safetensors", + "model.layers.27.mlp.experts.62.gate_proj.weight": "model-00029-of-00101.safetensors", + "model.layers.27.mlp.experts.62.up_proj.weight": "model-00029-of-00101.safetensors", + "model.layers.27.mlp.experts.62.down_proj.weight": "model-00029-of-00101.safetensors", + "model.layers.27.mlp.experts.63.gate_proj.weight": "model-00029-of-00101.safetensors", + "model.layers.27.mlp.experts.63.up_proj.weight": "model-00029-of-00101.safetensors", + "model.layers.27.mlp.experts.63.down_proj.weight": "model-00029-of-00101.safetensors", + "model.layers.27.mlp.experts.64.gate_proj.weight": "model-00029-of-00101.safetensors", + "model.layers.27.mlp.experts.64.up_proj.weight": "model-00029-of-00101.safetensors", + "model.layers.27.mlp.experts.64.down_proj.weight": "model-00029-of-00101.safetensors", + "model.layers.27.mlp.experts.65.gate_proj.weight": "model-00029-of-00101.safetensors", + "model.layers.27.mlp.experts.65.up_proj.weight": "model-00029-of-00101.safetensors", + "model.layers.27.mlp.experts.65.down_proj.weight": "model-00029-of-00101.safetensors", + "model.layers.27.mlp.experts.66.gate_proj.weight": "model-00029-of-00101.safetensors", + "model.layers.27.mlp.experts.66.up_proj.weight": "model-00029-of-00101.safetensors", + "model.layers.27.mlp.experts.66.down_proj.weight": "model-00029-of-00101.safetensors", + "model.layers.27.mlp.experts.67.gate_proj.weight": "model-00029-of-00101.safetensors", + "model.layers.27.mlp.experts.67.up_proj.weight": "model-00029-of-00101.safetensors", + "model.layers.27.mlp.experts.67.down_proj.weight": "model-00029-of-00101.safetensors", + "model.layers.27.mlp.experts.68.gate_proj.weight": "model-00029-of-00101.safetensors", + "model.layers.27.mlp.experts.68.up_proj.weight": "model-00029-of-00101.safetensors", + "model.layers.27.mlp.experts.68.down_proj.weight": "model-00029-of-00101.safetensors", + "model.layers.27.mlp.experts.69.gate_proj.weight": "model-00029-of-00101.safetensors", + "model.layers.27.mlp.experts.69.up_proj.weight": "model-00029-of-00101.safetensors", + "model.layers.27.mlp.experts.69.down_proj.weight": "model-00029-of-00101.safetensors", + "model.layers.27.mlp.experts.70.gate_proj.weight": "model-00029-of-00101.safetensors", + "model.layers.27.mlp.experts.70.up_proj.weight": "model-00029-of-00101.safetensors", + "model.layers.27.mlp.experts.70.down_proj.weight": "model-00029-of-00101.safetensors", + "model.layers.27.mlp.experts.71.gate_proj.weight": "model-00029-of-00101.safetensors", + "model.layers.27.mlp.experts.71.up_proj.weight": "model-00029-of-00101.safetensors", + "model.layers.27.mlp.experts.71.down_proj.weight": "model-00029-of-00101.safetensors", + "model.layers.27.mlp.experts.72.gate_proj.weight": "model-00029-of-00101.safetensors", + "model.layers.27.mlp.experts.72.up_proj.weight": "model-00029-of-00101.safetensors", + "model.layers.27.mlp.experts.72.down_proj.weight": "model-00029-of-00101.safetensors", + "model.layers.27.mlp.experts.73.gate_proj.weight": "model-00029-of-00101.safetensors", + "model.layers.27.mlp.experts.73.up_proj.weight": "model-00029-of-00101.safetensors", + "model.layers.27.mlp.experts.73.down_proj.weight": "model-00029-of-00101.safetensors", + "model.layers.27.mlp.experts.74.gate_proj.weight": "model-00029-of-00101.safetensors", + "model.layers.27.mlp.experts.74.up_proj.weight": "model-00029-of-00101.safetensors", + "model.layers.27.mlp.experts.74.down_proj.weight": "model-00029-of-00101.safetensors", + "model.layers.27.mlp.experts.75.gate_proj.weight": "model-00029-of-00101.safetensors", + "model.layers.27.mlp.experts.75.up_proj.weight": "model-00029-of-00101.safetensors", + "model.layers.27.mlp.experts.75.down_proj.weight": "model-00029-of-00101.safetensors", + "model.layers.27.mlp.experts.76.gate_proj.weight": "model-00029-of-00101.safetensors", + "model.layers.27.mlp.experts.76.up_proj.weight": "model-00029-of-00101.safetensors", + "model.layers.27.mlp.experts.76.down_proj.weight": "model-00029-of-00101.safetensors", + "model.layers.27.mlp.experts.77.gate_proj.weight": "model-00029-of-00101.safetensors", + "model.layers.27.mlp.experts.77.up_proj.weight": "model-00029-of-00101.safetensors", + "model.layers.27.mlp.experts.77.down_proj.weight": "model-00029-of-00101.safetensors", + "model.layers.27.mlp.experts.78.gate_proj.weight": "model-00029-of-00101.safetensors", + "model.layers.27.mlp.experts.78.up_proj.weight": "model-00029-of-00101.safetensors", + "model.layers.27.mlp.experts.78.down_proj.weight": "model-00029-of-00101.safetensors", + "model.layers.27.mlp.experts.79.gate_proj.weight": "model-00029-of-00101.safetensors", + "model.layers.27.mlp.experts.79.up_proj.weight": "model-00029-of-00101.safetensors", + "model.layers.27.mlp.experts.79.down_proj.weight": "model-00029-of-00101.safetensors", + "model.layers.27.mlp.experts.80.gate_proj.weight": "model-00029-of-00101.safetensors", + "model.layers.27.mlp.experts.80.up_proj.weight": "model-00029-of-00101.safetensors", + "model.layers.27.mlp.experts.80.down_proj.weight": "model-00029-of-00101.safetensors", + "model.layers.27.mlp.experts.81.gate_proj.weight": "model-00029-of-00101.safetensors", + "model.layers.27.mlp.experts.81.up_proj.weight": "model-00029-of-00101.safetensors", + "model.layers.27.mlp.experts.81.down_proj.weight": "model-00029-of-00101.safetensors", + "model.layers.27.mlp.experts.82.gate_proj.weight": "model-00029-of-00101.safetensors", + "model.layers.27.mlp.experts.82.up_proj.weight": "model-00029-of-00101.safetensors", + "model.layers.27.mlp.experts.82.down_proj.weight": "model-00029-of-00101.safetensors", + "model.layers.27.mlp.experts.83.gate_proj.weight": "model-00029-of-00101.safetensors", + "model.layers.27.mlp.experts.83.up_proj.weight": "model-00029-of-00101.safetensors", + "model.layers.27.mlp.experts.83.down_proj.weight": "model-00029-of-00101.safetensors", + "model.layers.27.mlp.experts.84.gate_proj.weight": "model-00029-of-00101.safetensors", + "model.layers.27.mlp.experts.84.up_proj.weight": "model-00029-of-00101.safetensors", + "model.layers.27.mlp.experts.84.down_proj.weight": "model-00029-of-00101.safetensors", + "model.layers.27.mlp.experts.85.gate_proj.weight": "model-00029-of-00101.safetensors", + "model.layers.27.mlp.experts.85.up_proj.weight": "model-00029-of-00101.safetensors", + "model.layers.27.mlp.experts.85.down_proj.weight": "model-00029-of-00101.safetensors", + "model.layers.27.mlp.experts.86.gate_proj.weight": "model-00029-of-00101.safetensors", + "model.layers.27.mlp.experts.86.up_proj.weight": "model-00029-of-00101.safetensors", + "model.layers.27.mlp.experts.86.down_proj.weight": "model-00029-of-00101.safetensors", + "model.layers.27.mlp.experts.87.gate_proj.weight": "model-00029-of-00101.safetensors", + "model.layers.27.mlp.experts.87.up_proj.weight": "model-00029-of-00101.safetensors", + "model.layers.27.mlp.experts.87.down_proj.weight": "model-00029-of-00101.safetensors", + "model.layers.27.mlp.experts.88.gate_proj.weight": "model-00029-of-00101.safetensors", + "model.layers.27.mlp.experts.88.up_proj.weight": "model-00029-of-00101.safetensors", + "model.layers.27.mlp.experts.88.down_proj.weight": "model-00029-of-00101.safetensors", + "model.layers.27.mlp.experts.89.gate_proj.weight": "model-00029-of-00101.safetensors", + "model.layers.27.mlp.experts.89.up_proj.weight": "model-00029-of-00101.safetensors", + "model.layers.27.mlp.experts.89.down_proj.weight": "model-00029-of-00101.safetensors", + "model.layers.27.mlp.experts.90.gate_proj.weight": "model-00029-of-00101.safetensors", + "model.layers.27.mlp.experts.90.up_proj.weight": "model-00029-of-00101.safetensors", + "model.layers.27.mlp.experts.90.down_proj.weight": "model-00029-of-00101.safetensors", + "model.layers.27.mlp.experts.91.gate_proj.weight": "model-00029-of-00101.safetensors", + "model.layers.27.mlp.experts.91.up_proj.weight": "model-00029-of-00101.safetensors", + "model.layers.27.mlp.experts.91.down_proj.weight": "model-00029-of-00101.safetensors", + "model.layers.27.mlp.experts.92.gate_proj.weight": "model-00029-of-00101.safetensors", + "model.layers.27.mlp.experts.92.up_proj.weight": "model-00029-of-00101.safetensors", + "model.layers.27.mlp.experts.92.down_proj.weight": "model-00029-of-00101.safetensors", + "model.layers.27.mlp.experts.93.gate_proj.weight": "model-00029-of-00101.safetensors", + "model.layers.27.mlp.experts.93.up_proj.weight": "model-00029-of-00101.safetensors", + "model.layers.27.mlp.experts.93.down_proj.weight": "model-00029-of-00101.safetensors", + "model.layers.27.mlp.experts.94.gate_proj.weight": "model-00029-of-00101.safetensors", + "model.layers.27.mlp.experts.94.up_proj.weight": "model-00029-of-00101.safetensors", + "model.layers.27.mlp.experts.94.down_proj.weight": "model-00029-of-00101.safetensors", + "model.layers.27.mlp.experts.95.gate_proj.weight": "model-00029-of-00101.safetensors", + "model.layers.27.mlp.experts.95.up_proj.weight": "model-00029-of-00101.safetensors", + "model.layers.27.mlp.experts.95.down_proj.weight": "model-00029-of-00101.safetensors", + "model.layers.27.mlp.experts.96.gate_proj.weight": "model-00029-of-00101.safetensors", + "model.layers.27.mlp.experts.96.up_proj.weight": "model-00029-of-00101.safetensors", + "model.layers.27.mlp.experts.96.down_proj.weight": "model-00029-of-00101.safetensors", + "model.layers.27.mlp.experts.97.gate_proj.weight": "model-00029-of-00101.safetensors", + "model.layers.27.mlp.experts.97.up_proj.weight": "model-00029-of-00101.safetensors", + "model.layers.27.mlp.experts.97.down_proj.weight": "model-00029-of-00101.safetensors", + "model.layers.27.mlp.experts.98.gate_proj.weight": "model-00029-of-00101.safetensors", + "model.layers.27.mlp.experts.98.up_proj.weight": "model-00029-of-00101.safetensors", + "model.layers.27.mlp.experts.98.down_proj.weight": "model-00029-of-00101.safetensors", + "model.layers.27.mlp.experts.99.gate_proj.weight": "model-00029-of-00101.safetensors", + "model.layers.27.mlp.experts.99.up_proj.weight": "model-00029-of-00101.safetensors", + "model.layers.27.mlp.experts.99.down_proj.weight": "model-00029-of-00101.safetensors", + "model.layers.27.mlp.experts.100.gate_proj.weight": "model-00029-of-00101.safetensors", + "model.layers.27.mlp.experts.100.up_proj.weight": "model-00029-of-00101.safetensors", + "model.layers.27.mlp.experts.100.down_proj.weight": "model-00029-of-00101.safetensors", + "model.layers.27.mlp.experts.101.gate_proj.weight": "model-00029-of-00101.safetensors", + "model.layers.27.mlp.experts.101.up_proj.weight": "model-00029-of-00101.safetensors", + "model.layers.27.mlp.experts.101.down_proj.weight": "model-00029-of-00101.safetensors", + "model.layers.27.mlp.experts.102.gate_proj.weight": "model-00029-of-00101.safetensors", + "model.layers.27.mlp.experts.102.up_proj.weight": "model-00029-of-00101.safetensors", + "model.layers.27.mlp.experts.102.down_proj.weight": "model-00029-of-00101.safetensors", + "model.layers.27.mlp.experts.103.gate_proj.weight": "model-00029-of-00101.safetensors", + "model.layers.27.mlp.experts.103.up_proj.weight": "model-00029-of-00101.safetensors", + "model.layers.27.mlp.experts.103.down_proj.weight": "model-00029-of-00101.safetensors", + "model.layers.27.mlp.experts.104.gate_proj.weight": "model-00029-of-00101.safetensors", + "model.layers.27.mlp.experts.104.up_proj.weight": "model-00029-of-00101.safetensors", + "model.layers.27.mlp.experts.104.down_proj.weight": "model-00029-of-00101.safetensors", + "model.layers.27.mlp.experts.105.gate_proj.weight": "model-00029-of-00101.safetensors", + "model.layers.27.mlp.experts.105.up_proj.weight": "model-00029-of-00101.safetensors", + "model.layers.27.mlp.experts.105.down_proj.weight": "model-00029-of-00101.safetensors", + "model.layers.27.mlp.experts.106.gate_proj.weight": "model-00029-of-00101.safetensors", + "model.layers.27.mlp.experts.106.up_proj.weight": "model-00029-of-00101.safetensors", + "model.layers.27.mlp.experts.106.down_proj.weight": "model-00029-of-00101.safetensors", + "model.layers.27.mlp.experts.107.gate_proj.weight": "model-00029-of-00101.safetensors", + "model.layers.27.mlp.experts.107.up_proj.weight": "model-00029-of-00101.safetensors", + "model.layers.27.mlp.experts.107.down_proj.weight": "model-00029-of-00101.safetensors", + "model.layers.27.mlp.experts.108.gate_proj.weight": "model-00029-of-00101.safetensors", + "model.layers.27.mlp.experts.108.up_proj.weight": "model-00029-of-00101.safetensors", + "model.layers.27.mlp.experts.108.down_proj.weight": "model-00029-of-00101.safetensors", + "model.layers.27.mlp.experts.109.gate_proj.weight": "model-00029-of-00101.safetensors", + "model.layers.27.mlp.experts.109.up_proj.weight": "model-00029-of-00101.safetensors", + "model.layers.27.mlp.experts.109.down_proj.weight": "model-00029-of-00101.safetensors", + "model.layers.27.mlp.experts.110.gate_proj.weight": "model-00029-of-00101.safetensors", + "model.layers.27.mlp.experts.110.up_proj.weight": "model-00029-of-00101.safetensors", + "model.layers.27.mlp.experts.110.down_proj.weight": "model-00029-of-00101.safetensors", + "model.layers.27.mlp.experts.111.gate_proj.weight": "model-00029-of-00101.safetensors", + "model.layers.27.mlp.experts.111.up_proj.weight": "model-00029-of-00101.safetensors", + "model.layers.27.mlp.experts.111.down_proj.weight": "model-00029-of-00101.safetensors", + "model.layers.27.mlp.experts.112.gate_proj.weight": "model-00029-of-00101.safetensors", + "model.layers.27.mlp.experts.112.up_proj.weight": "model-00029-of-00101.safetensors", + "model.layers.27.mlp.experts.112.down_proj.weight": "model-00029-of-00101.safetensors", + "model.layers.27.mlp.experts.113.gate_proj.weight": "model-00029-of-00101.safetensors", + "model.layers.27.mlp.experts.113.up_proj.weight": "model-00029-of-00101.safetensors", + "model.layers.27.mlp.experts.113.down_proj.weight": "model-00029-of-00101.safetensors", + "model.layers.27.mlp.experts.114.gate_proj.weight": "model-00029-of-00101.safetensors", + "model.layers.27.mlp.experts.114.up_proj.weight": "model-00029-of-00101.safetensors", + "model.layers.27.mlp.experts.114.down_proj.weight": "model-00029-of-00101.safetensors", + "model.layers.27.mlp.experts.115.gate_proj.weight": "model-00029-of-00101.safetensors", + "model.layers.27.mlp.experts.115.up_proj.weight": "model-00029-of-00101.safetensors", + "model.layers.27.mlp.experts.115.down_proj.weight": "model-00029-of-00101.safetensors", + "model.layers.27.mlp.experts.116.gate_proj.weight": "model-00029-of-00101.safetensors", + "model.layers.27.mlp.experts.116.up_proj.weight": "model-00029-of-00101.safetensors", + "model.layers.27.mlp.experts.116.down_proj.weight": "model-00029-of-00101.safetensors", + "model.layers.27.mlp.experts.117.gate_proj.weight": "model-00029-of-00101.safetensors", + "model.layers.27.mlp.experts.117.up_proj.weight": "model-00029-of-00101.safetensors", + "model.layers.27.mlp.experts.117.down_proj.weight": "model-00029-of-00101.safetensors", + "model.layers.27.mlp.experts.118.gate_proj.weight": "model-00029-of-00101.safetensors", + "model.layers.27.mlp.experts.118.up_proj.weight": "model-00029-of-00101.safetensors", + "model.layers.27.mlp.experts.118.down_proj.weight": "model-00029-of-00101.safetensors", + "model.layers.27.mlp.experts.119.gate_proj.weight": "model-00029-of-00101.safetensors", + "model.layers.27.mlp.experts.119.up_proj.weight": "model-00029-of-00101.safetensors", + "model.layers.27.mlp.experts.119.down_proj.weight": "model-00029-of-00101.safetensors", + "model.layers.27.mlp.gate.weight": "model-00029-of-00101.safetensors", + "model.layers.27.mlp.gate.e_score_correction_bias": "model-00029-of-00101.safetensors", + "model.layers.27.mlp.shared_experts.gate_proj.weight": "model-00029-of-00101.safetensors", + "model.layers.27.mlp.shared_experts.up_proj.weight": "model-00029-of-00101.safetensors", + "model.layers.27.mlp.shared_experts.down_proj.weight": "model-00029-of-00101.safetensors", + "model.layers.27.input_layernorm.weight": "model-00029-of-00101.safetensors", + "model.layers.27.post_attention_layernorm.weight": "model-00029-of-00101.safetensors", + "model.layers.28.self_attn.q_proj.weight": "model-00029-of-00101.safetensors", + "model.layers.28.self_attn.q_proj.bias": "model-00029-of-00101.safetensors", + "model.layers.28.self_attn.k_proj.weight": "model-00029-of-00101.safetensors", + "model.layers.28.self_attn.k_proj.bias": "model-00029-of-00101.safetensors", + "model.layers.28.self_attn.v_proj.weight": "model-00029-of-00101.safetensors", + "model.layers.28.self_attn.v_proj.bias": "model-00029-of-00101.safetensors", + "model.layers.28.self_attn.o_proj.weight": "model-00029-of-00101.safetensors", + "model.layers.28.self_attn.q_norm.weight": "model-00029-of-00101.safetensors", + "model.layers.28.self_attn.k_norm.weight": "model-00029-of-00101.safetensors", + "model.layers.28.mlp.experts.0.gate_proj.weight": "model-00029-of-00101.safetensors", + "model.layers.28.mlp.experts.0.up_proj.weight": "model-00029-of-00101.safetensors", + "model.layers.28.mlp.experts.0.down_proj.weight": "model-00029-of-00101.safetensors", + "model.layers.28.mlp.experts.1.gate_proj.weight": "model-00029-of-00101.safetensors", + "model.layers.28.mlp.experts.1.up_proj.weight": "model-00029-of-00101.safetensors", + "model.layers.28.mlp.experts.1.down_proj.weight": "model-00029-of-00101.safetensors", + "model.layers.28.mlp.experts.2.gate_proj.weight": "model-00029-of-00101.safetensors", + "model.layers.28.mlp.experts.2.up_proj.weight": "model-00029-of-00101.safetensors", + "model.layers.28.mlp.experts.2.down_proj.weight": "model-00029-of-00101.safetensors", + "model.layers.28.mlp.experts.3.gate_proj.weight": "model-00029-of-00101.safetensors", + "model.layers.28.mlp.experts.3.up_proj.weight": "model-00029-of-00101.safetensors", + "model.layers.28.mlp.experts.3.down_proj.weight": "model-00029-of-00101.safetensors", + "model.layers.28.mlp.experts.4.gate_proj.weight": "model-00029-of-00101.safetensors", + "model.layers.28.mlp.experts.4.up_proj.weight": "model-00029-of-00101.safetensors", + "model.layers.28.mlp.experts.4.down_proj.weight": "model-00029-of-00101.safetensors", + "model.layers.28.mlp.experts.5.gate_proj.weight": "model-00029-of-00101.safetensors", + "model.layers.28.mlp.experts.5.up_proj.weight": "model-00029-of-00101.safetensors", + "model.layers.28.mlp.experts.5.down_proj.weight": "model-00029-of-00101.safetensors", + "model.layers.28.mlp.experts.6.gate_proj.weight": "model-00029-of-00101.safetensors", + "model.layers.28.mlp.experts.6.up_proj.weight": "model-00029-of-00101.safetensors", + "model.layers.28.mlp.experts.6.down_proj.weight": "model-00029-of-00101.safetensors", + "model.layers.28.mlp.experts.7.gate_proj.weight": "model-00029-of-00101.safetensors", + "model.layers.28.mlp.experts.7.up_proj.weight": "model-00029-of-00101.safetensors", + "model.layers.28.mlp.experts.7.down_proj.weight": "model-00029-of-00101.safetensors", + "model.layers.28.mlp.experts.8.gate_proj.weight": "model-00029-of-00101.safetensors", + "model.layers.28.mlp.experts.8.up_proj.weight": "model-00029-of-00101.safetensors", + "model.layers.28.mlp.experts.8.down_proj.weight": "model-00029-of-00101.safetensors", + "model.layers.28.mlp.experts.9.gate_proj.weight": "model-00029-of-00101.safetensors", + "model.layers.28.mlp.experts.9.up_proj.weight": "model-00029-of-00101.safetensors", + "model.layers.28.mlp.experts.9.down_proj.weight": "model-00029-of-00101.safetensors", + "model.layers.28.mlp.experts.10.gate_proj.weight": "model-00029-of-00101.safetensors", + "model.layers.28.mlp.experts.10.up_proj.weight": "model-00029-of-00101.safetensors", + "model.layers.28.mlp.experts.10.down_proj.weight": "model-00029-of-00101.safetensors", + "model.layers.28.mlp.experts.11.gate_proj.weight": "model-00029-of-00101.safetensors", + "model.layers.28.mlp.experts.11.up_proj.weight": "model-00029-of-00101.safetensors", + "model.layers.28.mlp.experts.11.down_proj.weight": "model-00029-of-00101.safetensors", + "model.layers.28.mlp.experts.12.gate_proj.weight": "model-00029-of-00101.safetensors", + "model.layers.28.mlp.experts.12.up_proj.weight": "model-00029-of-00101.safetensors", + "model.layers.28.mlp.experts.12.down_proj.weight": "model-00029-of-00101.safetensors", + "model.layers.28.mlp.experts.13.gate_proj.weight": "model-00029-of-00101.safetensors", + "model.layers.28.mlp.experts.13.up_proj.weight": "model-00029-of-00101.safetensors", + "model.layers.28.mlp.experts.13.down_proj.weight": "model-00029-of-00101.safetensors", + "model.layers.28.mlp.experts.14.gate_proj.weight": "model-00029-of-00101.safetensors", + "model.layers.28.mlp.experts.14.up_proj.weight": "model-00029-of-00101.safetensors", + "model.layers.28.mlp.experts.14.down_proj.weight": "model-00029-of-00101.safetensors", + "model.layers.28.mlp.experts.15.gate_proj.weight": "model-00029-of-00101.safetensors", + "model.layers.28.mlp.experts.15.up_proj.weight": "model-00029-of-00101.safetensors", + "model.layers.28.mlp.experts.15.down_proj.weight": "model-00029-of-00101.safetensors", + "model.layers.28.mlp.experts.16.gate_proj.weight": "model-00029-of-00101.safetensors", + "model.layers.28.mlp.experts.16.up_proj.weight": "model-00029-of-00101.safetensors", + "model.layers.28.mlp.experts.16.down_proj.weight": "model-00029-of-00101.safetensors", + "model.layers.28.mlp.experts.17.gate_proj.weight": "model-00029-of-00101.safetensors", + "model.layers.28.mlp.experts.17.up_proj.weight": "model-00029-of-00101.safetensors", + "model.layers.28.mlp.experts.17.down_proj.weight": "model-00029-of-00101.safetensors", + "model.layers.28.mlp.experts.18.gate_proj.weight": "model-00029-of-00101.safetensors", + "model.layers.28.mlp.experts.18.up_proj.weight": "model-00029-of-00101.safetensors", + "model.layers.28.mlp.experts.18.down_proj.weight": "model-00029-of-00101.safetensors", + "model.layers.28.mlp.experts.19.gate_proj.weight": "model-00029-of-00101.safetensors", + "model.layers.28.mlp.experts.19.up_proj.weight": "model-00029-of-00101.safetensors", + "model.layers.28.mlp.experts.19.down_proj.weight": "model-00029-of-00101.safetensors", + "model.layers.28.mlp.experts.20.gate_proj.weight": "model-00029-of-00101.safetensors", + "model.layers.28.mlp.experts.20.up_proj.weight": "model-00029-of-00101.safetensors", + "model.layers.28.mlp.experts.20.down_proj.weight": "model-00029-of-00101.safetensors", + "model.layers.28.mlp.experts.21.gate_proj.weight": "model-00029-of-00101.safetensors", + "model.layers.28.mlp.experts.21.up_proj.weight": "model-00029-of-00101.safetensors", + "model.layers.28.mlp.experts.21.down_proj.weight": "model-00029-of-00101.safetensors", + "model.layers.28.mlp.experts.22.gate_proj.weight": "model-00029-of-00101.safetensors", + "model.layers.28.mlp.experts.22.up_proj.weight": "model-00029-of-00101.safetensors", + "model.layers.28.mlp.experts.22.down_proj.weight": "model-00029-of-00101.safetensors", + "model.layers.28.mlp.experts.23.gate_proj.weight": "model-00029-of-00101.safetensors", + "model.layers.28.mlp.experts.23.up_proj.weight": "model-00029-of-00101.safetensors", + "model.layers.28.mlp.experts.23.down_proj.weight": "model-00029-of-00101.safetensors", + "model.layers.28.mlp.experts.24.gate_proj.weight": "model-00029-of-00101.safetensors", + "model.layers.28.mlp.experts.24.up_proj.weight": "model-00029-of-00101.safetensors", + "model.layers.28.mlp.experts.24.down_proj.weight": "model-00029-of-00101.safetensors", + "model.layers.28.mlp.experts.25.gate_proj.weight": "model-00029-of-00101.safetensors", + "model.layers.28.mlp.experts.25.up_proj.weight": "model-00029-of-00101.safetensors", + "model.layers.28.mlp.experts.25.down_proj.weight": "model-00029-of-00101.safetensors", + "model.layers.28.mlp.experts.26.gate_proj.weight": "model-00029-of-00101.safetensors", + "model.layers.28.mlp.experts.26.up_proj.weight": "model-00029-of-00101.safetensors", + "model.layers.28.mlp.experts.26.down_proj.weight": "model-00029-of-00101.safetensors", + "model.layers.28.mlp.experts.27.gate_proj.weight": "model-00029-of-00101.safetensors", + "model.layers.28.mlp.experts.27.up_proj.weight": "model-00029-of-00101.safetensors", + "model.layers.28.mlp.experts.27.down_proj.weight": "model-00029-of-00101.safetensors", + "model.layers.28.mlp.experts.28.gate_proj.weight": "model-00029-of-00101.safetensors", + "model.layers.28.mlp.experts.28.up_proj.weight": "model-00029-of-00101.safetensors", + "model.layers.28.mlp.experts.28.down_proj.weight": "model-00029-of-00101.safetensors", + "model.layers.28.mlp.experts.29.gate_proj.weight": "model-00029-of-00101.safetensors", + "model.layers.28.mlp.experts.29.up_proj.weight": "model-00029-of-00101.safetensors", + "model.layers.28.mlp.experts.29.down_proj.weight": "model-00029-of-00101.safetensors", + "model.layers.28.mlp.experts.30.gate_proj.weight": "model-00029-of-00101.safetensors", + "model.layers.28.mlp.experts.30.up_proj.weight": "model-00029-of-00101.safetensors", + "model.layers.28.mlp.experts.30.down_proj.weight": "model-00029-of-00101.safetensors", + "model.layers.28.mlp.experts.31.gate_proj.weight": "model-00029-of-00101.safetensors", + "model.layers.28.mlp.experts.31.up_proj.weight": "model-00029-of-00101.safetensors", + "model.layers.28.mlp.experts.31.down_proj.weight": "model-00029-of-00101.safetensors", + "model.layers.28.mlp.experts.32.gate_proj.weight": "model-00029-of-00101.safetensors", + "model.layers.28.mlp.experts.32.up_proj.weight": "model-00029-of-00101.safetensors", + "model.layers.28.mlp.experts.32.down_proj.weight": "model-00029-of-00101.safetensors", + "model.layers.28.mlp.experts.33.gate_proj.weight": "model-00029-of-00101.safetensors", + "model.layers.28.mlp.experts.33.up_proj.weight": "model-00029-of-00101.safetensors", + "model.layers.28.mlp.experts.33.down_proj.weight": "model-00029-of-00101.safetensors", + "model.layers.28.mlp.experts.34.gate_proj.weight": "model-00029-of-00101.safetensors", + "model.layers.28.mlp.experts.34.up_proj.weight": "model-00029-of-00101.safetensors", + "model.layers.28.mlp.experts.34.down_proj.weight": "model-00029-of-00101.safetensors", + "model.layers.28.mlp.experts.35.gate_proj.weight": "model-00029-of-00101.safetensors", + "model.layers.28.mlp.experts.35.up_proj.weight": "model-00029-of-00101.safetensors", + "model.layers.28.mlp.experts.35.down_proj.weight": "model-00029-of-00101.safetensors", + "model.layers.28.mlp.experts.36.gate_proj.weight": "model-00029-of-00101.safetensors", + "model.layers.28.mlp.experts.36.up_proj.weight": "model-00029-of-00101.safetensors", + "model.layers.28.mlp.experts.36.down_proj.weight": "model-00029-of-00101.safetensors", + "model.layers.28.mlp.experts.37.gate_proj.weight": "model-00029-of-00101.safetensors", + "model.layers.28.mlp.experts.37.up_proj.weight": "model-00029-of-00101.safetensors", + "model.layers.28.mlp.experts.37.down_proj.weight": "model-00029-of-00101.safetensors", + "model.layers.28.mlp.experts.38.gate_proj.weight": "model-00029-of-00101.safetensors", + "model.layers.28.mlp.experts.38.up_proj.weight": "model-00029-of-00101.safetensors", + "model.layers.28.mlp.experts.38.down_proj.weight": "model-00029-of-00101.safetensors", + "model.layers.28.mlp.experts.39.gate_proj.weight": "model-00029-of-00101.safetensors", + "model.layers.28.mlp.experts.39.up_proj.weight": "model-00029-of-00101.safetensors", + "model.layers.28.mlp.experts.39.down_proj.weight": "model-00029-of-00101.safetensors", + "model.layers.28.mlp.experts.40.gate_proj.weight": "model-00029-of-00101.safetensors", + "model.layers.28.mlp.experts.40.up_proj.weight": "model-00029-of-00101.safetensors", + "model.layers.28.mlp.experts.40.down_proj.weight": "model-00029-of-00101.safetensors", + "model.layers.28.mlp.experts.41.gate_proj.weight": "model-00030-of-00101.safetensors", + "model.layers.28.mlp.experts.41.up_proj.weight": "model-00030-of-00101.safetensors", + "model.layers.28.mlp.experts.41.down_proj.weight": "model-00030-of-00101.safetensors", + "model.layers.28.mlp.experts.42.gate_proj.weight": "model-00030-of-00101.safetensors", + "model.layers.28.mlp.experts.42.up_proj.weight": "model-00030-of-00101.safetensors", + "model.layers.28.mlp.experts.42.down_proj.weight": "model-00030-of-00101.safetensors", + "model.layers.28.mlp.experts.43.gate_proj.weight": "model-00030-of-00101.safetensors", + "model.layers.28.mlp.experts.43.up_proj.weight": "model-00030-of-00101.safetensors", + "model.layers.28.mlp.experts.43.down_proj.weight": "model-00030-of-00101.safetensors", + "model.layers.28.mlp.experts.44.gate_proj.weight": "model-00030-of-00101.safetensors", + "model.layers.28.mlp.experts.44.up_proj.weight": "model-00030-of-00101.safetensors", + "model.layers.28.mlp.experts.44.down_proj.weight": "model-00030-of-00101.safetensors", + "model.layers.28.mlp.experts.45.gate_proj.weight": "model-00030-of-00101.safetensors", + "model.layers.28.mlp.experts.45.up_proj.weight": "model-00030-of-00101.safetensors", + "model.layers.28.mlp.experts.45.down_proj.weight": "model-00030-of-00101.safetensors", + "model.layers.28.mlp.experts.46.gate_proj.weight": "model-00030-of-00101.safetensors", + "model.layers.28.mlp.experts.46.up_proj.weight": "model-00030-of-00101.safetensors", + "model.layers.28.mlp.experts.46.down_proj.weight": "model-00030-of-00101.safetensors", + "model.layers.28.mlp.experts.47.gate_proj.weight": "model-00030-of-00101.safetensors", + "model.layers.28.mlp.experts.47.up_proj.weight": "model-00030-of-00101.safetensors", + "model.layers.28.mlp.experts.47.down_proj.weight": "model-00030-of-00101.safetensors", + "model.layers.28.mlp.experts.48.gate_proj.weight": "model-00030-of-00101.safetensors", + "model.layers.28.mlp.experts.48.up_proj.weight": "model-00030-of-00101.safetensors", + "model.layers.28.mlp.experts.48.down_proj.weight": "model-00030-of-00101.safetensors", + "model.layers.28.mlp.experts.49.gate_proj.weight": "model-00030-of-00101.safetensors", + "model.layers.28.mlp.experts.49.up_proj.weight": "model-00030-of-00101.safetensors", + "model.layers.28.mlp.experts.49.down_proj.weight": "model-00030-of-00101.safetensors", + "model.layers.28.mlp.experts.50.gate_proj.weight": "model-00030-of-00101.safetensors", + "model.layers.28.mlp.experts.50.up_proj.weight": "model-00030-of-00101.safetensors", + "model.layers.28.mlp.experts.50.down_proj.weight": "model-00030-of-00101.safetensors", + "model.layers.28.mlp.experts.51.gate_proj.weight": "model-00030-of-00101.safetensors", + "model.layers.28.mlp.experts.51.up_proj.weight": "model-00030-of-00101.safetensors", + "model.layers.28.mlp.experts.51.down_proj.weight": "model-00030-of-00101.safetensors", + "model.layers.28.mlp.experts.52.gate_proj.weight": "model-00030-of-00101.safetensors", + "model.layers.28.mlp.experts.52.up_proj.weight": "model-00030-of-00101.safetensors", + "model.layers.28.mlp.experts.52.down_proj.weight": "model-00030-of-00101.safetensors", + "model.layers.28.mlp.experts.53.gate_proj.weight": "model-00030-of-00101.safetensors", + "model.layers.28.mlp.experts.53.up_proj.weight": "model-00030-of-00101.safetensors", + "model.layers.28.mlp.experts.53.down_proj.weight": "model-00030-of-00101.safetensors", + "model.layers.28.mlp.experts.54.gate_proj.weight": "model-00030-of-00101.safetensors", + "model.layers.28.mlp.experts.54.up_proj.weight": "model-00030-of-00101.safetensors", + "model.layers.28.mlp.experts.54.down_proj.weight": "model-00030-of-00101.safetensors", + "model.layers.28.mlp.experts.55.gate_proj.weight": "model-00030-of-00101.safetensors", + "model.layers.28.mlp.experts.55.up_proj.weight": "model-00030-of-00101.safetensors", + "model.layers.28.mlp.experts.55.down_proj.weight": "model-00030-of-00101.safetensors", + "model.layers.28.mlp.experts.56.gate_proj.weight": "model-00030-of-00101.safetensors", + "model.layers.28.mlp.experts.56.up_proj.weight": "model-00030-of-00101.safetensors", + "model.layers.28.mlp.experts.56.down_proj.weight": "model-00030-of-00101.safetensors", + "model.layers.28.mlp.experts.57.gate_proj.weight": "model-00030-of-00101.safetensors", + "model.layers.28.mlp.experts.57.up_proj.weight": "model-00030-of-00101.safetensors", + "model.layers.28.mlp.experts.57.down_proj.weight": "model-00030-of-00101.safetensors", + "model.layers.28.mlp.experts.58.gate_proj.weight": "model-00030-of-00101.safetensors", + "model.layers.28.mlp.experts.58.up_proj.weight": "model-00030-of-00101.safetensors", + "model.layers.28.mlp.experts.58.down_proj.weight": "model-00030-of-00101.safetensors", + "model.layers.28.mlp.experts.59.gate_proj.weight": "model-00030-of-00101.safetensors", + "model.layers.28.mlp.experts.59.up_proj.weight": "model-00030-of-00101.safetensors", + "model.layers.28.mlp.experts.59.down_proj.weight": "model-00030-of-00101.safetensors", + "model.layers.28.mlp.experts.60.gate_proj.weight": "model-00030-of-00101.safetensors", + "model.layers.28.mlp.experts.60.up_proj.weight": "model-00030-of-00101.safetensors", + "model.layers.28.mlp.experts.60.down_proj.weight": "model-00030-of-00101.safetensors", + "model.layers.28.mlp.experts.61.gate_proj.weight": "model-00030-of-00101.safetensors", + "model.layers.28.mlp.experts.61.up_proj.weight": "model-00030-of-00101.safetensors", + "model.layers.28.mlp.experts.61.down_proj.weight": "model-00030-of-00101.safetensors", + "model.layers.28.mlp.experts.62.gate_proj.weight": "model-00030-of-00101.safetensors", + "model.layers.28.mlp.experts.62.up_proj.weight": "model-00030-of-00101.safetensors", + "model.layers.28.mlp.experts.62.down_proj.weight": "model-00030-of-00101.safetensors", + "model.layers.28.mlp.experts.63.gate_proj.weight": "model-00030-of-00101.safetensors", + "model.layers.28.mlp.experts.63.up_proj.weight": "model-00030-of-00101.safetensors", + "model.layers.28.mlp.experts.63.down_proj.weight": "model-00030-of-00101.safetensors", + "model.layers.28.mlp.experts.64.gate_proj.weight": "model-00030-of-00101.safetensors", + "model.layers.28.mlp.experts.64.up_proj.weight": "model-00030-of-00101.safetensors", + "model.layers.28.mlp.experts.64.down_proj.weight": "model-00030-of-00101.safetensors", + "model.layers.28.mlp.experts.65.gate_proj.weight": "model-00030-of-00101.safetensors", + "model.layers.28.mlp.experts.65.up_proj.weight": "model-00030-of-00101.safetensors", + "model.layers.28.mlp.experts.65.down_proj.weight": "model-00030-of-00101.safetensors", + "model.layers.28.mlp.experts.66.gate_proj.weight": "model-00030-of-00101.safetensors", + "model.layers.28.mlp.experts.66.up_proj.weight": "model-00030-of-00101.safetensors", + "model.layers.28.mlp.experts.66.down_proj.weight": "model-00030-of-00101.safetensors", + "model.layers.28.mlp.experts.67.gate_proj.weight": "model-00030-of-00101.safetensors", + "model.layers.28.mlp.experts.67.up_proj.weight": "model-00030-of-00101.safetensors", + "model.layers.28.mlp.experts.67.down_proj.weight": "model-00030-of-00101.safetensors", + "model.layers.28.mlp.experts.68.gate_proj.weight": "model-00030-of-00101.safetensors", + "model.layers.28.mlp.experts.68.up_proj.weight": "model-00030-of-00101.safetensors", + "model.layers.28.mlp.experts.68.down_proj.weight": "model-00030-of-00101.safetensors", + "model.layers.28.mlp.experts.69.gate_proj.weight": "model-00030-of-00101.safetensors", + "model.layers.28.mlp.experts.69.up_proj.weight": "model-00030-of-00101.safetensors", + "model.layers.28.mlp.experts.69.down_proj.weight": "model-00030-of-00101.safetensors", + "model.layers.28.mlp.experts.70.gate_proj.weight": "model-00030-of-00101.safetensors", + "model.layers.28.mlp.experts.70.up_proj.weight": "model-00030-of-00101.safetensors", + "model.layers.28.mlp.experts.70.down_proj.weight": "model-00030-of-00101.safetensors", + "model.layers.28.mlp.experts.71.gate_proj.weight": "model-00030-of-00101.safetensors", + "model.layers.28.mlp.experts.71.up_proj.weight": "model-00030-of-00101.safetensors", + "model.layers.28.mlp.experts.71.down_proj.weight": "model-00030-of-00101.safetensors", + "model.layers.28.mlp.experts.72.gate_proj.weight": "model-00030-of-00101.safetensors", + "model.layers.28.mlp.experts.72.up_proj.weight": "model-00030-of-00101.safetensors", + "model.layers.28.mlp.experts.72.down_proj.weight": "model-00030-of-00101.safetensors", + "model.layers.28.mlp.experts.73.gate_proj.weight": "model-00030-of-00101.safetensors", + "model.layers.28.mlp.experts.73.up_proj.weight": "model-00030-of-00101.safetensors", + "model.layers.28.mlp.experts.73.down_proj.weight": "model-00030-of-00101.safetensors", + "model.layers.28.mlp.experts.74.gate_proj.weight": "model-00030-of-00101.safetensors", + "model.layers.28.mlp.experts.74.up_proj.weight": "model-00030-of-00101.safetensors", + "model.layers.28.mlp.experts.74.down_proj.weight": "model-00030-of-00101.safetensors", + "model.layers.28.mlp.experts.75.gate_proj.weight": "model-00030-of-00101.safetensors", + "model.layers.28.mlp.experts.75.up_proj.weight": "model-00030-of-00101.safetensors", + "model.layers.28.mlp.experts.75.down_proj.weight": "model-00030-of-00101.safetensors", + "model.layers.28.mlp.experts.76.gate_proj.weight": "model-00030-of-00101.safetensors", + "model.layers.28.mlp.experts.76.up_proj.weight": "model-00030-of-00101.safetensors", + "model.layers.28.mlp.experts.76.down_proj.weight": "model-00030-of-00101.safetensors", + "model.layers.28.mlp.experts.77.gate_proj.weight": "model-00030-of-00101.safetensors", + "model.layers.28.mlp.experts.77.up_proj.weight": "model-00030-of-00101.safetensors", + "model.layers.28.mlp.experts.77.down_proj.weight": "model-00030-of-00101.safetensors", + "model.layers.28.mlp.experts.78.gate_proj.weight": "model-00030-of-00101.safetensors", + "model.layers.28.mlp.experts.78.up_proj.weight": "model-00030-of-00101.safetensors", + "model.layers.28.mlp.experts.78.down_proj.weight": "model-00030-of-00101.safetensors", + "model.layers.28.mlp.experts.79.gate_proj.weight": "model-00030-of-00101.safetensors", + "model.layers.28.mlp.experts.79.up_proj.weight": "model-00030-of-00101.safetensors", + "model.layers.28.mlp.experts.79.down_proj.weight": "model-00030-of-00101.safetensors", + "model.layers.28.mlp.experts.80.gate_proj.weight": "model-00030-of-00101.safetensors", + "model.layers.28.mlp.experts.80.up_proj.weight": "model-00030-of-00101.safetensors", + "model.layers.28.mlp.experts.80.down_proj.weight": "model-00030-of-00101.safetensors", + "model.layers.28.mlp.experts.81.gate_proj.weight": "model-00030-of-00101.safetensors", + "model.layers.28.mlp.experts.81.up_proj.weight": "model-00030-of-00101.safetensors", + "model.layers.28.mlp.experts.81.down_proj.weight": "model-00030-of-00101.safetensors", + "model.layers.28.mlp.experts.82.gate_proj.weight": "model-00030-of-00101.safetensors", + "model.layers.28.mlp.experts.82.up_proj.weight": "model-00030-of-00101.safetensors", + "model.layers.28.mlp.experts.82.down_proj.weight": "model-00030-of-00101.safetensors", + "model.layers.28.mlp.experts.83.gate_proj.weight": "model-00030-of-00101.safetensors", + "model.layers.28.mlp.experts.83.up_proj.weight": "model-00030-of-00101.safetensors", + "model.layers.28.mlp.experts.83.down_proj.weight": "model-00030-of-00101.safetensors", + "model.layers.28.mlp.experts.84.gate_proj.weight": "model-00030-of-00101.safetensors", + "model.layers.28.mlp.experts.84.up_proj.weight": "model-00030-of-00101.safetensors", + "model.layers.28.mlp.experts.84.down_proj.weight": "model-00030-of-00101.safetensors", + "model.layers.28.mlp.experts.85.gate_proj.weight": "model-00030-of-00101.safetensors", + "model.layers.28.mlp.experts.85.up_proj.weight": "model-00030-of-00101.safetensors", + "model.layers.28.mlp.experts.85.down_proj.weight": "model-00030-of-00101.safetensors", + "model.layers.28.mlp.experts.86.gate_proj.weight": "model-00030-of-00101.safetensors", + "model.layers.28.mlp.experts.86.up_proj.weight": "model-00030-of-00101.safetensors", + "model.layers.28.mlp.experts.86.down_proj.weight": "model-00030-of-00101.safetensors", + "model.layers.28.mlp.experts.87.gate_proj.weight": "model-00030-of-00101.safetensors", + "model.layers.28.mlp.experts.87.up_proj.weight": "model-00030-of-00101.safetensors", + "model.layers.28.mlp.experts.87.down_proj.weight": "model-00030-of-00101.safetensors", + "model.layers.28.mlp.experts.88.gate_proj.weight": "model-00030-of-00101.safetensors", + "model.layers.28.mlp.experts.88.up_proj.weight": "model-00030-of-00101.safetensors", + "model.layers.28.mlp.experts.88.down_proj.weight": "model-00030-of-00101.safetensors", + "model.layers.28.mlp.experts.89.gate_proj.weight": "model-00030-of-00101.safetensors", + "model.layers.28.mlp.experts.89.up_proj.weight": "model-00030-of-00101.safetensors", + "model.layers.28.mlp.experts.89.down_proj.weight": "model-00030-of-00101.safetensors", + "model.layers.28.mlp.experts.90.gate_proj.weight": "model-00030-of-00101.safetensors", + "model.layers.28.mlp.experts.90.up_proj.weight": "model-00030-of-00101.safetensors", + "model.layers.28.mlp.experts.90.down_proj.weight": "model-00030-of-00101.safetensors", + "model.layers.28.mlp.experts.91.gate_proj.weight": "model-00030-of-00101.safetensors", + "model.layers.28.mlp.experts.91.up_proj.weight": "model-00030-of-00101.safetensors", + "model.layers.28.mlp.experts.91.down_proj.weight": "model-00030-of-00101.safetensors", + "model.layers.28.mlp.experts.92.gate_proj.weight": "model-00030-of-00101.safetensors", + "model.layers.28.mlp.experts.92.up_proj.weight": "model-00030-of-00101.safetensors", + "model.layers.28.mlp.experts.92.down_proj.weight": "model-00030-of-00101.safetensors", + "model.layers.28.mlp.experts.93.gate_proj.weight": "model-00030-of-00101.safetensors", + "model.layers.28.mlp.experts.93.up_proj.weight": "model-00030-of-00101.safetensors", + "model.layers.28.mlp.experts.93.down_proj.weight": "model-00030-of-00101.safetensors", + "model.layers.28.mlp.experts.94.gate_proj.weight": "model-00030-of-00101.safetensors", + "model.layers.28.mlp.experts.94.up_proj.weight": "model-00030-of-00101.safetensors", + "model.layers.28.mlp.experts.94.down_proj.weight": "model-00030-of-00101.safetensors", + "model.layers.28.mlp.experts.95.gate_proj.weight": "model-00030-of-00101.safetensors", + "model.layers.28.mlp.experts.95.up_proj.weight": "model-00030-of-00101.safetensors", + "model.layers.28.mlp.experts.95.down_proj.weight": "model-00030-of-00101.safetensors", + "model.layers.28.mlp.experts.96.gate_proj.weight": "model-00030-of-00101.safetensors", + "model.layers.28.mlp.experts.96.up_proj.weight": "model-00030-of-00101.safetensors", + "model.layers.28.mlp.experts.96.down_proj.weight": "model-00030-of-00101.safetensors", + "model.layers.28.mlp.experts.97.gate_proj.weight": "model-00030-of-00101.safetensors", + "model.layers.28.mlp.experts.97.up_proj.weight": "model-00030-of-00101.safetensors", + "model.layers.28.mlp.experts.97.down_proj.weight": "model-00030-of-00101.safetensors", + "model.layers.28.mlp.experts.98.gate_proj.weight": "model-00030-of-00101.safetensors", + "model.layers.28.mlp.experts.98.up_proj.weight": "model-00030-of-00101.safetensors", + "model.layers.28.mlp.experts.98.down_proj.weight": "model-00030-of-00101.safetensors", + "model.layers.28.mlp.experts.99.gate_proj.weight": "model-00030-of-00101.safetensors", + "model.layers.28.mlp.experts.99.up_proj.weight": "model-00030-of-00101.safetensors", + "model.layers.28.mlp.experts.99.down_proj.weight": "model-00030-of-00101.safetensors", + "model.layers.28.mlp.experts.100.gate_proj.weight": "model-00030-of-00101.safetensors", + "model.layers.28.mlp.experts.100.up_proj.weight": "model-00030-of-00101.safetensors", + "model.layers.28.mlp.experts.100.down_proj.weight": "model-00030-of-00101.safetensors", + "model.layers.28.mlp.experts.101.gate_proj.weight": "model-00030-of-00101.safetensors", + "model.layers.28.mlp.experts.101.up_proj.weight": "model-00030-of-00101.safetensors", + "model.layers.28.mlp.experts.101.down_proj.weight": "model-00030-of-00101.safetensors", + "model.layers.28.mlp.experts.102.gate_proj.weight": "model-00030-of-00101.safetensors", + "model.layers.28.mlp.experts.102.up_proj.weight": "model-00030-of-00101.safetensors", + "model.layers.28.mlp.experts.102.down_proj.weight": "model-00030-of-00101.safetensors", + "model.layers.28.mlp.experts.103.gate_proj.weight": "model-00030-of-00101.safetensors", + "model.layers.28.mlp.experts.103.up_proj.weight": "model-00030-of-00101.safetensors", + "model.layers.28.mlp.experts.103.down_proj.weight": "model-00030-of-00101.safetensors", + "model.layers.28.mlp.experts.104.gate_proj.weight": "model-00030-of-00101.safetensors", + "model.layers.28.mlp.experts.104.up_proj.weight": "model-00030-of-00101.safetensors", + "model.layers.28.mlp.experts.104.down_proj.weight": "model-00030-of-00101.safetensors", + "model.layers.28.mlp.experts.105.gate_proj.weight": "model-00030-of-00101.safetensors", + "model.layers.28.mlp.experts.105.up_proj.weight": "model-00030-of-00101.safetensors", + "model.layers.28.mlp.experts.105.down_proj.weight": "model-00030-of-00101.safetensors", + "model.layers.28.mlp.experts.106.gate_proj.weight": "model-00030-of-00101.safetensors", + "model.layers.28.mlp.experts.106.up_proj.weight": "model-00030-of-00101.safetensors", + "model.layers.28.mlp.experts.106.down_proj.weight": "model-00030-of-00101.safetensors", + "model.layers.28.mlp.experts.107.gate_proj.weight": "model-00030-of-00101.safetensors", + "model.layers.28.mlp.experts.107.up_proj.weight": "model-00030-of-00101.safetensors", + "model.layers.28.mlp.experts.107.down_proj.weight": "model-00030-of-00101.safetensors", + "model.layers.28.mlp.experts.108.gate_proj.weight": "model-00030-of-00101.safetensors", + "model.layers.28.mlp.experts.108.up_proj.weight": "model-00030-of-00101.safetensors", + "model.layers.28.mlp.experts.108.down_proj.weight": "model-00030-of-00101.safetensors", + "model.layers.28.mlp.experts.109.gate_proj.weight": "model-00030-of-00101.safetensors", + "model.layers.28.mlp.experts.109.up_proj.weight": "model-00030-of-00101.safetensors", + "model.layers.28.mlp.experts.109.down_proj.weight": "model-00030-of-00101.safetensors", + "model.layers.28.mlp.experts.110.gate_proj.weight": "model-00030-of-00101.safetensors", + "model.layers.28.mlp.experts.110.up_proj.weight": "model-00030-of-00101.safetensors", + "model.layers.28.mlp.experts.110.down_proj.weight": "model-00030-of-00101.safetensors", + "model.layers.28.mlp.experts.111.gate_proj.weight": "model-00030-of-00101.safetensors", + "model.layers.28.mlp.experts.111.up_proj.weight": "model-00030-of-00101.safetensors", + "model.layers.28.mlp.experts.111.down_proj.weight": "model-00030-of-00101.safetensors", + "model.layers.28.mlp.experts.112.gate_proj.weight": "model-00030-of-00101.safetensors", + "model.layers.28.mlp.experts.112.up_proj.weight": "model-00030-of-00101.safetensors", + "model.layers.28.mlp.experts.112.down_proj.weight": "model-00030-of-00101.safetensors", + "model.layers.28.mlp.experts.113.gate_proj.weight": "model-00030-of-00101.safetensors", + "model.layers.28.mlp.experts.113.up_proj.weight": "model-00030-of-00101.safetensors", + "model.layers.28.mlp.experts.113.down_proj.weight": "model-00030-of-00101.safetensors", + "model.layers.28.mlp.experts.114.gate_proj.weight": "model-00030-of-00101.safetensors", + "model.layers.28.mlp.experts.114.up_proj.weight": "model-00030-of-00101.safetensors", + "model.layers.28.mlp.experts.114.down_proj.weight": "model-00030-of-00101.safetensors", + "model.layers.28.mlp.experts.115.gate_proj.weight": "model-00030-of-00101.safetensors", + "model.layers.28.mlp.experts.115.up_proj.weight": "model-00030-of-00101.safetensors", + "model.layers.28.mlp.experts.115.down_proj.weight": "model-00030-of-00101.safetensors", + "model.layers.28.mlp.experts.116.gate_proj.weight": "model-00030-of-00101.safetensors", + "model.layers.28.mlp.experts.116.up_proj.weight": "model-00030-of-00101.safetensors", + "model.layers.28.mlp.experts.116.down_proj.weight": "model-00030-of-00101.safetensors", + "model.layers.28.mlp.experts.117.gate_proj.weight": "model-00030-of-00101.safetensors", + "model.layers.28.mlp.experts.117.up_proj.weight": "model-00030-of-00101.safetensors", + "model.layers.28.mlp.experts.117.down_proj.weight": "model-00030-of-00101.safetensors", + "model.layers.28.mlp.experts.118.gate_proj.weight": "model-00030-of-00101.safetensors", + "model.layers.28.mlp.experts.118.up_proj.weight": "model-00030-of-00101.safetensors", + "model.layers.28.mlp.experts.118.down_proj.weight": "model-00030-of-00101.safetensors", + "model.layers.28.mlp.experts.119.gate_proj.weight": "model-00030-of-00101.safetensors", + "model.layers.28.mlp.experts.119.up_proj.weight": "model-00030-of-00101.safetensors", + "model.layers.28.mlp.experts.119.down_proj.weight": "model-00030-of-00101.safetensors", + "model.layers.28.mlp.gate.weight": "model-00030-of-00101.safetensors", + "model.layers.28.mlp.gate.e_score_correction_bias": "model-00030-of-00101.safetensors", + "model.layers.28.mlp.shared_experts.gate_proj.weight": "model-00030-of-00101.safetensors", + "model.layers.28.mlp.shared_experts.up_proj.weight": "model-00030-of-00101.safetensors", + "model.layers.28.mlp.shared_experts.down_proj.weight": "model-00030-of-00101.safetensors", + "model.layers.28.input_layernorm.weight": "model-00030-of-00101.safetensors", + "model.layers.28.post_attention_layernorm.weight": "model-00030-of-00101.safetensors", + "model.layers.29.self_attn.q_proj.weight": "model-00030-of-00101.safetensors", + "model.layers.29.self_attn.q_proj.bias": "model-00030-of-00101.safetensors", + "model.layers.29.self_attn.k_proj.weight": "model-00030-of-00101.safetensors", + "model.layers.29.self_attn.k_proj.bias": "model-00030-of-00101.safetensors", + "model.layers.29.self_attn.v_proj.weight": "model-00030-of-00101.safetensors", + "model.layers.29.self_attn.v_proj.bias": "model-00030-of-00101.safetensors", + "model.layers.29.self_attn.o_proj.weight": "model-00030-of-00101.safetensors", + "model.layers.29.self_attn.q_norm.weight": "model-00030-of-00101.safetensors", + "model.layers.29.self_attn.k_norm.weight": "model-00030-of-00101.safetensors", + "model.layers.29.mlp.experts.0.gate_proj.weight": "model-00030-of-00101.safetensors", + "model.layers.29.mlp.experts.0.up_proj.weight": "model-00030-of-00101.safetensors", + "model.layers.29.mlp.experts.0.down_proj.weight": "model-00030-of-00101.safetensors", + "model.layers.29.mlp.experts.1.gate_proj.weight": "model-00030-of-00101.safetensors", + "model.layers.29.mlp.experts.1.up_proj.weight": "model-00030-of-00101.safetensors", + "model.layers.29.mlp.experts.1.down_proj.weight": "model-00030-of-00101.safetensors", + "model.layers.29.mlp.experts.2.gate_proj.weight": "model-00030-of-00101.safetensors", + "model.layers.29.mlp.experts.2.up_proj.weight": "model-00030-of-00101.safetensors", + "model.layers.29.mlp.experts.2.down_proj.weight": "model-00030-of-00101.safetensors", + "model.layers.29.mlp.experts.3.gate_proj.weight": "model-00030-of-00101.safetensors", + "model.layers.29.mlp.experts.3.up_proj.weight": "model-00030-of-00101.safetensors", + "model.layers.29.mlp.experts.3.down_proj.weight": "model-00030-of-00101.safetensors", + "model.layers.29.mlp.experts.4.gate_proj.weight": "model-00030-of-00101.safetensors", + "model.layers.29.mlp.experts.4.up_proj.weight": "model-00030-of-00101.safetensors", + "model.layers.29.mlp.experts.4.down_proj.weight": "model-00030-of-00101.safetensors", + "model.layers.29.mlp.experts.5.gate_proj.weight": "model-00030-of-00101.safetensors", + "model.layers.29.mlp.experts.5.up_proj.weight": "model-00030-of-00101.safetensors", + "model.layers.29.mlp.experts.5.down_proj.weight": "model-00030-of-00101.safetensors", + "model.layers.29.mlp.experts.6.gate_proj.weight": "model-00030-of-00101.safetensors", + "model.layers.29.mlp.experts.6.up_proj.weight": "model-00030-of-00101.safetensors", + "model.layers.29.mlp.experts.6.down_proj.weight": "model-00030-of-00101.safetensors", + "model.layers.29.mlp.experts.7.gate_proj.weight": "model-00030-of-00101.safetensors", + "model.layers.29.mlp.experts.7.up_proj.weight": "model-00030-of-00101.safetensors", + "model.layers.29.mlp.experts.7.down_proj.weight": "model-00030-of-00101.safetensors", + "model.layers.29.mlp.experts.8.gate_proj.weight": "model-00030-of-00101.safetensors", + "model.layers.29.mlp.experts.8.up_proj.weight": "model-00030-of-00101.safetensors", + "model.layers.29.mlp.experts.8.down_proj.weight": "model-00030-of-00101.safetensors", + "model.layers.29.mlp.experts.9.gate_proj.weight": "model-00030-of-00101.safetensors", + "model.layers.29.mlp.experts.9.up_proj.weight": "model-00030-of-00101.safetensors", + "model.layers.29.mlp.experts.9.down_proj.weight": "model-00030-of-00101.safetensors", + "model.layers.29.mlp.experts.10.gate_proj.weight": "model-00030-of-00101.safetensors", + "model.layers.29.mlp.experts.10.up_proj.weight": "model-00030-of-00101.safetensors", + "model.layers.29.mlp.experts.10.down_proj.weight": "model-00030-of-00101.safetensors", + "model.layers.29.mlp.experts.11.gate_proj.weight": "model-00030-of-00101.safetensors", + "model.layers.29.mlp.experts.11.up_proj.weight": "model-00030-of-00101.safetensors", + "model.layers.29.mlp.experts.11.down_proj.weight": "model-00030-of-00101.safetensors", + "model.layers.29.mlp.experts.12.gate_proj.weight": "model-00030-of-00101.safetensors", + "model.layers.29.mlp.experts.12.up_proj.weight": "model-00030-of-00101.safetensors", + "model.layers.29.mlp.experts.12.down_proj.weight": "model-00030-of-00101.safetensors", + "model.layers.29.mlp.experts.13.gate_proj.weight": "model-00030-of-00101.safetensors", + "model.layers.29.mlp.experts.13.up_proj.weight": "model-00030-of-00101.safetensors", + "model.layers.29.mlp.experts.13.down_proj.weight": "model-00030-of-00101.safetensors", + "model.layers.29.mlp.experts.14.gate_proj.weight": "model-00030-of-00101.safetensors", + "model.layers.29.mlp.experts.14.up_proj.weight": "model-00030-of-00101.safetensors", + "model.layers.29.mlp.experts.14.down_proj.weight": "model-00030-of-00101.safetensors", + "model.layers.29.mlp.experts.15.gate_proj.weight": "model-00030-of-00101.safetensors", + "model.layers.29.mlp.experts.15.up_proj.weight": "model-00030-of-00101.safetensors", + "model.layers.29.mlp.experts.15.down_proj.weight": "model-00030-of-00101.safetensors", + "model.layers.29.mlp.experts.16.gate_proj.weight": "model-00030-of-00101.safetensors", + "model.layers.29.mlp.experts.16.up_proj.weight": "model-00030-of-00101.safetensors", + "model.layers.29.mlp.experts.16.down_proj.weight": "model-00030-of-00101.safetensors", + "model.layers.29.mlp.experts.17.gate_proj.weight": "model-00030-of-00101.safetensors", + "model.layers.29.mlp.experts.17.up_proj.weight": "model-00030-of-00101.safetensors", + "model.layers.29.mlp.experts.17.down_proj.weight": "model-00030-of-00101.safetensors", + "model.layers.29.mlp.experts.18.gate_proj.weight": "model-00030-of-00101.safetensors", + "model.layers.29.mlp.experts.18.up_proj.weight": "model-00030-of-00101.safetensors", + "model.layers.29.mlp.experts.18.down_proj.weight": "model-00030-of-00101.safetensors", + "model.layers.29.mlp.experts.19.gate_proj.weight": "model-00030-of-00101.safetensors", + "model.layers.29.mlp.experts.19.up_proj.weight": "model-00030-of-00101.safetensors", + "model.layers.29.mlp.experts.19.down_proj.weight": "model-00030-of-00101.safetensors", + "model.layers.29.mlp.experts.20.gate_proj.weight": "model-00030-of-00101.safetensors", + "model.layers.29.mlp.experts.20.up_proj.weight": "model-00030-of-00101.safetensors", + "model.layers.29.mlp.experts.20.down_proj.weight": "model-00030-of-00101.safetensors", + "model.layers.29.mlp.experts.21.gate_proj.weight": "model-00030-of-00101.safetensors", + "model.layers.29.mlp.experts.21.up_proj.weight": "model-00030-of-00101.safetensors", + "model.layers.29.mlp.experts.21.down_proj.weight": "model-00030-of-00101.safetensors", + "model.layers.29.mlp.experts.22.gate_proj.weight": "model-00030-of-00101.safetensors", + "model.layers.29.mlp.experts.22.up_proj.weight": "model-00030-of-00101.safetensors", + "model.layers.29.mlp.experts.22.down_proj.weight": "model-00030-of-00101.safetensors", + "model.layers.29.mlp.experts.23.gate_proj.weight": "model-00030-of-00101.safetensors", + "model.layers.29.mlp.experts.23.up_proj.weight": "model-00030-of-00101.safetensors", + "model.layers.29.mlp.experts.23.down_proj.weight": "model-00030-of-00101.safetensors", + "model.layers.29.mlp.experts.24.gate_proj.weight": "model-00030-of-00101.safetensors", + "model.layers.29.mlp.experts.24.up_proj.weight": "model-00030-of-00101.safetensors", + "model.layers.29.mlp.experts.24.down_proj.weight": "model-00030-of-00101.safetensors", + "model.layers.29.mlp.experts.25.gate_proj.weight": "model-00030-of-00101.safetensors", + "model.layers.29.mlp.experts.25.up_proj.weight": "model-00030-of-00101.safetensors", + "model.layers.29.mlp.experts.25.down_proj.weight": "model-00030-of-00101.safetensors", + "model.layers.29.mlp.experts.26.gate_proj.weight": "model-00030-of-00101.safetensors", + "model.layers.29.mlp.experts.26.up_proj.weight": "model-00030-of-00101.safetensors", + "model.layers.29.mlp.experts.26.down_proj.weight": "model-00030-of-00101.safetensors", + "model.layers.29.mlp.experts.27.gate_proj.weight": "model-00030-of-00101.safetensors", + "model.layers.29.mlp.experts.27.up_proj.weight": "model-00030-of-00101.safetensors", + "model.layers.29.mlp.experts.27.down_proj.weight": "model-00031-of-00101.safetensors", + "model.layers.29.mlp.experts.28.gate_proj.weight": "model-00031-of-00101.safetensors", + "model.layers.29.mlp.experts.28.up_proj.weight": "model-00031-of-00101.safetensors", + "model.layers.29.mlp.experts.28.down_proj.weight": "model-00031-of-00101.safetensors", + "model.layers.29.mlp.experts.29.gate_proj.weight": "model-00031-of-00101.safetensors", + "model.layers.29.mlp.experts.29.up_proj.weight": "model-00031-of-00101.safetensors", + "model.layers.29.mlp.experts.29.down_proj.weight": "model-00031-of-00101.safetensors", + "model.layers.29.mlp.experts.30.gate_proj.weight": "model-00031-of-00101.safetensors", + "model.layers.29.mlp.experts.30.up_proj.weight": "model-00031-of-00101.safetensors", + "model.layers.29.mlp.experts.30.down_proj.weight": "model-00031-of-00101.safetensors", + "model.layers.29.mlp.experts.31.gate_proj.weight": "model-00031-of-00101.safetensors", + "model.layers.29.mlp.experts.31.up_proj.weight": "model-00031-of-00101.safetensors", + "model.layers.29.mlp.experts.31.down_proj.weight": "model-00031-of-00101.safetensors", + "model.layers.29.mlp.experts.32.gate_proj.weight": "model-00031-of-00101.safetensors", + "model.layers.29.mlp.experts.32.up_proj.weight": "model-00031-of-00101.safetensors", + "model.layers.29.mlp.experts.32.down_proj.weight": "model-00031-of-00101.safetensors", + "model.layers.29.mlp.experts.33.gate_proj.weight": "model-00031-of-00101.safetensors", + "model.layers.29.mlp.experts.33.up_proj.weight": "model-00031-of-00101.safetensors", + "model.layers.29.mlp.experts.33.down_proj.weight": "model-00031-of-00101.safetensors", + "model.layers.29.mlp.experts.34.gate_proj.weight": "model-00031-of-00101.safetensors", + "model.layers.29.mlp.experts.34.up_proj.weight": "model-00031-of-00101.safetensors", + "model.layers.29.mlp.experts.34.down_proj.weight": "model-00031-of-00101.safetensors", + "model.layers.29.mlp.experts.35.gate_proj.weight": "model-00031-of-00101.safetensors", + "model.layers.29.mlp.experts.35.up_proj.weight": "model-00031-of-00101.safetensors", + "model.layers.29.mlp.experts.35.down_proj.weight": "model-00031-of-00101.safetensors", + "model.layers.29.mlp.experts.36.gate_proj.weight": "model-00031-of-00101.safetensors", + "model.layers.29.mlp.experts.36.up_proj.weight": "model-00031-of-00101.safetensors", + "model.layers.29.mlp.experts.36.down_proj.weight": "model-00031-of-00101.safetensors", + "model.layers.29.mlp.experts.37.gate_proj.weight": "model-00031-of-00101.safetensors", + "model.layers.29.mlp.experts.37.up_proj.weight": "model-00031-of-00101.safetensors", + "model.layers.29.mlp.experts.37.down_proj.weight": "model-00031-of-00101.safetensors", + "model.layers.29.mlp.experts.38.gate_proj.weight": "model-00031-of-00101.safetensors", + "model.layers.29.mlp.experts.38.up_proj.weight": "model-00031-of-00101.safetensors", + "model.layers.29.mlp.experts.38.down_proj.weight": "model-00031-of-00101.safetensors", + "model.layers.29.mlp.experts.39.gate_proj.weight": "model-00031-of-00101.safetensors", + "model.layers.29.mlp.experts.39.up_proj.weight": "model-00031-of-00101.safetensors", + "model.layers.29.mlp.experts.39.down_proj.weight": "model-00031-of-00101.safetensors", + "model.layers.29.mlp.experts.40.gate_proj.weight": "model-00031-of-00101.safetensors", + "model.layers.29.mlp.experts.40.up_proj.weight": "model-00031-of-00101.safetensors", + "model.layers.29.mlp.experts.40.down_proj.weight": "model-00031-of-00101.safetensors", + "model.layers.29.mlp.experts.41.gate_proj.weight": "model-00031-of-00101.safetensors", + "model.layers.29.mlp.experts.41.up_proj.weight": "model-00031-of-00101.safetensors", + "model.layers.29.mlp.experts.41.down_proj.weight": "model-00031-of-00101.safetensors", + "model.layers.29.mlp.experts.42.gate_proj.weight": "model-00031-of-00101.safetensors", + "model.layers.29.mlp.experts.42.up_proj.weight": "model-00031-of-00101.safetensors", + "model.layers.29.mlp.experts.42.down_proj.weight": "model-00031-of-00101.safetensors", + "model.layers.29.mlp.experts.43.gate_proj.weight": "model-00031-of-00101.safetensors", + "model.layers.29.mlp.experts.43.up_proj.weight": "model-00031-of-00101.safetensors", + "model.layers.29.mlp.experts.43.down_proj.weight": "model-00031-of-00101.safetensors", + "model.layers.29.mlp.experts.44.gate_proj.weight": "model-00031-of-00101.safetensors", + "model.layers.29.mlp.experts.44.up_proj.weight": "model-00031-of-00101.safetensors", + "model.layers.29.mlp.experts.44.down_proj.weight": "model-00031-of-00101.safetensors", + "model.layers.29.mlp.experts.45.gate_proj.weight": "model-00031-of-00101.safetensors", + "model.layers.29.mlp.experts.45.up_proj.weight": "model-00031-of-00101.safetensors", + "model.layers.29.mlp.experts.45.down_proj.weight": "model-00031-of-00101.safetensors", + "model.layers.29.mlp.experts.46.gate_proj.weight": "model-00031-of-00101.safetensors", + "model.layers.29.mlp.experts.46.up_proj.weight": "model-00031-of-00101.safetensors", + "model.layers.29.mlp.experts.46.down_proj.weight": "model-00031-of-00101.safetensors", + "model.layers.29.mlp.experts.47.gate_proj.weight": "model-00031-of-00101.safetensors", + "model.layers.29.mlp.experts.47.up_proj.weight": "model-00031-of-00101.safetensors", + "model.layers.29.mlp.experts.47.down_proj.weight": "model-00031-of-00101.safetensors", + "model.layers.29.mlp.experts.48.gate_proj.weight": "model-00031-of-00101.safetensors", + "model.layers.29.mlp.experts.48.up_proj.weight": "model-00031-of-00101.safetensors", + "model.layers.29.mlp.experts.48.down_proj.weight": "model-00031-of-00101.safetensors", + "model.layers.29.mlp.experts.49.gate_proj.weight": "model-00031-of-00101.safetensors", + "model.layers.29.mlp.experts.49.up_proj.weight": "model-00031-of-00101.safetensors", + "model.layers.29.mlp.experts.49.down_proj.weight": "model-00031-of-00101.safetensors", + "model.layers.29.mlp.experts.50.gate_proj.weight": "model-00031-of-00101.safetensors", + "model.layers.29.mlp.experts.50.up_proj.weight": "model-00031-of-00101.safetensors", + "model.layers.29.mlp.experts.50.down_proj.weight": "model-00031-of-00101.safetensors", + "model.layers.29.mlp.experts.51.gate_proj.weight": "model-00031-of-00101.safetensors", + "model.layers.29.mlp.experts.51.up_proj.weight": "model-00031-of-00101.safetensors", + "model.layers.29.mlp.experts.51.down_proj.weight": "model-00031-of-00101.safetensors", + "model.layers.29.mlp.experts.52.gate_proj.weight": "model-00031-of-00101.safetensors", + "model.layers.29.mlp.experts.52.up_proj.weight": "model-00031-of-00101.safetensors", + "model.layers.29.mlp.experts.52.down_proj.weight": "model-00031-of-00101.safetensors", + "model.layers.29.mlp.experts.53.gate_proj.weight": "model-00031-of-00101.safetensors", + "model.layers.29.mlp.experts.53.up_proj.weight": "model-00031-of-00101.safetensors", + "model.layers.29.mlp.experts.53.down_proj.weight": "model-00031-of-00101.safetensors", + "model.layers.29.mlp.experts.54.gate_proj.weight": "model-00031-of-00101.safetensors", + "model.layers.29.mlp.experts.54.up_proj.weight": "model-00031-of-00101.safetensors", + "model.layers.29.mlp.experts.54.down_proj.weight": "model-00031-of-00101.safetensors", + "model.layers.29.mlp.experts.55.gate_proj.weight": "model-00031-of-00101.safetensors", + "model.layers.29.mlp.experts.55.up_proj.weight": "model-00031-of-00101.safetensors", + "model.layers.29.mlp.experts.55.down_proj.weight": "model-00031-of-00101.safetensors", + "model.layers.29.mlp.experts.56.gate_proj.weight": "model-00031-of-00101.safetensors", + "model.layers.29.mlp.experts.56.up_proj.weight": "model-00031-of-00101.safetensors", + "model.layers.29.mlp.experts.56.down_proj.weight": "model-00031-of-00101.safetensors", + "model.layers.29.mlp.experts.57.gate_proj.weight": "model-00031-of-00101.safetensors", + "model.layers.29.mlp.experts.57.up_proj.weight": "model-00031-of-00101.safetensors", + "model.layers.29.mlp.experts.57.down_proj.weight": "model-00031-of-00101.safetensors", + "model.layers.29.mlp.experts.58.gate_proj.weight": "model-00031-of-00101.safetensors", + "model.layers.29.mlp.experts.58.up_proj.weight": "model-00031-of-00101.safetensors", + "model.layers.29.mlp.experts.58.down_proj.weight": "model-00031-of-00101.safetensors", + "model.layers.29.mlp.experts.59.gate_proj.weight": "model-00031-of-00101.safetensors", + "model.layers.29.mlp.experts.59.up_proj.weight": "model-00031-of-00101.safetensors", + "model.layers.29.mlp.experts.59.down_proj.weight": "model-00031-of-00101.safetensors", + "model.layers.29.mlp.experts.60.gate_proj.weight": "model-00031-of-00101.safetensors", + "model.layers.29.mlp.experts.60.up_proj.weight": "model-00031-of-00101.safetensors", + "model.layers.29.mlp.experts.60.down_proj.weight": "model-00031-of-00101.safetensors", + "model.layers.29.mlp.experts.61.gate_proj.weight": "model-00031-of-00101.safetensors", + "model.layers.29.mlp.experts.61.up_proj.weight": "model-00031-of-00101.safetensors", + "model.layers.29.mlp.experts.61.down_proj.weight": "model-00031-of-00101.safetensors", + "model.layers.29.mlp.experts.62.gate_proj.weight": "model-00031-of-00101.safetensors", + "model.layers.29.mlp.experts.62.up_proj.weight": "model-00031-of-00101.safetensors", + "model.layers.29.mlp.experts.62.down_proj.weight": "model-00031-of-00101.safetensors", + "model.layers.29.mlp.experts.63.gate_proj.weight": "model-00031-of-00101.safetensors", + "model.layers.29.mlp.experts.63.up_proj.weight": "model-00031-of-00101.safetensors", + "model.layers.29.mlp.experts.63.down_proj.weight": "model-00031-of-00101.safetensors", + "model.layers.29.mlp.experts.64.gate_proj.weight": "model-00031-of-00101.safetensors", + "model.layers.29.mlp.experts.64.up_proj.weight": "model-00031-of-00101.safetensors", + "model.layers.29.mlp.experts.64.down_proj.weight": "model-00031-of-00101.safetensors", + "model.layers.29.mlp.experts.65.gate_proj.weight": "model-00031-of-00101.safetensors", + "model.layers.29.mlp.experts.65.up_proj.weight": "model-00031-of-00101.safetensors", + "model.layers.29.mlp.experts.65.down_proj.weight": "model-00031-of-00101.safetensors", + "model.layers.29.mlp.experts.66.gate_proj.weight": "model-00031-of-00101.safetensors", + "model.layers.29.mlp.experts.66.up_proj.weight": "model-00031-of-00101.safetensors", + "model.layers.29.mlp.experts.66.down_proj.weight": "model-00031-of-00101.safetensors", + "model.layers.29.mlp.experts.67.gate_proj.weight": "model-00031-of-00101.safetensors", + "model.layers.29.mlp.experts.67.up_proj.weight": "model-00031-of-00101.safetensors", + "model.layers.29.mlp.experts.67.down_proj.weight": "model-00031-of-00101.safetensors", + "model.layers.29.mlp.experts.68.gate_proj.weight": "model-00031-of-00101.safetensors", + "model.layers.29.mlp.experts.68.up_proj.weight": "model-00031-of-00101.safetensors", + "model.layers.29.mlp.experts.68.down_proj.weight": "model-00031-of-00101.safetensors", + "model.layers.29.mlp.experts.69.gate_proj.weight": "model-00031-of-00101.safetensors", + "model.layers.29.mlp.experts.69.up_proj.weight": "model-00031-of-00101.safetensors", + "model.layers.29.mlp.experts.69.down_proj.weight": "model-00031-of-00101.safetensors", + "model.layers.29.mlp.experts.70.gate_proj.weight": "model-00031-of-00101.safetensors", + "model.layers.29.mlp.experts.70.up_proj.weight": "model-00031-of-00101.safetensors", + "model.layers.29.mlp.experts.70.down_proj.weight": "model-00031-of-00101.safetensors", + "model.layers.29.mlp.experts.71.gate_proj.weight": "model-00031-of-00101.safetensors", + "model.layers.29.mlp.experts.71.up_proj.weight": "model-00031-of-00101.safetensors", + "model.layers.29.mlp.experts.71.down_proj.weight": "model-00031-of-00101.safetensors", + "model.layers.29.mlp.experts.72.gate_proj.weight": "model-00031-of-00101.safetensors", + "model.layers.29.mlp.experts.72.up_proj.weight": "model-00031-of-00101.safetensors", + "model.layers.29.mlp.experts.72.down_proj.weight": "model-00031-of-00101.safetensors", + "model.layers.29.mlp.experts.73.gate_proj.weight": "model-00031-of-00101.safetensors", + "model.layers.29.mlp.experts.73.up_proj.weight": "model-00031-of-00101.safetensors", + "model.layers.29.mlp.experts.73.down_proj.weight": "model-00031-of-00101.safetensors", + "model.layers.29.mlp.experts.74.gate_proj.weight": "model-00031-of-00101.safetensors", + "model.layers.29.mlp.experts.74.up_proj.weight": "model-00031-of-00101.safetensors", + "model.layers.29.mlp.experts.74.down_proj.weight": "model-00031-of-00101.safetensors", + "model.layers.29.mlp.experts.75.gate_proj.weight": "model-00031-of-00101.safetensors", + "model.layers.29.mlp.experts.75.up_proj.weight": "model-00031-of-00101.safetensors", + "model.layers.29.mlp.experts.75.down_proj.weight": "model-00031-of-00101.safetensors", + "model.layers.29.mlp.experts.76.gate_proj.weight": "model-00031-of-00101.safetensors", + "model.layers.29.mlp.experts.76.up_proj.weight": "model-00031-of-00101.safetensors", + "model.layers.29.mlp.experts.76.down_proj.weight": "model-00031-of-00101.safetensors", + "model.layers.29.mlp.experts.77.gate_proj.weight": "model-00031-of-00101.safetensors", + "model.layers.29.mlp.experts.77.up_proj.weight": "model-00031-of-00101.safetensors", + "model.layers.29.mlp.experts.77.down_proj.weight": "model-00031-of-00101.safetensors", + "model.layers.29.mlp.experts.78.gate_proj.weight": "model-00031-of-00101.safetensors", + "model.layers.29.mlp.experts.78.up_proj.weight": "model-00031-of-00101.safetensors", + "model.layers.29.mlp.experts.78.down_proj.weight": "model-00031-of-00101.safetensors", + "model.layers.29.mlp.experts.79.gate_proj.weight": "model-00031-of-00101.safetensors", + "model.layers.29.mlp.experts.79.up_proj.weight": "model-00031-of-00101.safetensors", + "model.layers.29.mlp.experts.79.down_proj.weight": "model-00031-of-00101.safetensors", + "model.layers.29.mlp.experts.80.gate_proj.weight": "model-00031-of-00101.safetensors", + "model.layers.29.mlp.experts.80.up_proj.weight": "model-00031-of-00101.safetensors", + "model.layers.29.mlp.experts.80.down_proj.weight": "model-00031-of-00101.safetensors", + "model.layers.29.mlp.experts.81.gate_proj.weight": "model-00031-of-00101.safetensors", + "model.layers.29.mlp.experts.81.up_proj.weight": "model-00031-of-00101.safetensors", + "model.layers.29.mlp.experts.81.down_proj.weight": "model-00031-of-00101.safetensors", + "model.layers.29.mlp.experts.82.gate_proj.weight": "model-00031-of-00101.safetensors", + "model.layers.29.mlp.experts.82.up_proj.weight": "model-00031-of-00101.safetensors", + "model.layers.29.mlp.experts.82.down_proj.weight": "model-00031-of-00101.safetensors", + "model.layers.29.mlp.experts.83.gate_proj.weight": "model-00031-of-00101.safetensors", + "model.layers.29.mlp.experts.83.up_proj.weight": "model-00031-of-00101.safetensors", + "model.layers.29.mlp.experts.83.down_proj.weight": "model-00031-of-00101.safetensors", + "model.layers.29.mlp.experts.84.gate_proj.weight": "model-00031-of-00101.safetensors", + "model.layers.29.mlp.experts.84.up_proj.weight": "model-00031-of-00101.safetensors", + "model.layers.29.mlp.experts.84.down_proj.weight": "model-00031-of-00101.safetensors", + "model.layers.29.mlp.experts.85.gate_proj.weight": "model-00031-of-00101.safetensors", + "model.layers.29.mlp.experts.85.up_proj.weight": "model-00031-of-00101.safetensors", + "model.layers.29.mlp.experts.85.down_proj.weight": "model-00031-of-00101.safetensors", + "model.layers.29.mlp.experts.86.gate_proj.weight": "model-00031-of-00101.safetensors", + "model.layers.29.mlp.experts.86.up_proj.weight": "model-00031-of-00101.safetensors", + "model.layers.29.mlp.experts.86.down_proj.weight": "model-00031-of-00101.safetensors", + "model.layers.29.mlp.experts.87.gate_proj.weight": "model-00031-of-00101.safetensors", + "model.layers.29.mlp.experts.87.up_proj.weight": "model-00031-of-00101.safetensors", + "model.layers.29.mlp.experts.87.down_proj.weight": "model-00031-of-00101.safetensors", + "model.layers.29.mlp.experts.88.gate_proj.weight": "model-00031-of-00101.safetensors", + "model.layers.29.mlp.experts.88.up_proj.weight": "model-00031-of-00101.safetensors", + "model.layers.29.mlp.experts.88.down_proj.weight": "model-00031-of-00101.safetensors", + "model.layers.29.mlp.experts.89.gate_proj.weight": "model-00031-of-00101.safetensors", + "model.layers.29.mlp.experts.89.up_proj.weight": "model-00031-of-00101.safetensors", + "model.layers.29.mlp.experts.89.down_proj.weight": "model-00031-of-00101.safetensors", + "model.layers.29.mlp.experts.90.gate_proj.weight": "model-00031-of-00101.safetensors", + "model.layers.29.mlp.experts.90.up_proj.weight": "model-00031-of-00101.safetensors", + "model.layers.29.mlp.experts.90.down_proj.weight": "model-00031-of-00101.safetensors", + "model.layers.29.mlp.experts.91.gate_proj.weight": "model-00031-of-00101.safetensors", + "model.layers.29.mlp.experts.91.up_proj.weight": "model-00031-of-00101.safetensors", + "model.layers.29.mlp.experts.91.down_proj.weight": "model-00031-of-00101.safetensors", + "model.layers.29.mlp.experts.92.gate_proj.weight": "model-00031-of-00101.safetensors", + "model.layers.29.mlp.experts.92.up_proj.weight": "model-00031-of-00101.safetensors", + "model.layers.29.mlp.experts.92.down_proj.weight": "model-00031-of-00101.safetensors", + "model.layers.29.mlp.experts.93.gate_proj.weight": "model-00031-of-00101.safetensors", + "model.layers.29.mlp.experts.93.up_proj.weight": "model-00031-of-00101.safetensors", + "model.layers.29.mlp.experts.93.down_proj.weight": "model-00031-of-00101.safetensors", + "model.layers.29.mlp.experts.94.gate_proj.weight": "model-00031-of-00101.safetensors", + "model.layers.29.mlp.experts.94.up_proj.weight": "model-00031-of-00101.safetensors", + "model.layers.29.mlp.experts.94.down_proj.weight": "model-00031-of-00101.safetensors", + "model.layers.29.mlp.experts.95.gate_proj.weight": "model-00031-of-00101.safetensors", + "model.layers.29.mlp.experts.95.up_proj.weight": "model-00031-of-00101.safetensors", + "model.layers.29.mlp.experts.95.down_proj.weight": "model-00031-of-00101.safetensors", + "model.layers.29.mlp.experts.96.gate_proj.weight": "model-00031-of-00101.safetensors", + "model.layers.29.mlp.experts.96.up_proj.weight": "model-00031-of-00101.safetensors", + "model.layers.29.mlp.experts.96.down_proj.weight": "model-00031-of-00101.safetensors", + "model.layers.29.mlp.experts.97.gate_proj.weight": "model-00031-of-00101.safetensors", + "model.layers.29.mlp.experts.97.up_proj.weight": "model-00031-of-00101.safetensors", + "model.layers.29.mlp.experts.97.down_proj.weight": "model-00031-of-00101.safetensors", + "model.layers.29.mlp.experts.98.gate_proj.weight": "model-00031-of-00101.safetensors", + "model.layers.29.mlp.experts.98.up_proj.weight": "model-00031-of-00101.safetensors", + "model.layers.29.mlp.experts.98.down_proj.weight": "model-00031-of-00101.safetensors", + "model.layers.29.mlp.experts.99.gate_proj.weight": "model-00031-of-00101.safetensors", + "model.layers.29.mlp.experts.99.up_proj.weight": "model-00031-of-00101.safetensors", + "model.layers.29.mlp.experts.99.down_proj.weight": "model-00031-of-00101.safetensors", + "model.layers.29.mlp.experts.100.gate_proj.weight": "model-00031-of-00101.safetensors", + "model.layers.29.mlp.experts.100.up_proj.weight": "model-00031-of-00101.safetensors", + "model.layers.29.mlp.experts.100.down_proj.weight": "model-00031-of-00101.safetensors", + "model.layers.29.mlp.experts.101.gate_proj.weight": "model-00031-of-00101.safetensors", + "model.layers.29.mlp.experts.101.up_proj.weight": "model-00031-of-00101.safetensors", + "model.layers.29.mlp.experts.101.down_proj.weight": "model-00031-of-00101.safetensors", + "model.layers.29.mlp.experts.102.gate_proj.weight": "model-00031-of-00101.safetensors", + "model.layers.29.mlp.experts.102.up_proj.weight": "model-00031-of-00101.safetensors", + "model.layers.29.mlp.experts.102.down_proj.weight": "model-00031-of-00101.safetensors", + "model.layers.29.mlp.experts.103.gate_proj.weight": "model-00031-of-00101.safetensors", + "model.layers.29.mlp.experts.103.up_proj.weight": "model-00031-of-00101.safetensors", + "model.layers.29.mlp.experts.103.down_proj.weight": "model-00031-of-00101.safetensors", + "model.layers.29.mlp.experts.104.gate_proj.weight": "model-00031-of-00101.safetensors", + "model.layers.29.mlp.experts.104.up_proj.weight": "model-00031-of-00101.safetensors", + "model.layers.29.mlp.experts.104.down_proj.weight": "model-00031-of-00101.safetensors", + "model.layers.29.mlp.experts.105.gate_proj.weight": "model-00031-of-00101.safetensors", + "model.layers.29.mlp.experts.105.up_proj.weight": "model-00031-of-00101.safetensors", + "model.layers.29.mlp.experts.105.down_proj.weight": "model-00031-of-00101.safetensors", + "model.layers.29.mlp.experts.106.gate_proj.weight": "model-00031-of-00101.safetensors", + "model.layers.29.mlp.experts.106.up_proj.weight": "model-00031-of-00101.safetensors", + "model.layers.29.mlp.experts.106.down_proj.weight": "model-00031-of-00101.safetensors", + "model.layers.29.mlp.experts.107.gate_proj.weight": "model-00031-of-00101.safetensors", + "model.layers.29.mlp.experts.107.up_proj.weight": "model-00031-of-00101.safetensors", + "model.layers.29.mlp.experts.107.down_proj.weight": "model-00031-of-00101.safetensors", + "model.layers.29.mlp.experts.108.gate_proj.weight": "model-00031-of-00101.safetensors", + "model.layers.29.mlp.experts.108.up_proj.weight": "model-00031-of-00101.safetensors", + "model.layers.29.mlp.experts.108.down_proj.weight": "model-00031-of-00101.safetensors", + "model.layers.29.mlp.experts.109.gate_proj.weight": "model-00031-of-00101.safetensors", + "model.layers.29.mlp.experts.109.up_proj.weight": "model-00031-of-00101.safetensors", + "model.layers.29.mlp.experts.109.down_proj.weight": "model-00031-of-00101.safetensors", + "model.layers.29.mlp.experts.110.gate_proj.weight": "model-00031-of-00101.safetensors", + "model.layers.29.mlp.experts.110.up_proj.weight": "model-00031-of-00101.safetensors", + "model.layers.29.mlp.experts.110.down_proj.weight": "model-00031-of-00101.safetensors", + "model.layers.29.mlp.experts.111.gate_proj.weight": "model-00031-of-00101.safetensors", + "model.layers.29.mlp.experts.111.up_proj.weight": "model-00031-of-00101.safetensors", + "model.layers.29.mlp.experts.111.down_proj.weight": "model-00031-of-00101.safetensors", + "model.layers.29.mlp.experts.112.gate_proj.weight": "model-00031-of-00101.safetensors", + "model.layers.29.mlp.experts.112.up_proj.weight": "model-00031-of-00101.safetensors", + "model.layers.29.mlp.experts.112.down_proj.weight": "model-00031-of-00101.safetensors", + "model.layers.29.mlp.experts.113.gate_proj.weight": "model-00031-of-00101.safetensors", + "model.layers.29.mlp.experts.113.up_proj.weight": "model-00031-of-00101.safetensors", + "model.layers.29.mlp.experts.113.down_proj.weight": "model-00031-of-00101.safetensors", + "model.layers.29.mlp.experts.114.gate_proj.weight": "model-00031-of-00101.safetensors", + "model.layers.29.mlp.experts.114.up_proj.weight": "model-00031-of-00101.safetensors", + "model.layers.29.mlp.experts.114.down_proj.weight": "model-00031-of-00101.safetensors", + "model.layers.29.mlp.experts.115.gate_proj.weight": "model-00031-of-00101.safetensors", + "model.layers.29.mlp.experts.115.up_proj.weight": "model-00031-of-00101.safetensors", + "model.layers.29.mlp.experts.115.down_proj.weight": "model-00031-of-00101.safetensors", + "model.layers.29.mlp.experts.116.gate_proj.weight": "model-00031-of-00101.safetensors", + "model.layers.29.mlp.experts.116.up_proj.weight": "model-00031-of-00101.safetensors", + "model.layers.29.mlp.experts.116.down_proj.weight": "model-00031-of-00101.safetensors", + "model.layers.29.mlp.experts.117.gate_proj.weight": "model-00031-of-00101.safetensors", + "model.layers.29.mlp.experts.117.up_proj.weight": "model-00031-of-00101.safetensors", + "model.layers.29.mlp.experts.117.down_proj.weight": "model-00031-of-00101.safetensors", + "model.layers.29.mlp.experts.118.gate_proj.weight": "model-00031-of-00101.safetensors", + "model.layers.29.mlp.experts.118.up_proj.weight": "model-00031-of-00101.safetensors", + "model.layers.29.mlp.experts.118.down_proj.weight": "model-00031-of-00101.safetensors", + "model.layers.29.mlp.experts.119.gate_proj.weight": "model-00031-of-00101.safetensors", + "model.layers.29.mlp.experts.119.up_proj.weight": "model-00031-of-00101.safetensors", + "model.layers.29.mlp.experts.119.down_proj.weight": "model-00031-of-00101.safetensors", + "model.layers.29.mlp.gate.weight": "model-00031-of-00101.safetensors", + "model.layers.29.mlp.gate.e_score_correction_bias": "model-00031-of-00101.safetensors", + "model.layers.29.mlp.shared_experts.gate_proj.weight": "model-00031-of-00101.safetensors", + "model.layers.29.mlp.shared_experts.up_proj.weight": "model-00031-of-00101.safetensors", + "model.layers.29.mlp.shared_experts.down_proj.weight": "model-00031-of-00101.safetensors", + "model.layers.29.input_layernorm.weight": "model-00031-of-00101.safetensors", + "model.layers.29.post_attention_layernorm.weight": "model-00031-of-00101.safetensors", + "model.layers.30.self_attn.q_proj.weight": "model-00031-of-00101.safetensors", + "model.layers.30.self_attn.q_proj.bias": "model-00031-of-00101.safetensors", + "model.layers.30.self_attn.k_proj.weight": "model-00031-of-00101.safetensors", + "model.layers.30.self_attn.k_proj.bias": "model-00031-of-00101.safetensors", + "model.layers.30.self_attn.v_proj.weight": "model-00031-of-00101.safetensors", + "model.layers.30.self_attn.v_proj.bias": "model-00031-of-00101.safetensors", + "model.layers.30.self_attn.o_proj.weight": "model-00031-of-00101.safetensors", + "model.layers.30.self_attn.q_norm.weight": "model-00031-of-00101.safetensors", + "model.layers.30.self_attn.k_norm.weight": "model-00031-of-00101.safetensors", + "model.layers.30.mlp.experts.0.gate_proj.weight": "model-00031-of-00101.safetensors", + "model.layers.30.mlp.experts.0.up_proj.weight": "model-00031-of-00101.safetensors", + "model.layers.30.mlp.experts.0.down_proj.weight": "model-00031-of-00101.safetensors", + "model.layers.30.mlp.experts.1.gate_proj.weight": "model-00031-of-00101.safetensors", + "model.layers.30.mlp.experts.1.up_proj.weight": "model-00031-of-00101.safetensors", + "model.layers.30.mlp.experts.1.down_proj.weight": "model-00031-of-00101.safetensors", + "model.layers.30.mlp.experts.2.gate_proj.weight": "model-00031-of-00101.safetensors", + "model.layers.30.mlp.experts.2.up_proj.weight": "model-00031-of-00101.safetensors", + "model.layers.30.mlp.experts.2.down_proj.weight": "model-00031-of-00101.safetensors", + "model.layers.30.mlp.experts.3.gate_proj.weight": "model-00031-of-00101.safetensors", + "model.layers.30.mlp.experts.3.up_proj.weight": "model-00031-of-00101.safetensors", + "model.layers.30.mlp.experts.3.down_proj.weight": "model-00031-of-00101.safetensors", + "model.layers.30.mlp.experts.4.gate_proj.weight": "model-00031-of-00101.safetensors", + "model.layers.30.mlp.experts.4.up_proj.weight": "model-00031-of-00101.safetensors", + "model.layers.30.mlp.experts.4.down_proj.weight": "model-00031-of-00101.safetensors", + "model.layers.30.mlp.experts.5.gate_proj.weight": "model-00031-of-00101.safetensors", + "model.layers.30.mlp.experts.5.up_proj.weight": "model-00031-of-00101.safetensors", + "model.layers.30.mlp.experts.5.down_proj.weight": "model-00031-of-00101.safetensors", + "model.layers.30.mlp.experts.6.gate_proj.weight": "model-00031-of-00101.safetensors", + "model.layers.30.mlp.experts.6.up_proj.weight": "model-00031-of-00101.safetensors", + "model.layers.30.mlp.experts.6.down_proj.weight": "model-00031-of-00101.safetensors", + "model.layers.30.mlp.experts.7.gate_proj.weight": "model-00031-of-00101.safetensors", + "model.layers.30.mlp.experts.7.up_proj.weight": "model-00031-of-00101.safetensors", + "model.layers.30.mlp.experts.7.down_proj.weight": "model-00031-of-00101.safetensors", + "model.layers.30.mlp.experts.8.gate_proj.weight": "model-00031-of-00101.safetensors", + "model.layers.30.mlp.experts.8.up_proj.weight": "model-00031-of-00101.safetensors", + "model.layers.30.mlp.experts.8.down_proj.weight": "model-00031-of-00101.safetensors", + "model.layers.30.mlp.experts.9.gate_proj.weight": "model-00031-of-00101.safetensors", + "model.layers.30.mlp.experts.9.up_proj.weight": "model-00031-of-00101.safetensors", + "model.layers.30.mlp.experts.9.down_proj.weight": "model-00031-of-00101.safetensors", + "model.layers.30.mlp.experts.10.gate_proj.weight": "model-00031-of-00101.safetensors", + "model.layers.30.mlp.experts.10.up_proj.weight": "model-00031-of-00101.safetensors", + "model.layers.30.mlp.experts.10.down_proj.weight": "model-00031-of-00101.safetensors", + "model.layers.30.mlp.experts.11.gate_proj.weight": "model-00031-of-00101.safetensors", + "model.layers.30.mlp.experts.11.up_proj.weight": "model-00031-of-00101.safetensors", + "model.layers.30.mlp.experts.11.down_proj.weight": "model-00031-of-00101.safetensors", + "model.layers.30.mlp.experts.12.gate_proj.weight": "model-00031-of-00101.safetensors", + "model.layers.30.mlp.experts.12.up_proj.weight": "model-00031-of-00101.safetensors", + "model.layers.30.mlp.experts.12.down_proj.weight": "model-00031-of-00101.safetensors", + "model.layers.30.mlp.experts.13.gate_proj.weight": "model-00031-of-00101.safetensors", + "model.layers.30.mlp.experts.13.up_proj.weight": "model-00031-of-00101.safetensors", + "model.layers.30.mlp.experts.13.down_proj.weight": "model-00031-of-00101.safetensors", + "model.layers.30.mlp.experts.14.gate_proj.weight": "model-00031-of-00101.safetensors", + "model.layers.30.mlp.experts.14.up_proj.weight": "model-00032-of-00101.safetensors", + "model.layers.30.mlp.experts.14.down_proj.weight": "model-00032-of-00101.safetensors", + "model.layers.30.mlp.experts.15.gate_proj.weight": "model-00032-of-00101.safetensors", + "model.layers.30.mlp.experts.15.up_proj.weight": "model-00032-of-00101.safetensors", + "model.layers.30.mlp.experts.15.down_proj.weight": "model-00032-of-00101.safetensors", + "model.layers.30.mlp.experts.16.gate_proj.weight": "model-00032-of-00101.safetensors", + "model.layers.30.mlp.experts.16.up_proj.weight": "model-00032-of-00101.safetensors", + "model.layers.30.mlp.experts.16.down_proj.weight": "model-00032-of-00101.safetensors", + "model.layers.30.mlp.experts.17.gate_proj.weight": "model-00032-of-00101.safetensors", + "model.layers.30.mlp.experts.17.up_proj.weight": "model-00032-of-00101.safetensors", + "model.layers.30.mlp.experts.17.down_proj.weight": "model-00032-of-00101.safetensors", + "model.layers.30.mlp.experts.18.gate_proj.weight": "model-00032-of-00101.safetensors", + "model.layers.30.mlp.experts.18.up_proj.weight": "model-00032-of-00101.safetensors", + "model.layers.30.mlp.experts.18.down_proj.weight": "model-00032-of-00101.safetensors", + "model.layers.30.mlp.experts.19.gate_proj.weight": "model-00032-of-00101.safetensors", + "model.layers.30.mlp.experts.19.up_proj.weight": "model-00032-of-00101.safetensors", + "model.layers.30.mlp.experts.19.down_proj.weight": "model-00032-of-00101.safetensors", + "model.layers.30.mlp.experts.20.gate_proj.weight": "model-00032-of-00101.safetensors", + "model.layers.30.mlp.experts.20.up_proj.weight": "model-00032-of-00101.safetensors", + "model.layers.30.mlp.experts.20.down_proj.weight": "model-00032-of-00101.safetensors", + "model.layers.30.mlp.experts.21.gate_proj.weight": "model-00032-of-00101.safetensors", + "model.layers.30.mlp.experts.21.up_proj.weight": "model-00032-of-00101.safetensors", + "model.layers.30.mlp.experts.21.down_proj.weight": "model-00032-of-00101.safetensors", + "model.layers.30.mlp.experts.22.gate_proj.weight": "model-00032-of-00101.safetensors", + "model.layers.30.mlp.experts.22.up_proj.weight": "model-00032-of-00101.safetensors", + "model.layers.30.mlp.experts.22.down_proj.weight": "model-00032-of-00101.safetensors", + "model.layers.30.mlp.experts.23.gate_proj.weight": "model-00032-of-00101.safetensors", + "model.layers.30.mlp.experts.23.up_proj.weight": "model-00032-of-00101.safetensors", + "model.layers.30.mlp.experts.23.down_proj.weight": "model-00032-of-00101.safetensors", + "model.layers.30.mlp.experts.24.gate_proj.weight": "model-00032-of-00101.safetensors", + "model.layers.30.mlp.experts.24.up_proj.weight": "model-00032-of-00101.safetensors", + "model.layers.30.mlp.experts.24.down_proj.weight": "model-00032-of-00101.safetensors", + "model.layers.30.mlp.experts.25.gate_proj.weight": "model-00032-of-00101.safetensors", + "model.layers.30.mlp.experts.25.up_proj.weight": "model-00032-of-00101.safetensors", + "model.layers.30.mlp.experts.25.down_proj.weight": "model-00032-of-00101.safetensors", + "model.layers.30.mlp.experts.26.gate_proj.weight": "model-00032-of-00101.safetensors", + "model.layers.30.mlp.experts.26.up_proj.weight": "model-00032-of-00101.safetensors", + "model.layers.30.mlp.experts.26.down_proj.weight": "model-00032-of-00101.safetensors", + "model.layers.30.mlp.experts.27.gate_proj.weight": "model-00032-of-00101.safetensors", + "model.layers.30.mlp.experts.27.up_proj.weight": "model-00032-of-00101.safetensors", + "model.layers.30.mlp.experts.27.down_proj.weight": "model-00032-of-00101.safetensors", + "model.layers.30.mlp.experts.28.gate_proj.weight": "model-00032-of-00101.safetensors", + "model.layers.30.mlp.experts.28.up_proj.weight": "model-00032-of-00101.safetensors", + "model.layers.30.mlp.experts.28.down_proj.weight": "model-00032-of-00101.safetensors", + "model.layers.30.mlp.experts.29.gate_proj.weight": "model-00032-of-00101.safetensors", + "model.layers.30.mlp.experts.29.up_proj.weight": "model-00032-of-00101.safetensors", + "model.layers.30.mlp.experts.29.down_proj.weight": "model-00032-of-00101.safetensors", + "model.layers.30.mlp.experts.30.gate_proj.weight": "model-00032-of-00101.safetensors", + "model.layers.30.mlp.experts.30.up_proj.weight": "model-00032-of-00101.safetensors", + "model.layers.30.mlp.experts.30.down_proj.weight": "model-00032-of-00101.safetensors", + "model.layers.30.mlp.experts.31.gate_proj.weight": "model-00032-of-00101.safetensors", + "model.layers.30.mlp.experts.31.up_proj.weight": "model-00032-of-00101.safetensors", + "model.layers.30.mlp.experts.31.down_proj.weight": "model-00032-of-00101.safetensors", + "model.layers.30.mlp.experts.32.gate_proj.weight": "model-00032-of-00101.safetensors", + "model.layers.30.mlp.experts.32.up_proj.weight": "model-00032-of-00101.safetensors", + "model.layers.30.mlp.experts.32.down_proj.weight": "model-00032-of-00101.safetensors", + "model.layers.30.mlp.experts.33.gate_proj.weight": "model-00032-of-00101.safetensors", + "model.layers.30.mlp.experts.33.up_proj.weight": "model-00032-of-00101.safetensors", + "model.layers.30.mlp.experts.33.down_proj.weight": "model-00032-of-00101.safetensors", + "model.layers.30.mlp.experts.34.gate_proj.weight": "model-00032-of-00101.safetensors", + "model.layers.30.mlp.experts.34.up_proj.weight": "model-00032-of-00101.safetensors", + "model.layers.30.mlp.experts.34.down_proj.weight": "model-00032-of-00101.safetensors", + "model.layers.30.mlp.experts.35.gate_proj.weight": "model-00032-of-00101.safetensors", + "model.layers.30.mlp.experts.35.up_proj.weight": "model-00032-of-00101.safetensors", + "model.layers.30.mlp.experts.35.down_proj.weight": "model-00032-of-00101.safetensors", + "model.layers.30.mlp.experts.36.gate_proj.weight": "model-00032-of-00101.safetensors", + "model.layers.30.mlp.experts.36.up_proj.weight": "model-00032-of-00101.safetensors", + "model.layers.30.mlp.experts.36.down_proj.weight": "model-00032-of-00101.safetensors", + "model.layers.30.mlp.experts.37.gate_proj.weight": "model-00032-of-00101.safetensors", + "model.layers.30.mlp.experts.37.up_proj.weight": "model-00032-of-00101.safetensors", + "model.layers.30.mlp.experts.37.down_proj.weight": "model-00032-of-00101.safetensors", + "model.layers.30.mlp.experts.38.gate_proj.weight": "model-00032-of-00101.safetensors", + "model.layers.30.mlp.experts.38.up_proj.weight": "model-00032-of-00101.safetensors", + "model.layers.30.mlp.experts.38.down_proj.weight": "model-00032-of-00101.safetensors", + "model.layers.30.mlp.experts.39.gate_proj.weight": "model-00032-of-00101.safetensors", + "model.layers.30.mlp.experts.39.up_proj.weight": "model-00032-of-00101.safetensors", + "model.layers.30.mlp.experts.39.down_proj.weight": "model-00032-of-00101.safetensors", + "model.layers.30.mlp.experts.40.gate_proj.weight": "model-00032-of-00101.safetensors", + "model.layers.30.mlp.experts.40.up_proj.weight": "model-00032-of-00101.safetensors", + "model.layers.30.mlp.experts.40.down_proj.weight": "model-00032-of-00101.safetensors", + "model.layers.30.mlp.experts.41.gate_proj.weight": "model-00032-of-00101.safetensors", + "model.layers.30.mlp.experts.41.up_proj.weight": "model-00032-of-00101.safetensors", + "model.layers.30.mlp.experts.41.down_proj.weight": "model-00032-of-00101.safetensors", + "model.layers.30.mlp.experts.42.gate_proj.weight": "model-00032-of-00101.safetensors", + "model.layers.30.mlp.experts.42.up_proj.weight": "model-00032-of-00101.safetensors", + "model.layers.30.mlp.experts.42.down_proj.weight": "model-00032-of-00101.safetensors", + "model.layers.30.mlp.experts.43.gate_proj.weight": "model-00032-of-00101.safetensors", + "model.layers.30.mlp.experts.43.up_proj.weight": "model-00032-of-00101.safetensors", + "model.layers.30.mlp.experts.43.down_proj.weight": "model-00032-of-00101.safetensors", + "model.layers.30.mlp.experts.44.gate_proj.weight": "model-00032-of-00101.safetensors", + "model.layers.30.mlp.experts.44.up_proj.weight": "model-00032-of-00101.safetensors", + "model.layers.30.mlp.experts.44.down_proj.weight": "model-00032-of-00101.safetensors", + "model.layers.30.mlp.experts.45.gate_proj.weight": "model-00032-of-00101.safetensors", + "model.layers.30.mlp.experts.45.up_proj.weight": "model-00032-of-00101.safetensors", + "model.layers.30.mlp.experts.45.down_proj.weight": "model-00032-of-00101.safetensors", + "model.layers.30.mlp.experts.46.gate_proj.weight": "model-00032-of-00101.safetensors", + "model.layers.30.mlp.experts.46.up_proj.weight": "model-00032-of-00101.safetensors", + "model.layers.30.mlp.experts.46.down_proj.weight": "model-00032-of-00101.safetensors", + "model.layers.30.mlp.experts.47.gate_proj.weight": "model-00032-of-00101.safetensors", + "model.layers.30.mlp.experts.47.up_proj.weight": "model-00032-of-00101.safetensors", + "model.layers.30.mlp.experts.47.down_proj.weight": "model-00032-of-00101.safetensors", + "model.layers.30.mlp.experts.48.gate_proj.weight": "model-00032-of-00101.safetensors", + "model.layers.30.mlp.experts.48.up_proj.weight": "model-00032-of-00101.safetensors", + "model.layers.30.mlp.experts.48.down_proj.weight": "model-00032-of-00101.safetensors", + "model.layers.30.mlp.experts.49.gate_proj.weight": "model-00032-of-00101.safetensors", + "model.layers.30.mlp.experts.49.up_proj.weight": "model-00032-of-00101.safetensors", + "model.layers.30.mlp.experts.49.down_proj.weight": "model-00032-of-00101.safetensors", + "model.layers.30.mlp.experts.50.gate_proj.weight": "model-00032-of-00101.safetensors", + "model.layers.30.mlp.experts.50.up_proj.weight": "model-00032-of-00101.safetensors", + "model.layers.30.mlp.experts.50.down_proj.weight": "model-00032-of-00101.safetensors", + "model.layers.30.mlp.experts.51.gate_proj.weight": "model-00032-of-00101.safetensors", + "model.layers.30.mlp.experts.51.up_proj.weight": "model-00032-of-00101.safetensors", + "model.layers.30.mlp.experts.51.down_proj.weight": "model-00032-of-00101.safetensors", + "model.layers.30.mlp.experts.52.gate_proj.weight": "model-00032-of-00101.safetensors", + "model.layers.30.mlp.experts.52.up_proj.weight": "model-00032-of-00101.safetensors", + "model.layers.30.mlp.experts.52.down_proj.weight": "model-00032-of-00101.safetensors", + "model.layers.30.mlp.experts.53.gate_proj.weight": "model-00032-of-00101.safetensors", + "model.layers.30.mlp.experts.53.up_proj.weight": "model-00032-of-00101.safetensors", + "model.layers.30.mlp.experts.53.down_proj.weight": "model-00032-of-00101.safetensors", + "model.layers.30.mlp.experts.54.gate_proj.weight": "model-00032-of-00101.safetensors", + "model.layers.30.mlp.experts.54.up_proj.weight": "model-00032-of-00101.safetensors", + "model.layers.30.mlp.experts.54.down_proj.weight": "model-00032-of-00101.safetensors", + "model.layers.30.mlp.experts.55.gate_proj.weight": "model-00032-of-00101.safetensors", + "model.layers.30.mlp.experts.55.up_proj.weight": "model-00032-of-00101.safetensors", + "model.layers.30.mlp.experts.55.down_proj.weight": "model-00032-of-00101.safetensors", + "model.layers.30.mlp.experts.56.gate_proj.weight": "model-00032-of-00101.safetensors", + "model.layers.30.mlp.experts.56.up_proj.weight": "model-00032-of-00101.safetensors", + "model.layers.30.mlp.experts.56.down_proj.weight": "model-00032-of-00101.safetensors", + "model.layers.30.mlp.experts.57.gate_proj.weight": "model-00032-of-00101.safetensors", + "model.layers.30.mlp.experts.57.up_proj.weight": "model-00032-of-00101.safetensors", + "model.layers.30.mlp.experts.57.down_proj.weight": "model-00032-of-00101.safetensors", + "model.layers.30.mlp.experts.58.gate_proj.weight": "model-00032-of-00101.safetensors", + "model.layers.30.mlp.experts.58.up_proj.weight": "model-00032-of-00101.safetensors", + "model.layers.30.mlp.experts.58.down_proj.weight": "model-00032-of-00101.safetensors", + "model.layers.30.mlp.experts.59.gate_proj.weight": "model-00032-of-00101.safetensors", + "model.layers.30.mlp.experts.59.up_proj.weight": "model-00032-of-00101.safetensors", + "model.layers.30.mlp.experts.59.down_proj.weight": "model-00032-of-00101.safetensors", + "model.layers.30.mlp.experts.60.gate_proj.weight": "model-00032-of-00101.safetensors", + "model.layers.30.mlp.experts.60.up_proj.weight": "model-00032-of-00101.safetensors", + "model.layers.30.mlp.experts.60.down_proj.weight": "model-00032-of-00101.safetensors", + "model.layers.30.mlp.experts.61.gate_proj.weight": "model-00032-of-00101.safetensors", + "model.layers.30.mlp.experts.61.up_proj.weight": "model-00032-of-00101.safetensors", + "model.layers.30.mlp.experts.61.down_proj.weight": "model-00032-of-00101.safetensors", + "model.layers.30.mlp.experts.62.gate_proj.weight": "model-00032-of-00101.safetensors", + "model.layers.30.mlp.experts.62.up_proj.weight": "model-00032-of-00101.safetensors", + "model.layers.30.mlp.experts.62.down_proj.weight": "model-00032-of-00101.safetensors", + "model.layers.30.mlp.experts.63.gate_proj.weight": "model-00032-of-00101.safetensors", + "model.layers.30.mlp.experts.63.up_proj.weight": "model-00032-of-00101.safetensors", + "model.layers.30.mlp.experts.63.down_proj.weight": "model-00032-of-00101.safetensors", + "model.layers.30.mlp.experts.64.gate_proj.weight": "model-00032-of-00101.safetensors", + "model.layers.30.mlp.experts.64.up_proj.weight": "model-00032-of-00101.safetensors", + "model.layers.30.mlp.experts.64.down_proj.weight": "model-00032-of-00101.safetensors", + "model.layers.30.mlp.experts.65.gate_proj.weight": "model-00032-of-00101.safetensors", + "model.layers.30.mlp.experts.65.up_proj.weight": "model-00032-of-00101.safetensors", + "model.layers.30.mlp.experts.65.down_proj.weight": "model-00032-of-00101.safetensors", + "model.layers.30.mlp.experts.66.gate_proj.weight": "model-00032-of-00101.safetensors", + "model.layers.30.mlp.experts.66.up_proj.weight": "model-00032-of-00101.safetensors", + "model.layers.30.mlp.experts.66.down_proj.weight": "model-00032-of-00101.safetensors", + "model.layers.30.mlp.experts.67.gate_proj.weight": "model-00032-of-00101.safetensors", + "model.layers.30.mlp.experts.67.up_proj.weight": "model-00032-of-00101.safetensors", + "model.layers.30.mlp.experts.67.down_proj.weight": "model-00032-of-00101.safetensors", + "model.layers.30.mlp.experts.68.gate_proj.weight": "model-00032-of-00101.safetensors", + "model.layers.30.mlp.experts.68.up_proj.weight": "model-00032-of-00101.safetensors", + "model.layers.30.mlp.experts.68.down_proj.weight": "model-00032-of-00101.safetensors", + "model.layers.30.mlp.experts.69.gate_proj.weight": "model-00032-of-00101.safetensors", + "model.layers.30.mlp.experts.69.up_proj.weight": "model-00032-of-00101.safetensors", + "model.layers.30.mlp.experts.69.down_proj.weight": "model-00032-of-00101.safetensors", + "model.layers.30.mlp.experts.70.gate_proj.weight": "model-00032-of-00101.safetensors", + "model.layers.30.mlp.experts.70.up_proj.weight": "model-00032-of-00101.safetensors", + "model.layers.30.mlp.experts.70.down_proj.weight": "model-00032-of-00101.safetensors", + "model.layers.30.mlp.experts.71.gate_proj.weight": "model-00032-of-00101.safetensors", + "model.layers.30.mlp.experts.71.up_proj.weight": "model-00032-of-00101.safetensors", + "model.layers.30.mlp.experts.71.down_proj.weight": "model-00032-of-00101.safetensors", + "model.layers.30.mlp.experts.72.gate_proj.weight": "model-00032-of-00101.safetensors", + "model.layers.30.mlp.experts.72.up_proj.weight": "model-00032-of-00101.safetensors", + "model.layers.30.mlp.experts.72.down_proj.weight": "model-00032-of-00101.safetensors", + "model.layers.30.mlp.experts.73.gate_proj.weight": "model-00032-of-00101.safetensors", + "model.layers.30.mlp.experts.73.up_proj.weight": "model-00032-of-00101.safetensors", + "model.layers.30.mlp.experts.73.down_proj.weight": "model-00032-of-00101.safetensors", + "model.layers.30.mlp.experts.74.gate_proj.weight": "model-00032-of-00101.safetensors", + "model.layers.30.mlp.experts.74.up_proj.weight": "model-00032-of-00101.safetensors", + "model.layers.30.mlp.experts.74.down_proj.weight": "model-00032-of-00101.safetensors", + "model.layers.30.mlp.experts.75.gate_proj.weight": "model-00032-of-00101.safetensors", + "model.layers.30.mlp.experts.75.up_proj.weight": "model-00032-of-00101.safetensors", + "model.layers.30.mlp.experts.75.down_proj.weight": "model-00032-of-00101.safetensors", + "model.layers.30.mlp.experts.76.gate_proj.weight": "model-00032-of-00101.safetensors", + "model.layers.30.mlp.experts.76.up_proj.weight": "model-00032-of-00101.safetensors", + "model.layers.30.mlp.experts.76.down_proj.weight": "model-00032-of-00101.safetensors", + "model.layers.30.mlp.experts.77.gate_proj.weight": "model-00032-of-00101.safetensors", + "model.layers.30.mlp.experts.77.up_proj.weight": "model-00032-of-00101.safetensors", + "model.layers.30.mlp.experts.77.down_proj.weight": "model-00032-of-00101.safetensors", + "model.layers.30.mlp.experts.78.gate_proj.weight": "model-00032-of-00101.safetensors", + "model.layers.30.mlp.experts.78.up_proj.weight": "model-00032-of-00101.safetensors", + "model.layers.30.mlp.experts.78.down_proj.weight": "model-00032-of-00101.safetensors", + "model.layers.30.mlp.experts.79.gate_proj.weight": "model-00032-of-00101.safetensors", + "model.layers.30.mlp.experts.79.up_proj.weight": "model-00032-of-00101.safetensors", + "model.layers.30.mlp.experts.79.down_proj.weight": "model-00032-of-00101.safetensors", + "model.layers.30.mlp.experts.80.gate_proj.weight": "model-00032-of-00101.safetensors", + "model.layers.30.mlp.experts.80.up_proj.weight": "model-00032-of-00101.safetensors", + "model.layers.30.mlp.experts.80.down_proj.weight": "model-00032-of-00101.safetensors", + "model.layers.30.mlp.experts.81.gate_proj.weight": "model-00032-of-00101.safetensors", + "model.layers.30.mlp.experts.81.up_proj.weight": "model-00032-of-00101.safetensors", + "model.layers.30.mlp.experts.81.down_proj.weight": "model-00032-of-00101.safetensors", + "model.layers.30.mlp.experts.82.gate_proj.weight": "model-00032-of-00101.safetensors", + "model.layers.30.mlp.experts.82.up_proj.weight": "model-00032-of-00101.safetensors", + "model.layers.30.mlp.experts.82.down_proj.weight": "model-00032-of-00101.safetensors", + "model.layers.30.mlp.experts.83.gate_proj.weight": "model-00032-of-00101.safetensors", + "model.layers.30.mlp.experts.83.up_proj.weight": "model-00032-of-00101.safetensors", + "model.layers.30.mlp.experts.83.down_proj.weight": "model-00032-of-00101.safetensors", + "model.layers.30.mlp.experts.84.gate_proj.weight": "model-00032-of-00101.safetensors", + "model.layers.30.mlp.experts.84.up_proj.weight": "model-00032-of-00101.safetensors", + "model.layers.30.mlp.experts.84.down_proj.weight": "model-00032-of-00101.safetensors", + "model.layers.30.mlp.experts.85.gate_proj.weight": "model-00032-of-00101.safetensors", + "model.layers.30.mlp.experts.85.up_proj.weight": "model-00032-of-00101.safetensors", + "model.layers.30.mlp.experts.85.down_proj.weight": "model-00032-of-00101.safetensors", + "model.layers.30.mlp.experts.86.gate_proj.weight": "model-00032-of-00101.safetensors", + "model.layers.30.mlp.experts.86.up_proj.weight": "model-00032-of-00101.safetensors", + "model.layers.30.mlp.experts.86.down_proj.weight": "model-00032-of-00101.safetensors", + "model.layers.30.mlp.experts.87.gate_proj.weight": "model-00032-of-00101.safetensors", + "model.layers.30.mlp.experts.87.up_proj.weight": "model-00032-of-00101.safetensors", + "model.layers.30.mlp.experts.87.down_proj.weight": "model-00032-of-00101.safetensors", + "model.layers.30.mlp.experts.88.gate_proj.weight": "model-00032-of-00101.safetensors", + "model.layers.30.mlp.experts.88.up_proj.weight": "model-00032-of-00101.safetensors", + "model.layers.30.mlp.experts.88.down_proj.weight": "model-00032-of-00101.safetensors", + "model.layers.30.mlp.experts.89.gate_proj.weight": "model-00032-of-00101.safetensors", + "model.layers.30.mlp.experts.89.up_proj.weight": "model-00032-of-00101.safetensors", + "model.layers.30.mlp.experts.89.down_proj.weight": "model-00032-of-00101.safetensors", + "model.layers.30.mlp.experts.90.gate_proj.weight": "model-00032-of-00101.safetensors", + "model.layers.30.mlp.experts.90.up_proj.weight": "model-00032-of-00101.safetensors", + "model.layers.30.mlp.experts.90.down_proj.weight": "model-00032-of-00101.safetensors", + "model.layers.30.mlp.experts.91.gate_proj.weight": "model-00032-of-00101.safetensors", + "model.layers.30.mlp.experts.91.up_proj.weight": "model-00032-of-00101.safetensors", + "model.layers.30.mlp.experts.91.down_proj.weight": "model-00032-of-00101.safetensors", + "model.layers.30.mlp.experts.92.gate_proj.weight": "model-00032-of-00101.safetensors", + "model.layers.30.mlp.experts.92.up_proj.weight": "model-00032-of-00101.safetensors", + "model.layers.30.mlp.experts.92.down_proj.weight": "model-00032-of-00101.safetensors", + "model.layers.30.mlp.experts.93.gate_proj.weight": "model-00032-of-00101.safetensors", + "model.layers.30.mlp.experts.93.up_proj.weight": "model-00032-of-00101.safetensors", + "model.layers.30.mlp.experts.93.down_proj.weight": "model-00032-of-00101.safetensors", + "model.layers.30.mlp.experts.94.gate_proj.weight": "model-00032-of-00101.safetensors", + "model.layers.30.mlp.experts.94.up_proj.weight": "model-00032-of-00101.safetensors", + "model.layers.30.mlp.experts.94.down_proj.weight": "model-00032-of-00101.safetensors", + "model.layers.30.mlp.experts.95.gate_proj.weight": "model-00032-of-00101.safetensors", + "model.layers.30.mlp.experts.95.up_proj.weight": "model-00032-of-00101.safetensors", + "model.layers.30.mlp.experts.95.down_proj.weight": "model-00032-of-00101.safetensors", + "model.layers.30.mlp.experts.96.gate_proj.weight": "model-00032-of-00101.safetensors", + "model.layers.30.mlp.experts.96.up_proj.weight": "model-00032-of-00101.safetensors", + "model.layers.30.mlp.experts.96.down_proj.weight": "model-00032-of-00101.safetensors", + "model.layers.30.mlp.experts.97.gate_proj.weight": "model-00032-of-00101.safetensors", + "model.layers.30.mlp.experts.97.up_proj.weight": "model-00032-of-00101.safetensors", + "model.layers.30.mlp.experts.97.down_proj.weight": "model-00032-of-00101.safetensors", + "model.layers.30.mlp.experts.98.gate_proj.weight": "model-00032-of-00101.safetensors", + "model.layers.30.mlp.experts.98.up_proj.weight": "model-00032-of-00101.safetensors", + "model.layers.30.mlp.experts.98.down_proj.weight": "model-00032-of-00101.safetensors", + "model.layers.30.mlp.experts.99.gate_proj.weight": "model-00032-of-00101.safetensors", + "model.layers.30.mlp.experts.99.up_proj.weight": "model-00032-of-00101.safetensors", + "model.layers.30.mlp.experts.99.down_proj.weight": "model-00032-of-00101.safetensors", + "model.layers.30.mlp.experts.100.gate_proj.weight": "model-00032-of-00101.safetensors", + "model.layers.30.mlp.experts.100.up_proj.weight": "model-00032-of-00101.safetensors", + "model.layers.30.mlp.experts.100.down_proj.weight": "model-00032-of-00101.safetensors", + "model.layers.30.mlp.experts.101.gate_proj.weight": "model-00032-of-00101.safetensors", + "model.layers.30.mlp.experts.101.up_proj.weight": "model-00032-of-00101.safetensors", + "model.layers.30.mlp.experts.101.down_proj.weight": "model-00032-of-00101.safetensors", + "model.layers.30.mlp.experts.102.gate_proj.weight": "model-00032-of-00101.safetensors", + "model.layers.30.mlp.experts.102.up_proj.weight": "model-00032-of-00101.safetensors", + "model.layers.30.mlp.experts.102.down_proj.weight": "model-00032-of-00101.safetensors", + "model.layers.30.mlp.experts.103.gate_proj.weight": "model-00032-of-00101.safetensors", + "model.layers.30.mlp.experts.103.up_proj.weight": "model-00032-of-00101.safetensors", + "model.layers.30.mlp.experts.103.down_proj.weight": "model-00032-of-00101.safetensors", + "model.layers.30.mlp.experts.104.gate_proj.weight": "model-00032-of-00101.safetensors", + "model.layers.30.mlp.experts.104.up_proj.weight": "model-00032-of-00101.safetensors", + "model.layers.30.mlp.experts.104.down_proj.weight": "model-00032-of-00101.safetensors", + "model.layers.30.mlp.experts.105.gate_proj.weight": "model-00032-of-00101.safetensors", + "model.layers.30.mlp.experts.105.up_proj.weight": "model-00032-of-00101.safetensors", + "model.layers.30.mlp.experts.105.down_proj.weight": "model-00032-of-00101.safetensors", + "model.layers.30.mlp.experts.106.gate_proj.weight": "model-00032-of-00101.safetensors", + "model.layers.30.mlp.experts.106.up_proj.weight": "model-00032-of-00101.safetensors", + "model.layers.30.mlp.experts.106.down_proj.weight": "model-00032-of-00101.safetensors", + "model.layers.30.mlp.experts.107.gate_proj.weight": "model-00032-of-00101.safetensors", + "model.layers.30.mlp.experts.107.up_proj.weight": "model-00032-of-00101.safetensors", + "model.layers.30.mlp.experts.107.down_proj.weight": "model-00032-of-00101.safetensors", + "model.layers.30.mlp.experts.108.gate_proj.weight": "model-00032-of-00101.safetensors", + "model.layers.30.mlp.experts.108.up_proj.weight": "model-00032-of-00101.safetensors", + "model.layers.30.mlp.experts.108.down_proj.weight": "model-00032-of-00101.safetensors", + "model.layers.30.mlp.experts.109.gate_proj.weight": "model-00032-of-00101.safetensors", + "model.layers.30.mlp.experts.109.up_proj.weight": "model-00032-of-00101.safetensors", + "model.layers.30.mlp.experts.109.down_proj.weight": "model-00032-of-00101.safetensors", + "model.layers.30.mlp.experts.110.gate_proj.weight": "model-00032-of-00101.safetensors", + "model.layers.30.mlp.experts.110.up_proj.weight": "model-00032-of-00101.safetensors", + "model.layers.30.mlp.experts.110.down_proj.weight": "model-00032-of-00101.safetensors", + "model.layers.30.mlp.experts.111.gate_proj.weight": "model-00032-of-00101.safetensors", + "model.layers.30.mlp.experts.111.up_proj.weight": "model-00032-of-00101.safetensors", + "model.layers.30.mlp.experts.111.down_proj.weight": "model-00032-of-00101.safetensors", + "model.layers.30.mlp.experts.112.gate_proj.weight": "model-00032-of-00101.safetensors", + "model.layers.30.mlp.experts.112.up_proj.weight": "model-00032-of-00101.safetensors", + "model.layers.30.mlp.experts.112.down_proj.weight": "model-00032-of-00101.safetensors", + "model.layers.30.mlp.experts.113.gate_proj.weight": "model-00032-of-00101.safetensors", + "model.layers.30.mlp.experts.113.up_proj.weight": "model-00032-of-00101.safetensors", + "model.layers.30.mlp.experts.113.down_proj.weight": "model-00032-of-00101.safetensors", + "model.layers.30.mlp.experts.114.gate_proj.weight": "model-00032-of-00101.safetensors", + "model.layers.30.mlp.experts.114.up_proj.weight": "model-00032-of-00101.safetensors", + "model.layers.30.mlp.experts.114.down_proj.weight": "model-00032-of-00101.safetensors", + "model.layers.30.mlp.experts.115.gate_proj.weight": "model-00032-of-00101.safetensors", + "model.layers.30.mlp.experts.115.up_proj.weight": "model-00032-of-00101.safetensors", + "model.layers.30.mlp.experts.115.down_proj.weight": "model-00032-of-00101.safetensors", + "model.layers.30.mlp.experts.116.gate_proj.weight": "model-00032-of-00101.safetensors", + "model.layers.30.mlp.experts.116.up_proj.weight": "model-00032-of-00101.safetensors", + "model.layers.30.mlp.experts.116.down_proj.weight": "model-00032-of-00101.safetensors", + "model.layers.30.mlp.experts.117.gate_proj.weight": "model-00032-of-00101.safetensors", + "model.layers.30.mlp.experts.117.up_proj.weight": "model-00032-of-00101.safetensors", + "model.layers.30.mlp.experts.117.down_proj.weight": "model-00032-of-00101.safetensors", + "model.layers.30.mlp.experts.118.gate_proj.weight": "model-00032-of-00101.safetensors", + "model.layers.30.mlp.experts.118.up_proj.weight": "model-00032-of-00101.safetensors", + "model.layers.30.mlp.experts.118.down_proj.weight": "model-00032-of-00101.safetensors", + "model.layers.30.mlp.experts.119.gate_proj.weight": "model-00032-of-00101.safetensors", + "model.layers.30.mlp.experts.119.up_proj.weight": "model-00032-of-00101.safetensors", + "model.layers.30.mlp.experts.119.down_proj.weight": "model-00032-of-00101.safetensors", + "model.layers.30.mlp.gate.weight": "model-00032-of-00101.safetensors", + "model.layers.30.mlp.gate.e_score_correction_bias": "model-00032-of-00101.safetensors", + "model.layers.30.mlp.shared_experts.gate_proj.weight": "model-00032-of-00101.safetensors", + "model.layers.30.mlp.shared_experts.up_proj.weight": "model-00032-of-00101.safetensors", + "model.layers.30.mlp.shared_experts.down_proj.weight": "model-00032-of-00101.safetensors", + "model.layers.30.input_layernorm.weight": "model-00032-of-00101.safetensors", + "model.layers.30.post_attention_layernorm.weight": "model-00032-of-00101.safetensors", + "model.layers.31.self_attn.q_proj.weight": "model-00032-of-00101.safetensors", + "model.layers.31.self_attn.q_proj.bias": "model-00032-of-00101.safetensors", + "model.layers.31.self_attn.k_proj.weight": "model-00032-of-00101.safetensors", + "model.layers.31.self_attn.k_proj.bias": "model-00032-of-00101.safetensors", + "model.layers.31.self_attn.v_proj.weight": "model-00032-of-00101.safetensors", + "model.layers.31.self_attn.v_proj.bias": "model-00032-of-00101.safetensors", + "model.layers.31.self_attn.o_proj.weight": "model-00032-of-00101.safetensors", + "model.layers.31.self_attn.q_norm.weight": "model-00032-of-00101.safetensors", + "model.layers.31.self_attn.k_norm.weight": "model-00032-of-00101.safetensors", + "model.layers.31.mlp.experts.0.gate_proj.weight": "model-00032-of-00101.safetensors", + "model.layers.31.mlp.experts.0.up_proj.weight": "model-00032-of-00101.safetensors", + "model.layers.31.mlp.experts.0.down_proj.weight": "model-00032-of-00101.safetensors", + "model.layers.31.mlp.experts.1.gate_proj.weight": "model-00033-of-00101.safetensors", + "model.layers.31.mlp.experts.1.up_proj.weight": "model-00033-of-00101.safetensors", + "model.layers.31.mlp.experts.1.down_proj.weight": "model-00033-of-00101.safetensors", + "model.layers.31.mlp.experts.2.gate_proj.weight": "model-00033-of-00101.safetensors", + "model.layers.31.mlp.experts.2.up_proj.weight": "model-00033-of-00101.safetensors", + "model.layers.31.mlp.experts.2.down_proj.weight": "model-00033-of-00101.safetensors", + "model.layers.31.mlp.experts.3.gate_proj.weight": "model-00033-of-00101.safetensors", + "model.layers.31.mlp.experts.3.up_proj.weight": "model-00033-of-00101.safetensors", + "model.layers.31.mlp.experts.3.down_proj.weight": "model-00033-of-00101.safetensors", + "model.layers.31.mlp.experts.4.gate_proj.weight": "model-00033-of-00101.safetensors", + "model.layers.31.mlp.experts.4.up_proj.weight": "model-00033-of-00101.safetensors", + "model.layers.31.mlp.experts.4.down_proj.weight": "model-00033-of-00101.safetensors", + "model.layers.31.mlp.experts.5.gate_proj.weight": "model-00033-of-00101.safetensors", + "model.layers.31.mlp.experts.5.up_proj.weight": "model-00033-of-00101.safetensors", + "model.layers.31.mlp.experts.5.down_proj.weight": "model-00033-of-00101.safetensors", + "model.layers.31.mlp.experts.6.gate_proj.weight": "model-00033-of-00101.safetensors", + "model.layers.31.mlp.experts.6.up_proj.weight": "model-00033-of-00101.safetensors", + "model.layers.31.mlp.experts.6.down_proj.weight": "model-00033-of-00101.safetensors", + "model.layers.31.mlp.experts.7.gate_proj.weight": "model-00033-of-00101.safetensors", + "model.layers.31.mlp.experts.7.up_proj.weight": "model-00033-of-00101.safetensors", + "model.layers.31.mlp.experts.7.down_proj.weight": "model-00033-of-00101.safetensors", + "model.layers.31.mlp.experts.8.gate_proj.weight": "model-00033-of-00101.safetensors", + "model.layers.31.mlp.experts.8.up_proj.weight": "model-00033-of-00101.safetensors", + "model.layers.31.mlp.experts.8.down_proj.weight": "model-00033-of-00101.safetensors", + "model.layers.31.mlp.experts.9.gate_proj.weight": "model-00033-of-00101.safetensors", + "model.layers.31.mlp.experts.9.up_proj.weight": "model-00033-of-00101.safetensors", + "model.layers.31.mlp.experts.9.down_proj.weight": "model-00033-of-00101.safetensors", + "model.layers.31.mlp.experts.10.gate_proj.weight": "model-00033-of-00101.safetensors", + "model.layers.31.mlp.experts.10.up_proj.weight": "model-00033-of-00101.safetensors", + "model.layers.31.mlp.experts.10.down_proj.weight": "model-00033-of-00101.safetensors", + "model.layers.31.mlp.experts.11.gate_proj.weight": "model-00033-of-00101.safetensors", + "model.layers.31.mlp.experts.11.up_proj.weight": "model-00033-of-00101.safetensors", + "model.layers.31.mlp.experts.11.down_proj.weight": "model-00033-of-00101.safetensors", + "model.layers.31.mlp.experts.12.gate_proj.weight": "model-00033-of-00101.safetensors", + "model.layers.31.mlp.experts.12.up_proj.weight": "model-00033-of-00101.safetensors", + "model.layers.31.mlp.experts.12.down_proj.weight": "model-00033-of-00101.safetensors", + "model.layers.31.mlp.experts.13.gate_proj.weight": "model-00033-of-00101.safetensors", + "model.layers.31.mlp.experts.13.up_proj.weight": "model-00033-of-00101.safetensors", + "model.layers.31.mlp.experts.13.down_proj.weight": "model-00033-of-00101.safetensors", + "model.layers.31.mlp.experts.14.gate_proj.weight": "model-00033-of-00101.safetensors", + "model.layers.31.mlp.experts.14.up_proj.weight": "model-00033-of-00101.safetensors", + "model.layers.31.mlp.experts.14.down_proj.weight": "model-00033-of-00101.safetensors", + "model.layers.31.mlp.experts.15.gate_proj.weight": "model-00033-of-00101.safetensors", + "model.layers.31.mlp.experts.15.up_proj.weight": "model-00033-of-00101.safetensors", + "model.layers.31.mlp.experts.15.down_proj.weight": "model-00033-of-00101.safetensors", + "model.layers.31.mlp.experts.16.gate_proj.weight": "model-00033-of-00101.safetensors", + "model.layers.31.mlp.experts.16.up_proj.weight": "model-00033-of-00101.safetensors", + "model.layers.31.mlp.experts.16.down_proj.weight": "model-00033-of-00101.safetensors", + "model.layers.31.mlp.experts.17.gate_proj.weight": "model-00033-of-00101.safetensors", + "model.layers.31.mlp.experts.17.up_proj.weight": "model-00033-of-00101.safetensors", + "model.layers.31.mlp.experts.17.down_proj.weight": "model-00033-of-00101.safetensors", + "model.layers.31.mlp.experts.18.gate_proj.weight": "model-00033-of-00101.safetensors", + "model.layers.31.mlp.experts.18.up_proj.weight": "model-00033-of-00101.safetensors", + "model.layers.31.mlp.experts.18.down_proj.weight": "model-00033-of-00101.safetensors", + "model.layers.31.mlp.experts.19.gate_proj.weight": "model-00033-of-00101.safetensors", + "model.layers.31.mlp.experts.19.up_proj.weight": "model-00033-of-00101.safetensors", + "model.layers.31.mlp.experts.19.down_proj.weight": "model-00033-of-00101.safetensors", + "model.layers.31.mlp.experts.20.gate_proj.weight": "model-00033-of-00101.safetensors", + "model.layers.31.mlp.experts.20.up_proj.weight": "model-00033-of-00101.safetensors", + "model.layers.31.mlp.experts.20.down_proj.weight": "model-00033-of-00101.safetensors", + "model.layers.31.mlp.experts.21.gate_proj.weight": "model-00033-of-00101.safetensors", + "model.layers.31.mlp.experts.21.up_proj.weight": "model-00033-of-00101.safetensors", + "model.layers.31.mlp.experts.21.down_proj.weight": "model-00033-of-00101.safetensors", + "model.layers.31.mlp.experts.22.gate_proj.weight": "model-00033-of-00101.safetensors", + "model.layers.31.mlp.experts.22.up_proj.weight": "model-00033-of-00101.safetensors", + "model.layers.31.mlp.experts.22.down_proj.weight": "model-00033-of-00101.safetensors", + "model.layers.31.mlp.experts.23.gate_proj.weight": "model-00033-of-00101.safetensors", + "model.layers.31.mlp.experts.23.up_proj.weight": "model-00033-of-00101.safetensors", + "model.layers.31.mlp.experts.23.down_proj.weight": "model-00033-of-00101.safetensors", + "model.layers.31.mlp.experts.24.gate_proj.weight": "model-00033-of-00101.safetensors", + "model.layers.31.mlp.experts.24.up_proj.weight": "model-00033-of-00101.safetensors", + "model.layers.31.mlp.experts.24.down_proj.weight": "model-00033-of-00101.safetensors", + "model.layers.31.mlp.experts.25.gate_proj.weight": "model-00033-of-00101.safetensors", + "model.layers.31.mlp.experts.25.up_proj.weight": "model-00033-of-00101.safetensors", + "model.layers.31.mlp.experts.25.down_proj.weight": "model-00033-of-00101.safetensors", + "model.layers.31.mlp.experts.26.gate_proj.weight": "model-00033-of-00101.safetensors", + "model.layers.31.mlp.experts.26.up_proj.weight": "model-00033-of-00101.safetensors", + "model.layers.31.mlp.experts.26.down_proj.weight": "model-00033-of-00101.safetensors", + "model.layers.31.mlp.experts.27.gate_proj.weight": "model-00033-of-00101.safetensors", + "model.layers.31.mlp.experts.27.up_proj.weight": "model-00033-of-00101.safetensors", + "model.layers.31.mlp.experts.27.down_proj.weight": "model-00033-of-00101.safetensors", + "model.layers.31.mlp.experts.28.gate_proj.weight": "model-00033-of-00101.safetensors", + "model.layers.31.mlp.experts.28.up_proj.weight": "model-00033-of-00101.safetensors", + "model.layers.31.mlp.experts.28.down_proj.weight": "model-00033-of-00101.safetensors", + "model.layers.31.mlp.experts.29.gate_proj.weight": "model-00033-of-00101.safetensors", + "model.layers.31.mlp.experts.29.up_proj.weight": "model-00033-of-00101.safetensors", + "model.layers.31.mlp.experts.29.down_proj.weight": "model-00033-of-00101.safetensors", + "model.layers.31.mlp.experts.30.gate_proj.weight": "model-00033-of-00101.safetensors", + "model.layers.31.mlp.experts.30.up_proj.weight": "model-00033-of-00101.safetensors", + "model.layers.31.mlp.experts.30.down_proj.weight": "model-00033-of-00101.safetensors", + "model.layers.31.mlp.experts.31.gate_proj.weight": "model-00033-of-00101.safetensors", + "model.layers.31.mlp.experts.31.up_proj.weight": "model-00033-of-00101.safetensors", + "model.layers.31.mlp.experts.31.down_proj.weight": "model-00033-of-00101.safetensors", + "model.layers.31.mlp.experts.32.gate_proj.weight": "model-00033-of-00101.safetensors", + "model.layers.31.mlp.experts.32.up_proj.weight": "model-00033-of-00101.safetensors", + "model.layers.31.mlp.experts.32.down_proj.weight": "model-00033-of-00101.safetensors", + "model.layers.31.mlp.experts.33.gate_proj.weight": "model-00033-of-00101.safetensors", + "model.layers.31.mlp.experts.33.up_proj.weight": "model-00033-of-00101.safetensors", + "model.layers.31.mlp.experts.33.down_proj.weight": "model-00033-of-00101.safetensors", + "model.layers.31.mlp.experts.34.gate_proj.weight": "model-00033-of-00101.safetensors", + "model.layers.31.mlp.experts.34.up_proj.weight": "model-00033-of-00101.safetensors", + "model.layers.31.mlp.experts.34.down_proj.weight": "model-00033-of-00101.safetensors", + "model.layers.31.mlp.experts.35.gate_proj.weight": "model-00033-of-00101.safetensors", + "model.layers.31.mlp.experts.35.up_proj.weight": "model-00033-of-00101.safetensors", + "model.layers.31.mlp.experts.35.down_proj.weight": "model-00033-of-00101.safetensors", + "model.layers.31.mlp.experts.36.gate_proj.weight": "model-00033-of-00101.safetensors", + "model.layers.31.mlp.experts.36.up_proj.weight": "model-00033-of-00101.safetensors", + "model.layers.31.mlp.experts.36.down_proj.weight": "model-00033-of-00101.safetensors", + "model.layers.31.mlp.experts.37.gate_proj.weight": "model-00033-of-00101.safetensors", + "model.layers.31.mlp.experts.37.up_proj.weight": "model-00033-of-00101.safetensors", + "model.layers.31.mlp.experts.37.down_proj.weight": "model-00033-of-00101.safetensors", + "model.layers.31.mlp.experts.38.gate_proj.weight": "model-00033-of-00101.safetensors", + "model.layers.31.mlp.experts.38.up_proj.weight": "model-00033-of-00101.safetensors", + "model.layers.31.mlp.experts.38.down_proj.weight": "model-00033-of-00101.safetensors", + "model.layers.31.mlp.experts.39.gate_proj.weight": "model-00033-of-00101.safetensors", + "model.layers.31.mlp.experts.39.up_proj.weight": "model-00033-of-00101.safetensors", + "model.layers.31.mlp.experts.39.down_proj.weight": "model-00033-of-00101.safetensors", + "model.layers.31.mlp.experts.40.gate_proj.weight": "model-00033-of-00101.safetensors", + "model.layers.31.mlp.experts.40.up_proj.weight": "model-00033-of-00101.safetensors", + "model.layers.31.mlp.experts.40.down_proj.weight": "model-00033-of-00101.safetensors", + "model.layers.31.mlp.experts.41.gate_proj.weight": "model-00033-of-00101.safetensors", + "model.layers.31.mlp.experts.41.up_proj.weight": "model-00033-of-00101.safetensors", + "model.layers.31.mlp.experts.41.down_proj.weight": "model-00033-of-00101.safetensors", + "model.layers.31.mlp.experts.42.gate_proj.weight": "model-00033-of-00101.safetensors", + "model.layers.31.mlp.experts.42.up_proj.weight": "model-00033-of-00101.safetensors", + "model.layers.31.mlp.experts.42.down_proj.weight": "model-00033-of-00101.safetensors", + "model.layers.31.mlp.experts.43.gate_proj.weight": "model-00033-of-00101.safetensors", + "model.layers.31.mlp.experts.43.up_proj.weight": "model-00033-of-00101.safetensors", + "model.layers.31.mlp.experts.43.down_proj.weight": "model-00033-of-00101.safetensors", + "model.layers.31.mlp.experts.44.gate_proj.weight": "model-00033-of-00101.safetensors", + "model.layers.31.mlp.experts.44.up_proj.weight": "model-00033-of-00101.safetensors", + "model.layers.31.mlp.experts.44.down_proj.weight": "model-00033-of-00101.safetensors", + "model.layers.31.mlp.experts.45.gate_proj.weight": "model-00033-of-00101.safetensors", + "model.layers.31.mlp.experts.45.up_proj.weight": "model-00033-of-00101.safetensors", + "model.layers.31.mlp.experts.45.down_proj.weight": "model-00033-of-00101.safetensors", + "model.layers.31.mlp.experts.46.gate_proj.weight": "model-00033-of-00101.safetensors", + "model.layers.31.mlp.experts.46.up_proj.weight": "model-00033-of-00101.safetensors", + "model.layers.31.mlp.experts.46.down_proj.weight": "model-00033-of-00101.safetensors", + "model.layers.31.mlp.experts.47.gate_proj.weight": "model-00033-of-00101.safetensors", + "model.layers.31.mlp.experts.47.up_proj.weight": "model-00033-of-00101.safetensors", + "model.layers.31.mlp.experts.47.down_proj.weight": "model-00033-of-00101.safetensors", + "model.layers.31.mlp.experts.48.gate_proj.weight": "model-00033-of-00101.safetensors", + "model.layers.31.mlp.experts.48.up_proj.weight": "model-00033-of-00101.safetensors", + "model.layers.31.mlp.experts.48.down_proj.weight": "model-00033-of-00101.safetensors", + "model.layers.31.mlp.experts.49.gate_proj.weight": "model-00033-of-00101.safetensors", + "model.layers.31.mlp.experts.49.up_proj.weight": "model-00033-of-00101.safetensors", + "model.layers.31.mlp.experts.49.down_proj.weight": "model-00033-of-00101.safetensors", + "model.layers.31.mlp.experts.50.gate_proj.weight": "model-00033-of-00101.safetensors", + "model.layers.31.mlp.experts.50.up_proj.weight": "model-00033-of-00101.safetensors", + "model.layers.31.mlp.experts.50.down_proj.weight": "model-00033-of-00101.safetensors", + "model.layers.31.mlp.experts.51.gate_proj.weight": "model-00033-of-00101.safetensors", + "model.layers.31.mlp.experts.51.up_proj.weight": "model-00033-of-00101.safetensors", + "model.layers.31.mlp.experts.51.down_proj.weight": "model-00033-of-00101.safetensors", + "model.layers.31.mlp.experts.52.gate_proj.weight": "model-00033-of-00101.safetensors", + "model.layers.31.mlp.experts.52.up_proj.weight": "model-00033-of-00101.safetensors", + "model.layers.31.mlp.experts.52.down_proj.weight": "model-00033-of-00101.safetensors", + "model.layers.31.mlp.experts.53.gate_proj.weight": "model-00033-of-00101.safetensors", + "model.layers.31.mlp.experts.53.up_proj.weight": "model-00033-of-00101.safetensors", + "model.layers.31.mlp.experts.53.down_proj.weight": "model-00033-of-00101.safetensors", + "model.layers.31.mlp.experts.54.gate_proj.weight": "model-00033-of-00101.safetensors", + "model.layers.31.mlp.experts.54.up_proj.weight": "model-00033-of-00101.safetensors", + "model.layers.31.mlp.experts.54.down_proj.weight": "model-00033-of-00101.safetensors", + "model.layers.31.mlp.experts.55.gate_proj.weight": "model-00033-of-00101.safetensors", + "model.layers.31.mlp.experts.55.up_proj.weight": "model-00033-of-00101.safetensors", + "model.layers.31.mlp.experts.55.down_proj.weight": "model-00033-of-00101.safetensors", + "model.layers.31.mlp.experts.56.gate_proj.weight": "model-00033-of-00101.safetensors", + "model.layers.31.mlp.experts.56.up_proj.weight": "model-00033-of-00101.safetensors", + "model.layers.31.mlp.experts.56.down_proj.weight": "model-00033-of-00101.safetensors", + "model.layers.31.mlp.experts.57.gate_proj.weight": "model-00033-of-00101.safetensors", + "model.layers.31.mlp.experts.57.up_proj.weight": "model-00033-of-00101.safetensors", + "model.layers.31.mlp.experts.57.down_proj.weight": "model-00033-of-00101.safetensors", + "model.layers.31.mlp.experts.58.gate_proj.weight": "model-00033-of-00101.safetensors", + "model.layers.31.mlp.experts.58.up_proj.weight": "model-00033-of-00101.safetensors", + "model.layers.31.mlp.experts.58.down_proj.weight": "model-00033-of-00101.safetensors", + "model.layers.31.mlp.experts.59.gate_proj.weight": "model-00033-of-00101.safetensors", + "model.layers.31.mlp.experts.59.up_proj.weight": "model-00033-of-00101.safetensors", + "model.layers.31.mlp.experts.59.down_proj.weight": "model-00033-of-00101.safetensors", + "model.layers.31.mlp.experts.60.gate_proj.weight": "model-00033-of-00101.safetensors", + "model.layers.31.mlp.experts.60.up_proj.weight": "model-00033-of-00101.safetensors", + "model.layers.31.mlp.experts.60.down_proj.weight": "model-00033-of-00101.safetensors", + "model.layers.31.mlp.experts.61.gate_proj.weight": "model-00033-of-00101.safetensors", + "model.layers.31.mlp.experts.61.up_proj.weight": "model-00033-of-00101.safetensors", + "model.layers.31.mlp.experts.61.down_proj.weight": "model-00033-of-00101.safetensors", + "model.layers.31.mlp.experts.62.gate_proj.weight": "model-00033-of-00101.safetensors", + "model.layers.31.mlp.experts.62.up_proj.weight": "model-00033-of-00101.safetensors", + "model.layers.31.mlp.experts.62.down_proj.weight": "model-00033-of-00101.safetensors", + "model.layers.31.mlp.experts.63.gate_proj.weight": "model-00033-of-00101.safetensors", + "model.layers.31.mlp.experts.63.up_proj.weight": "model-00033-of-00101.safetensors", + "model.layers.31.mlp.experts.63.down_proj.weight": "model-00033-of-00101.safetensors", + "model.layers.31.mlp.experts.64.gate_proj.weight": "model-00033-of-00101.safetensors", + "model.layers.31.mlp.experts.64.up_proj.weight": "model-00033-of-00101.safetensors", + "model.layers.31.mlp.experts.64.down_proj.weight": "model-00033-of-00101.safetensors", + "model.layers.31.mlp.experts.65.gate_proj.weight": "model-00033-of-00101.safetensors", + "model.layers.31.mlp.experts.65.up_proj.weight": "model-00033-of-00101.safetensors", + "model.layers.31.mlp.experts.65.down_proj.weight": "model-00033-of-00101.safetensors", + "model.layers.31.mlp.experts.66.gate_proj.weight": "model-00033-of-00101.safetensors", + "model.layers.31.mlp.experts.66.up_proj.weight": "model-00033-of-00101.safetensors", + "model.layers.31.mlp.experts.66.down_proj.weight": "model-00033-of-00101.safetensors", + "model.layers.31.mlp.experts.67.gate_proj.weight": "model-00033-of-00101.safetensors", + "model.layers.31.mlp.experts.67.up_proj.weight": "model-00033-of-00101.safetensors", + "model.layers.31.mlp.experts.67.down_proj.weight": "model-00033-of-00101.safetensors", + "model.layers.31.mlp.experts.68.gate_proj.weight": "model-00033-of-00101.safetensors", + "model.layers.31.mlp.experts.68.up_proj.weight": "model-00033-of-00101.safetensors", + "model.layers.31.mlp.experts.68.down_proj.weight": "model-00033-of-00101.safetensors", + "model.layers.31.mlp.experts.69.gate_proj.weight": "model-00033-of-00101.safetensors", + "model.layers.31.mlp.experts.69.up_proj.weight": "model-00033-of-00101.safetensors", + "model.layers.31.mlp.experts.69.down_proj.weight": "model-00033-of-00101.safetensors", + "model.layers.31.mlp.experts.70.gate_proj.weight": "model-00033-of-00101.safetensors", + "model.layers.31.mlp.experts.70.up_proj.weight": "model-00033-of-00101.safetensors", + "model.layers.31.mlp.experts.70.down_proj.weight": "model-00033-of-00101.safetensors", + "model.layers.31.mlp.experts.71.gate_proj.weight": "model-00033-of-00101.safetensors", + "model.layers.31.mlp.experts.71.up_proj.weight": "model-00033-of-00101.safetensors", + "model.layers.31.mlp.experts.71.down_proj.weight": "model-00033-of-00101.safetensors", + "model.layers.31.mlp.experts.72.gate_proj.weight": "model-00033-of-00101.safetensors", + "model.layers.31.mlp.experts.72.up_proj.weight": "model-00033-of-00101.safetensors", + "model.layers.31.mlp.experts.72.down_proj.weight": "model-00033-of-00101.safetensors", + "model.layers.31.mlp.experts.73.gate_proj.weight": "model-00033-of-00101.safetensors", + "model.layers.31.mlp.experts.73.up_proj.weight": "model-00033-of-00101.safetensors", + "model.layers.31.mlp.experts.73.down_proj.weight": "model-00033-of-00101.safetensors", + "model.layers.31.mlp.experts.74.gate_proj.weight": "model-00033-of-00101.safetensors", + "model.layers.31.mlp.experts.74.up_proj.weight": "model-00033-of-00101.safetensors", + "model.layers.31.mlp.experts.74.down_proj.weight": "model-00033-of-00101.safetensors", + "model.layers.31.mlp.experts.75.gate_proj.weight": "model-00033-of-00101.safetensors", + "model.layers.31.mlp.experts.75.up_proj.weight": "model-00033-of-00101.safetensors", + "model.layers.31.mlp.experts.75.down_proj.weight": "model-00033-of-00101.safetensors", + "model.layers.31.mlp.experts.76.gate_proj.weight": "model-00033-of-00101.safetensors", + "model.layers.31.mlp.experts.76.up_proj.weight": "model-00033-of-00101.safetensors", + "model.layers.31.mlp.experts.76.down_proj.weight": "model-00033-of-00101.safetensors", + "model.layers.31.mlp.experts.77.gate_proj.weight": "model-00033-of-00101.safetensors", + "model.layers.31.mlp.experts.77.up_proj.weight": "model-00033-of-00101.safetensors", + "model.layers.31.mlp.experts.77.down_proj.weight": "model-00033-of-00101.safetensors", + "model.layers.31.mlp.experts.78.gate_proj.weight": "model-00033-of-00101.safetensors", + "model.layers.31.mlp.experts.78.up_proj.weight": "model-00033-of-00101.safetensors", + "model.layers.31.mlp.experts.78.down_proj.weight": "model-00033-of-00101.safetensors", + "model.layers.31.mlp.experts.79.gate_proj.weight": "model-00033-of-00101.safetensors", + "model.layers.31.mlp.experts.79.up_proj.weight": "model-00033-of-00101.safetensors", + "model.layers.31.mlp.experts.79.down_proj.weight": "model-00033-of-00101.safetensors", + "model.layers.31.mlp.experts.80.gate_proj.weight": "model-00033-of-00101.safetensors", + "model.layers.31.mlp.experts.80.up_proj.weight": "model-00033-of-00101.safetensors", + "model.layers.31.mlp.experts.80.down_proj.weight": "model-00033-of-00101.safetensors", + "model.layers.31.mlp.experts.81.gate_proj.weight": "model-00033-of-00101.safetensors", + "model.layers.31.mlp.experts.81.up_proj.weight": "model-00033-of-00101.safetensors", + "model.layers.31.mlp.experts.81.down_proj.weight": "model-00033-of-00101.safetensors", + "model.layers.31.mlp.experts.82.gate_proj.weight": "model-00033-of-00101.safetensors", + "model.layers.31.mlp.experts.82.up_proj.weight": "model-00033-of-00101.safetensors", + "model.layers.31.mlp.experts.82.down_proj.weight": "model-00033-of-00101.safetensors", + "model.layers.31.mlp.experts.83.gate_proj.weight": "model-00033-of-00101.safetensors", + "model.layers.31.mlp.experts.83.up_proj.weight": "model-00033-of-00101.safetensors", + "model.layers.31.mlp.experts.83.down_proj.weight": "model-00033-of-00101.safetensors", + "model.layers.31.mlp.experts.84.gate_proj.weight": "model-00033-of-00101.safetensors", + "model.layers.31.mlp.experts.84.up_proj.weight": "model-00033-of-00101.safetensors", + "model.layers.31.mlp.experts.84.down_proj.weight": "model-00033-of-00101.safetensors", + "model.layers.31.mlp.experts.85.gate_proj.weight": "model-00033-of-00101.safetensors", + "model.layers.31.mlp.experts.85.up_proj.weight": "model-00033-of-00101.safetensors", + "model.layers.31.mlp.experts.85.down_proj.weight": "model-00033-of-00101.safetensors", + "model.layers.31.mlp.experts.86.gate_proj.weight": "model-00033-of-00101.safetensors", + "model.layers.31.mlp.experts.86.up_proj.weight": "model-00033-of-00101.safetensors", + "model.layers.31.mlp.experts.86.down_proj.weight": "model-00033-of-00101.safetensors", + "model.layers.31.mlp.experts.87.gate_proj.weight": "model-00033-of-00101.safetensors", + "model.layers.31.mlp.experts.87.up_proj.weight": "model-00033-of-00101.safetensors", + "model.layers.31.mlp.experts.87.down_proj.weight": "model-00033-of-00101.safetensors", + "model.layers.31.mlp.experts.88.gate_proj.weight": "model-00033-of-00101.safetensors", + "model.layers.31.mlp.experts.88.up_proj.weight": "model-00033-of-00101.safetensors", + "model.layers.31.mlp.experts.88.down_proj.weight": "model-00033-of-00101.safetensors", + "model.layers.31.mlp.experts.89.gate_proj.weight": "model-00033-of-00101.safetensors", + "model.layers.31.mlp.experts.89.up_proj.weight": "model-00033-of-00101.safetensors", + "model.layers.31.mlp.experts.89.down_proj.weight": "model-00033-of-00101.safetensors", + "model.layers.31.mlp.experts.90.gate_proj.weight": "model-00033-of-00101.safetensors", + "model.layers.31.mlp.experts.90.up_proj.weight": "model-00033-of-00101.safetensors", + "model.layers.31.mlp.experts.90.down_proj.weight": "model-00033-of-00101.safetensors", + "model.layers.31.mlp.experts.91.gate_proj.weight": "model-00033-of-00101.safetensors", + "model.layers.31.mlp.experts.91.up_proj.weight": "model-00033-of-00101.safetensors", + "model.layers.31.mlp.experts.91.down_proj.weight": "model-00033-of-00101.safetensors", + "model.layers.31.mlp.experts.92.gate_proj.weight": "model-00033-of-00101.safetensors", + "model.layers.31.mlp.experts.92.up_proj.weight": "model-00033-of-00101.safetensors", + "model.layers.31.mlp.experts.92.down_proj.weight": "model-00033-of-00101.safetensors", + "model.layers.31.mlp.experts.93.gate_proj.weight": "model-00033-of-00101.safetensors", + "model.layers.31.mlp.experts.93.up_proj.weight": "model-00033-of-00101.safetensors", + "model.layers.31.mlp.experts.93.down_proj.weight": "model-00033-of-00101.safetensors", + "model.layers.31.mlp.experts.94.gate_proj.weight": "model-00033-of-00101.safetensors", + "model.layers.31.mlp.experts.94.up_proj.weight": "model-00033-of-00101.safetensors", + "model.layers.31.mlp.experts.94.down_proj.weight": "model-00033-of-00101.safetensors", + "model.layers.31.mlp.experts.95.gate_proj.weight": "model-00033-of-00101.safetensors", + "model.layers.31.mlp.experts.95.up_proj.weight": "model-00033-of-00101.safetensors", + "model.layers.31.mlp.experts.95.down_proj.weight": "model-00033-of-00101.safetensors", + "model.layers.31.mlp.experts.96.gate_proj.weight": "model-00033-of-00101.safetensors", + "model.layers.31.mlp.experts.96.up_proj.weight": "model-00033-of-00101.safetensors", + "model.layers.31.mlp.experts.96.down_proj.weight": "model-00033-of-00101.safetensors", + "model.layers.31.mlp.experts.97.gate_proj.weight": "model-00033-of-00101.safetensors", + "model.layers.31.mlp.experts.97.up_proj.weight": "model-00033-of-00101.safetensors", + "model.layers.31.mlp.experts.97.down_proj.weight": "model-00033-of-00101.safetensors", + "model.layers.31.mlp.experts.98.gate_proj.weight": "model-00033-of-00101.safetensors", + "model.layers.31.mlp.experts.98.up_proj.weight": "model-00033-of-00101.safetensors", + "model.layers.31.mlp.experts.98.down_proj.weight": "model-00033-of-00101.safetensors", + "model.layers.31.mlp.experts.99.gate_proj.weight": "model-00033-of-00101.safetensors", + "model.layers.31.mlp.experts.99.up_proj.weight": "model-00033-of-00101.safetensors", + "model.layers.31.mlp.experts.99.down_proj.weight": "model-00033-of-00101.safetensors", + "model.layers.31.mlp.experts.100.gate_proj.weight": "model-00033-of-00101.safetensors", + "model.layers.31.mlp.experts.100.up_proj.weight": "model-00033-of-00101.safetensors", + "model.layers.31.mlp.experts.100.down_proj.weight": "model-00033-of-00101.safetensors", + "model.layers.31.mlp.experts.101.gate_proj.weight": "model-00033-of-00101.safetensors", + "model.layers.31.mlp.experts.101.up_proj.weight": "model-00033-of-00101.safetensors", + "model.layers.31.mlp.experts.101.down_proj.weight": "model-00033-of-00101.safetensors", + "model.layers.31.mlp.experts.102.gate_proj.weight": "model-00033-of-00101.safetensors", + "model.layers.31.mlp.experts.102.up_proj.weight": "model-00033-of-00101.safetensors", + "model.layers.31.mlp.experts.102.down_proj.weight": "model-00033-of-00101.safetensors", + "model.layers.31.mlp.experts.103.gate_proj.weight": "model-00033-of-00101.safetensors", + "model.layers.31.mlp.experts.103.up_proj.weight": "model-00033-of-00101.safetensors", + "model.layers.31.mlp.experts.103.down_proj.weight": "model-00033-of-00101.safetensors", + "model.layers.31.mlp.experts.104.gate_proj.weight": "model-00033-of-00101.safetensors", + "model.layers.31.mlp.experts.104.up_proj.weight": "model-00033-of-00101.safetensors", + "model.layers.31.mlp.experts.104.down_proj.weight": "model-00033-of-00101.safetensors", + "model.layers.31.mlp.experts.105.gate_proj.weight": "model-00033-of-00101.safetensors", + "model.layers.31.mlp.experts.105.up_proj.weight": "model-00033-of-00101.safetensors", + "model.layers.31.mlp.experts.105.down_proj.weight": "model-00033-of-00101.safetensors", + "model.layers.31.mlp.experts.106.gate_proj.weight": "model-00033-of-00101.safetensors", + "model.layers.31.mlp.experts.106.up_proj.weight": "model-00033-of-00101.safetensors", + "model.layers.31.mlp.experts.106.down_proj.weight": "model-00033-of-00101.safetensors", + "model.layers.31.mlp.experts.107.gate_proj.weight": "model-00033-of-00101.safetensors", + "model.layers.31.mlp.experts.107.up_proj.weight": "model-00033-of-00101.safetensors", + "model.layers.31.mlp.experts.107.down_proj.weight": "model-00033-of-00101.safetensors", + "model.layers.31.mlp.experts.108.gate_proj.weight": "model-00033-of-00101.safetensors", + "model.layers.31.mlp.experts.108.up_proj.weight": "model-00033-of-00101.safetensors", + "model.layers.31.mlp.experts.108.down_proj.weight": "model-00033-of-00101.safetensors", + "model.layers.31.mlp.experts.109.gate_proj.weight": "model-00033-of-00101.safetensors", + "model.layers.31.mlp.experts.109.up_proj.weight": "model-00033-of-00101.safetensors", + "model.layers.31.mlp.experts.109.down_proj.weight": "model-00033-of-00101.safetensors", + "model.layers.31.mlp.experts.110.gate_proj.weight": "model-00033-of-00101.safetensors", + "model.layers.31.mlp.experts.110.up_proj.weight": "model-00033-of-00101.safetensors", + "model.layers.31.mlp.experts.110.down_proj.weight": "model-00033-of-00101.safetensors", + "model.layers.31.mlp.experts.111.gate_proj.weight": "model-00033-of-00101.safetensors", + "model.layers.31.mlp.experts.111.up_proj.weight": "model-00033-of-00101.safetensors", + "model.layers.31.mlp.experts.111.down_proj.weight": "model-00033-of-00101.safetensors", + "model.layers.31.mlp.experts.112.gate_proj.weight": "model-00033-of-00101.safetensors", + "model.layers.31.mlp.experts.112.up_proj.weight": "model-00033-of-00101.safetensors", + "model.layers.31.mlp.experts.112.down_proj.weight": "model-00033-of-00101.safetensors", + "model.layers.31.mlp.experts.113.gate_proj.weight": "model-00033-of-00101.safetensors", + "model.layers.31.mlp.experts.113.up_proj.weight": "model-00033-of-00101.safetensors", + "model.layers.31.mlp.experts.113.down_proj.weight": "model-00033-of-00101.safetensors", + "model.layers.31.mlp.experts.114.gate_proj.weight": "model-00033-of-00101.safetensors", + "model.layers.31.mlp.experts.114.up_proj.weight": "model-00033-of-00101.safetensors", + "model.layers.31.mlp.experts.114.down_proj.weight": "model-00034-of-00101.safetensors", + "model.layers.31.mlp.experts.115.gate_proj.weight": "model-00034-of-00101.safetensors", + "model.layers.31.mlp.experts.115.up_proj.weight": "model-00034-of-00101.safetensors", + "model.layers.31.mlp.experts.115.down_proj.weight": "model-00034-of-00101.safetensors", + "model.layers.31.mlp.experts.116.gate_proj.weight": "model-00034-of-00101.safetensors", + "model.layers.31.mlp.experts.116.up_proj.weight": "model-00034-of-00101.safetensors", + "model.layers.31.mlp.experts.116.down_proj.weight": "model-00034-of-00101.safetensors", + "model.layers.31.mlp.experts.117.gate_proj.weight": "model-00034-of-00101.safetensors", + "model.layers.31.mlp.experts.117.up_proj.weight": "model-00034-of-00101.safetensors", + "model.layers.31.mlp.experts.117.down_proj.weight": "model-00034-of-00101.safetensors", + "model.layers.31.mlp.experts.118.gate_proj.weight": "model-00034-of-00101.safetensors", + "model.layers.31.mlp.experts.118.up_proj.weight": "model-00034-of-00101.safetensors", + "model.layers.31.mlp.experts.118.down_proj.weight": "model-00034-of-00101.safetensors", + "model.layers.31.mlp.experts.119.gate_proj.weight": "model-00034-of-00101.safetensors", + "model.layers.31.mlp.experts.119.up_proj.weight": "model-00034-of-00101.safetensors", + "model.layers.31.mlp.experts.119.down_proj.weight": "model-00034-of-00101.safetensors", + "model.layers.31.mlp.gate.weight": "model-00034-of-00101.safetensors", + "model.layers.31.mlp.gate.e_score_correction_bias": "model-00034-of-00101.safetensors", + "model.layers.31.mlp.shared_experts.gate_proj.weight": "model-00034-of-00101.safetensors", + "model.layers.31.mlp.shared_experts.up_proj.weight": "model-00034-of-00101.safetensors", + "model.layers.31.mlp.shared_experts.down_proj.weight": "model-00034-of-00101.safetensors", + "model.layers.31.input_layernorm.weight": "model-00034-of-00101.safetensors", + "model.layers.31.post_attention_layernorm.weight": "model-00034-of-00101.safetensors", + "model.layers.32.self_attn.q_proj.weight": "model-00034-of-00101.safetensors", + "model.layers.32.self_attn.q_proj.bias": "model-00034-of-00101.safetensors", + "model.layers.32.self_attn.k_proj.weight": "model-00034-of-00101.safetensors", + "model.layers.32.self_attn.k_proj.bias": "model-00034-of-00101.safetensors", + "model.layers.32.self_attn.v_proj.weight": "model-00034-of-00101.safetensors", + "model.layers.32.self_attn.v_proj.bias": "model-00034-of-00101.safetensors", + "model.layers.32.self_attn.o_proj.weight": "model-00034-of-00101.safetensors", + "model.layers.32.self_attn.q_norm.weight": "model-00034-of-00101.safetensors", + "model.layers.32.self_attn.k_norm.weight": "model-00034-of-00101.safetensors", + "model.layers.32.mlp.experts.0.gate_proj.weight": "model-00034-of-00101.safetensors", + "model.layers.32.mlp.experts.0.up_proj.weight": "model-00034-of-00101.safetensors", + "model.layers.32.mlp.experts.0.down_proj.weight": "model-00034-of-00101.safetensors", + "model.layers.32.mlp.experts.1.gate_proj.weight": "model-00034-of-00101.safetensors", + "model.layers.32.mlp.experts.1.up_proj.weight": "model-00034-of-00101.safetensors", + "model.layers.32.mlp.experts.1.down_proj.weight": "model-00034-of-00101.safetensors", + "model.layers.32.mlp.experts.2.gate_proj.weight": "model-00034-of-00101.safetensors", + "model.layers.32.mlp.experts.2.up_proj.weight": "model-00034-of-00101.safetensors", + "model.layers.32.mlp.experts.2.down_proj.weight": "model-00034-of-00101.safetensors", + "model.layers.32.mlp.experts.3.gate_proj.weight": "model-00034-of-00101.safetensors", + "model.layers.32.mlp.experts.3.up_proj.weight": "model-00034-of-00101.safetensors", + "model.layers.32.mlp.experts.3.down_proj.weight": "model-00034-of-00101.safetensors", + "model.layers.32.mlp.experts.4.gate_proj.weight": "model-00034-of-00101.safetensors", + "model.layers.32.mlp.experts.4.up_proj.weight": "model-00034-of-00101.safetensors", + "model.layers.32.mlp.experts.4.down_proj.weight": "model-00034-of-00101.safetensors", + "model.layers.32.mlp.experts.5.gate_proj.weight": "model-00034-of-00101.safetensors", + "model.layers.32.mlp.experts.5.up_proj.weight": "model-00034-of-00101.safetensors", + "model.layers.32.mlp.experts.5.down_proj.weight": "model-00034-of-00101.safetensors", + "model.layers.32.mlp.experts.6.gate_proj.weight": "model-00034-of-00101.safetensors", + "model.layers.32.mlp.experts.6.up_proj.weight": "model-00034-of-00101.safetensors", + "model.layers.32.mlp.experts.6.down_proj.weight": "model-00034-of-00101.safetensors", + "model.layers.32.mlp.experts.7.gate_proj.weight": "model-00034-of-00101.safetensors", + "model.layers.32.mlp.experts.7.up_proj.weight": "model-00034-of-00101.safetensors", + "model.layers.32.mlp.experts.7.down_proj.weight": "model-00034-of-00101.safetensors", + "model.layers.32.mlp.experts.8.gate_proj.weight": "model-00034-of-00101.safetensors", + "model.layers.32.mlp.experts.8.up_proj.weight": "model-00034-of-00101.safetensors", + "model.layers.32.mlp.experts.8.down_proj.weight": "model-00034-of-00101.safetensors", + "model.layers.32.mlp.experts.9.gate_proj.weight": "model-00034-of-00101.safetensors", + "model.layers.32.mlp.experts.9.up_proj.weight": "model-00034-of-00101.safetensors", + "model.layers.32.mlp.experts.9.down_proj.weight": "model-00034-of-00101.safetensors", + "model.layers.32.mlp.experts.10.gate_proj.weight": "model-00034-of-00101.safetensors", + "model.layers.32.mlp.experts.10.up_proj.weight": "model-00034-of-00101.safetensors", + "model.layers.32.mlp.experts.10.down_proj.weight": "model-00034-of-00101.safetensors", + "model.layers.32.mlp.experts.11.gate_proj.weight": "model-00034-of-00101.safetensors", + "model.layers.32.mlp.experts.11.up_proj.weight": "model-00034-of-00101.safetensors", + "model.layers.32.mlp.experts.11.down_proj.weight": "model-00034-of-00101.safetensors", + "model.layers.32.mlp.experts.12.gate_proj.weight": "model-00034-of-00101.safetensors", + "model.layers.32.mlp.experts.12.up_proj.weight": "model-00034-of-00101.safetensors", + "model.layers.32.mlp.experts.12.down_proj.weight": "model-00034-of-00101.safetensors", + "model.layers.32.mlp.experts.13.gate_proj.weight": "model-00034-of-00101.safetensors", + "model.layers.32.mlp.experts.13.up_proj.weight": "model-00034-of-00101.safetensors", + "model.layers.32.mlp.experts.13.down_proj.weight": "model-00034-of-00101.safetensors", + "model.layers.32.mlp.experts.14.gate_proj.weight": "model-00034-of-00101.safetensors", + "model.layers.32.mlp.experts.14.up_proj.weight": "model-00034-of-00101.safetensors", + "model.layers.32.mlp.experts.14.down_proj.weight": "model-00034-of-00101.safetensors", + "model.layers.32.mlp.experts.15.gate_proj.weight": "model-00034-of-00101.safetensors", + "model.layers.32.mlp.experts.15.up_proj.weight": "model-00034-of-00101.safetensors", + "model.layers.32.mlp.experts.15.down_proj.weight": "model-00034-of-00101.safetensors", + "model.layers.32.mlp.experts.16.gate_proj.weight": "model-00034-of-00101.safetensors", + "model.layers.32.mlp.experts.16.up_proj.weight": "model-00034-of-00101.safetensors", + "model.layers.32.mlp.experts.16.down_proj.weight": "model-00034-of-00101.safetensors", + "model.layers.32.mlp.experts.17.gate_proj.weight": "model-00034-of-00101.safetensors", + "model.layers.32.mlp.experts.17.up_proj.weight": "model-00034-of-00101.safetensors", + "model.layers.32.mlp.experts.17.down_proj.weight": "model-00034-of-00101.safetensors", + "model.layers.32.mlp.experts.18.gate_proj.weight": "model-00034-of-00101.safetensors", + "model.layers.32.mlp.experts.18.up_proj.weight": "model-00034-of-00101.safetensors", + "model.layers.32.mlp.experts.18.down_proj.weight": "model-00034-of-00101.safetensors", + "model.layers.32.mlp.experts.19.gate_proj.weight": "model-00034-of-00101.safetensors", + "model.layers.32.mlp.experts.19.up_proj.weight": "model-00034-of-00101.safetensors", + "model.layers.32.mlp.experts.19.down_proj.weight": "model-00034-of-00101.safetensors", + "model.layers.32.mlp.experts.20.gate_proj.weight": "model-00034-of-00101.safetensors", + "model.layers.32.mlp.experts.20.up_proj.weight": "model-00034-of-00101.safetensors", + "model.layers.32.mlp.experts.20.down_proj.weight": "model-00034-of-00101.safetensors", + "model.layers.32.mlp.experts.21.gate_proj.weight": "model-00034-of-00101.safetensors", + "model.layers.32.mlp.experts.21.up_proj.weight": "model-00034-of-00101.safetensors", + "model.layers.32.mlp.experts.21.down_proj.weight": "model-00034-of-00101.safetensors", + "model.layers.32.mlp.experts.22.gate_proj.weight": "model-00034-of-00101.safetensors", + "model.layers.32.mlp.experts.22.up_proj.weight": "model-00034-of-00101.safetensors", + "model.layers.32.mlp.experts.22.down_proj.weight": "model-00034-of-00101.safetensors", + "model.layers.32.mlp.experts.23.gate_proj.weight": "model-00034-of-00101.safetensors", + "model.layers.32.mlp.experts.23.up_proj.weight": "model-00034-of-00101.safetensors", + "model.layers.32.mlp.experts.23.down_proj.weight": "model-00034-of-00101.safetensors", + "model.layers.32.mlp.experts.24.gate_proj.weight": "model-00034-of-00101.safetensors", + "model.layers.32.mlp.experts.24.up_proj.weight": "model-00034-of-00101.safetensors", + "model.layers.32.mlp.experts.24.down_proj.weight": "model-00034-of-00101.safetensors", + "model.layers.32.mlp.experts.25.gate_proj.weight": "model-00034-of-00101.safetensors", + "model.layers.32.mlp.experts.25.up_proj.weight": "model-00034-of-00101.safetensors", + "model.layers.32.mlp.experts.25.down_proj.weight": "model-00034-of-00101.safetensors", + "model.layers.32.mlp.experts.26.gate_proj.weight": "model-00034-of-00101.safetensors", + "model.layers.32.mlp.experts.26.up_proj.weight": "model-00034-of-00101.safetensors", + "model.layers.32.mlp.experts.26.down_proj.weight": "model-00034-of-00101.safetensors", + "model.layers.32.mlp.experts.27.gate_proj.weight": "model-00034-of-00101.safetensors", + "model.layers.32.mlp.experts.27.up_proj.weight": "model-00034-of-00101.safetensors", + "model.layers.32.mlp.experts.27.down_proj.weight": "model-00034-of-00101.safetensors", + "model.layers.32.mlp.experts.28.gate_proj.weight": "model-00034-of-00101.safetensors", + "model.layers.32.mlp.experts.28.up_proj.weight": "model-00034-of-00101.safetensors", + "model.layers.32.mlp.experts.28.down_proj.weight": "model-00034-of-00101.safetensors", + "model.layers.32.mlp.experts.29.gate_proj.weight": "model-00034-of-00101.safetensors", + "model.layers.32.mlp.experts.29.up_proj.weight": "model-00034-of-00101.safetensors", + "model.layers.32.mlp.experts.29.down_proj.weight": "model-00034-of-00101.safetensors", + "model.layers.32.mlp.experts.30.gate_proj.weight": "model-00034-of-00101.safetensors", + "model.layers.32.mlp.experts.30.up_proj.weight": "model-00034-of-00101.safetensors", + "model.layers.32.mlp.experts.30.down_proj.weight": "model-00034-of-00101.safetensors", + "model.layers.32.mlp.experts.31.gate_proj.weight": "model-00034-of-00101.safetensors", + "model.layers.32.mlp.experts.31.up_proj.weight": "model-00034-of-00101.safetensors", + "model.layers.32.mlp.experts.31.down_proj.weight": "model-00034-of-00101.safetensors", + "model.layers.32.mlp.experts.32.gate_proj.weight": "model-00034-of-00101.safetensors", + "model.layers.32.mlp.experts.32.up_proj.weight": "model-00034-of-00101.safetensors", + "model.layers.32.mlp.experts.32.down_proj.weight": "model-00034-of-00101.safetensors", + "model.layers.32.mlp.experts.33.gate_proj.weight": "model-00034-of-00101.safetensors", + "model.layers.32.mlp.experts.33.up_proj.weight": "model-00034-of-00101.safetensors", + "model.layers.32.mlp.experts.33.down_proj.weight": "model-00034-of-00101.safetensors", + "model.layers.32.mlp.experts.34.gate_proj.weight": "model-00034-of-00101.safetensors", + "model.layers.32.mlp.experts.34.up_proj.weight": "model-00034-of-00101.safetensors", + "model.layers.32.mlp.experts.34.down_proj.weight": "model-00034-of-00101.safetensors", + "model.layers.32.mlp.experts.35.gate_proj.weight": "model-00034-of-00101.safetensors", + "model.layers.32.mlp.experts.35.up_proj.weight": "model-00034-of-00101.safetensors", + "model.layers.32.mlp.experts.35.down_proj.weight": "model-00034-of-00101.safetensors", + "model.layers.32.mlp.experts.36.gate_proj.weight": "model-00034-of-00101.safetensors", + "model.layers.32.mlp.experts.36.up_proj.weight": "model-00034-of-00101.safetensors", + "model.layers.32.mlp.experts.36.down_proj.weight": "model-00034-of-00101.safetensors", + "model.layers.32.mlp.experts.37.gate_proj.weight": "model-00034-of-00101.safetensors", + "model.layers.32.mlp.experts.37.up_proj.weight": "model-00034-of-00101.safetensors", + "model.layers.32.mlp.experts.37.down_proj.weight": "model-00034-of-00101.safetensors", + "model.layers.32.mlp.experts.38.gate_proj.weight": "model-00034-of-00101.safetensors", + "model.layers.32.mlp.experts.38.up_proj.weight": "model-00034-of-00101.safetensors", + "model.layers.32.mlp.experts.38.down_proj.weight": "model-00034-of-00101.safetensors", + "model.layers.32.mlp.experts.39.gate_proj.weight": "model-00034-of-00101.safetensors", + "model.layers.32.mlp.experts.39.up_proj.weight": "model-00034-of-00101.safetensors", + "model.layers.32.mlp.experts.39.down_proj.weight": "model-00034-of-00101.safetensors", + "model.layers.32.mlp.experts.40.gate_proj.weight": "model-00034-of-00101.safetensors", + "model.layers.32.mlp.experts.40.up_proj.weight": "model-00034-of-00101.safetensors", + "model.layers.32.mlp.experts.40.down_proj.weight": "model-00034-of-00101.safetensors", + "model.layers.32.mlp.experts.41.gate_proj.weight": "model-00034-of-00101.safetensors", + "model.layers.32.mlp.experts.41.up_proj.weight": "model-00034-of-00101.safetensors", + "model.layers.32.mlp.experts.41.down_proj.weight": "model-00034-of-00101.safetensors", + "model.layers.32.mlp.experts.42.gate_proj.weight": "model-00034-of-00101.safetensors", + "model.layers.32.mlp.experts.42.up_proj.weight": "model-00034-of-00101.safetensors", + "model.layers.32.mlp.experts.42.down_proj.weight": "model-00034-of-00101.safetensors", + "model.layers.32.mlp.experts.43.gate_proj.weight": "model-00034-of-00101.safetensors", + "model.layers.32.mlp.experts.43.up_proj.weight": "model-00034-of-00101.safetensors", + "model.layers.32.mlp.experts.43.down_proj.weight": "model-00034-of-00101.safetensors", + "model.layers.32.mlp.experts.44.gate_proj.weight": "model-00034-of-00101.safetensors", + "model.layers.32.mlp.experts.44.up_proj.weight": "model-00034-of-00101.safetensors", + "model.layers.32.mlp.experts.44.down_proj.weight": "model-00034-of-00101.safetensors", + "model.layers.32.mlp.experts.45.gate_proj.weight": "model-00034-of-00101.safetensors", + "model.layers.32.mlp.experts.45.up_proj.weight": "model-00034-of-00101.safetensors", + "model.layers.32.mlp.experts.45.down_proj.weight": "model-00034-of-00101.safetensors", + "model.layers.32.mlp.experts.46.gate_proj.weight": "model-00034-of-00101.safetensors", + "model.layers.32.mlp.experts.46.up_proj.weight": "model-00034-of-00101.safetensors", + "model.layers.32.mlp.experts.46.down_proj.weight": "model-00034-of-00101.safetensors", + "model.layers.32.mlp.experts.47.gate_proj.weight": "model-00034-of-00101.safetensors", + "model.layers.32.mlp.experts.47.up_proj.weight": "model-00034-of-00101.safetensors", + "model.layers.32.mlp.experts.47.down_proj.weight": "model-00034-of-00101.safetensors", + "model.layers.32.mlp.experts.48.gate_proj.weight": "model-00034-of-00101.safetensors", + "model.layers.32.mlp.experts.48.up_proj.weight": "model-00034-of-00101.safetensors", + "model.layers.32.mlp.experts.48.down_proj.weight": "model-00034-of-00101.safetensors", + "model.layers.32.mlp.experts.49.gate_proj.weight": "model-00034-of-00101.safetensors", + "model.layers.32.mlp.experts.49.up_proj.weight": "model-00034-of-00101.safetensors", + "model.layers.32.mlp.experts.49.down_proj.weight": "model-00034-of-00101.safetensors", + "model.layers.32.mlp.experts.50.gate_proj.weight": "model-00034-of-00101.safetensors", + "model.layers.32.mlp.experts.50.up_proj.weight": "model-00034-of-00101.safetensors", + "model.layers.32.mlp.experts.50.down_proj.weight": "model-00034-of-00101.safetensors", + "model.layers.32.mlp.experts.51.gate_proj.weight": "model-00034-of-00101.safetensors", + "model.layers.32.mlp.experts.51.up_proj.weight": "model-00034-of-00101.safetensors", + "model.layers.32.mlp.experts.51.down_proj.weight": "model-00034-of-00101.safetensors", + "model.layers.32.mlp.experts.52.gate_proj.weight": "model-00034-of-00101.safetensors", + "model.layers.32.mlp.experts.52.up_proj.weight": "model-00034-of-00101.safetensors", + "model.layers.32.mlp.experts.52.down_proj.weight": "model-00034-of-00101.safetensors", + "model.layers.32.mlp.experts.53.gate_proj.weight": "model-00034-of-00101.safetensors", + "model.layers.32.mlp.experts.53.up_proj.weight": "model-00034-of-00101.safetensors", + "model.layers.32.mlp.experts.53.down_proj.weight": "model-00034-of-00101.safetensors", + "model.layers.32.mlp.experts.54.gate_proj.weight": "model-00034-of-00101.safetensors", + "model.layers.32.mlp.experts.54.up_proj.weight": "model-00034-of-00101.safetensors", + "model.layers.32.mlp.experts.54.down_proj.weight": "model-00034-of-00101.safetensors", + "model.layers.32.mlp.experts.55.gate_proj.weight": "model-00034-of-00101.safetensors", + "model.layers.32.mlp.experts.55.up_proj.weight": "model-00034-of-00101.safetensors", + "model.layers.32.mlp.experts.55.down_proj.weight": "model-00034-of-00101.safetensors", + "model.layers.32.mlp.experts.56.gate_proj.weight": "model-00034-of-00101.safetensors", + "model.layers.32.mlp.experts.56.up_proj.weight": "model-00034-of-00101.safetensors", + "model.layers.32.mlp.experts.56.down_proj.weight": "model-00034-of-00101.safetensors", + "model.layers.32.mlp.experts.57.gate_proj.weight": "model-00034-of-00101.safetensors", + "model.layers.32.mlp.experts.57.up_proj.weight": "model-00034-of-00101.safetensors", + "model.layers.32.mlp.experts.57.down_proj.weight": "model-00034-of-00101.safetensors", + "model.layers.32.mlp.experts.58.gate_proj.weight": "model-00034-of-00101.safetensors", + "model.layers.32.mlp.experts.58.up_proj.weight": "model-00034-of-00101.safetensors", + "model.layers.32.mlp.experts.58.down_proj.weight": "model-00034-of-00101.safetensors", + "model.layers.32.mlp.experts.59.gate_proj.weight": "model-00034-of-00101.safetensors", + "model.layers.32.mlp.experts.59.up_proj.weight": "model-00034-of-00101.safetensors", + "model.layers.32.mlp.experts.59.down_proj.weight": "model-00034-of-00101.safetensors", + "model.layers.32.mlp.experts.60.gate_proj.weight": "model-00034-of-00101.safetensors", + "model.layers.32.mlp.experts.60.up_proj.weight": "model-00034-of-00101.safetensors", + "model.layers.32.mlp.experts.60.down_proj.weight": "model-00034-of-00101.safetensors", + "model.layers.32.mlp.experts.61.gate_proj.weight": "model-00034-of-00101.safetensors", + "model.layers.32.mlp.experts.61.up_proj.weight": "model-00034-of-00101.safetensors", + "model.layers.32.mlp.experts.61.down_proj.weight": "model-00034-of-00101.safetensors", + "model.layers.32.mlp.experts.62.gate_proj.weight": "model-00034-of-00101.safetensors", + "model.layers.32.mlp.experts.62.up_proj.weight": "model-00034-of-00101.safetensors", + "model.layers.32.mlp.experts.62.down_proj.weight": "model-00034-of-00101.safetensors", + "model.layers.32.mlp.experts.63.gate_proj.weight": "model-00034-of-00101.safetensors", + "model.layers.32.mlp.experts.63.up_proj.weight": "model-00034-of-00101.safetensors", + "model.layers.32.mlp.experts.63.down_proj.weight": "model-00034-of-00101.safetensors", + "model.layers.32.mlp.experts.64.gate_proj.weight": "model-00034-of-00101.safetensors", + "model.layers.32.mlp.experts.64.up_proj.weight": "model-00034-of-00101.safetensors", + "model.layers.32.mlp.experts.64.down_proj.weight": "model-00034-of-00101.safetensors", + "model.layers.32.mlp.experts.65.gate_proj.weight": "model-00034-of-00101.safetensors", + "model.layers.32.mlp.experts.65.up_proj.weight": "model-00034-of-00101.safetensors", + "model.layers.32.mlp.experts.65.down_proj.weight": "model-00034-of-00101.safetensors", + "model.layers.32.mlp.experts.66.gate_proj.weight": "model-00034-of-00101.safetensors", + "model.layers.32.mlp.experts.66.up_proj.weight": "model-00034-of-00101.safetensors", + "model.layers.32.mlp.experts.66.down_proj.weight": "model-00034-of-00101.safetensors", + "model.layers.32.mlp.experts.67.gate_proj.weight": "model-00034-of-00101.safetensors", + "model.layers.32.mlp.experts.67.up_proj.weight": "model-00034-of-00101.safetensors", + "model.layers.32.mlp.experts.67.down_proj.weight": "model-00034-of-00101.safetensors", + "model.layers.32.mlp.experts.68.gate_proj.weight": "model-00034-of-00101.safetensors", + "model.layers.32.mlp.experts.68.up_proj.weight": "model-00034-of-00101.safetensors", + "model.layers.32.mlp.experts.68.down_proj.weight": "model-00034-of-00101.safetensors", + "model.layers.32.mlp.experts.69.gate_proj.weight": "model-00034-of-00101.safetensors", + "model.layers.32.mlp.experts.69.up_proj.weight": "model-00034-of-00101.safetensors", + "model.layers.32.mlp.experts.69.down_proj.weight": "model-00034-of-00101.safetensors", + "model.layers.32.mlp.experts.70.gate_proj.weight": "model-00034-of-00101.safetensors", + "model.layers.32.mlp.experts.70.up_proj.weight": "model-00034-of-00101.safetensors", + "model.layers.32.mlp.experts.70.down_proj.weight": "model-00034-of-00101.safetensors", + "model.layers.32.mlp.experts.71.gate_proj.weight": "model-00034-of-00101.safetensors", + "model.layers.32.mlp.experts.71.up_proj.weight": "model-00034-of-00101.safetensors", + "model.layers.32.mlp.experts.71.down_proj.weight": "model-00034-of-00101.safetensors", + "model.layers.32.mlp.experts.72.gate_proj.weight": "model-00034-of-00101.safetensors", + "model.layers.32.mlp.experts.72.up_proj.weight": "model-00034-of-00101.safetensors", + "model.layers.32.mlp.experts.72.down_proj.weight": "model-00034-of-00101.safetensors", + "model.layers.32.mlp.experts.73.gate_proj.weight": "model-00034-of-00101.safetensors", + "model.layers.32.mlp.experts.73.up_proj.weight": "model-00034-of-00101.safetensors", + "model.layers.32.mlp.experts.73.down_proj.weight": "model-00034-of-00101.safetensors", + "model.layers.32.mlp.experts.74.gate_proj.weight": "model-00034-of-00101.safetensors", + "model.layers.32.mlp.experts.74.up_proj.weight": "model-00034-of-00101.safetensors", + "model.layers.32.mlp.experts.74.down_proj.weight": "model-00034-of-00101.safetensors", + "model.layers.32.mlp.experts.75.gate_proj.weight": "model-00034-of-00101.safetensors", + "model.layers.32.mlp.experts.75.up_proj.weight": "model-00034-of-00101.safetensors", + "model.layers.32.mlp.experts.75.down_proj.weight": "model-00034-of-00101.safetensors", + "model.layers.32.mlp.experts.76.gate_proj.weight": "model-00034-of-00101.safetensors", + "model.layers.32.mlp.experts.76.up_proj.weight": "model-00034-of-00101.safetensors", + "model.layers.32.mlp.experts.76.down_proj.weight": "model-00034-of-00101.safetensors", + "model.layers.32.mlp.experts.77.gate_proj.weight": "model-00034-of-00101.safetensors", + "model.layers.32.mlp.experts.77.up_proj.weight": "model-00034-of-00101.safetensors", + "model.layers.32.mlp.experts.77.down_proj.weight": "model-00034-of-00101.safetensors", + "model.layers.32.mlp.experts.78.gate_proj.weight": "model-00034-of-00101.safetensors", + "model.layers.32.mlp.experts.78.up_proj.weight": "model-00034-of-00101.safetensors", + "model.layers.32.mlp.experts.78.down_proj.weight": "model-00034-of-00101.safetensors", + "model.layers.32.mlp.experts.79.gate_proj.weight": "model-00034-of-00101.safetensors", + "model.layers.32.mlp.experts.79.up_proj.weight": "model-00034-of-00101.safetensors", + "model.layers.32.mlp.experts.79.down_proj.weight": "model-00034-of-00101.safetensors", + "model.layers.32.mlp.experts.80.gate_proj.weight": "model-00034-of-00101.safetensors", + "model.layers.32.mlp.experts.80.up_proj.weight": "model-00034-of-00101.safetensors", + "model.layers.32.mlp.experts.80.down_proj.weight": "model-00034-of-00101.safetensors", + "model.layers.32.mlp.experts.81.gate_proj.weight": "model-00034-of-00101.safetensors", + "model.layers.32.mlp.experts.81.up_proj.weight": "model-00034-of-00101.safetensors", + "model.layers.32.mlp.experts.81.down_proj.weight": "model-00034-of-00101.safetensors", + "model.layers.32.mlp.experts.82.gate_proj.weight": "model-00034-of-00101.safetensors", + "model.layers.32.mlp.experts.82.up_proj.weight": "model-00034-of-00101.safetensors", + "model.layers.32.mlp.experts.82.down_proj.weight": "model-00034-of-00101.safetensors", + "model.layers.32.mlp.experts.83.gate_proj.weight": "model-00034-of-00101.safetensors", + "model.layers.32.mlp.experts.83.up_proj.weight": "model-00034-of-00101.safetensors", + "model.layers.32.mlp.experts.83.down_proj.weight": "model-00034-of-00101.safetensors", + "model.layers.32.mlp.experts.84.gate_proj.weight": "model-00034-of-00101.safetensors", + "model.layers.32.mlp.experts.84.up_proj.weight": "model-00034-of-00101.safetensors", + "model.layers.32.mlp.experts.84.down_proj.weight": "model-00034-of-00101.safetensors", + "model.layers.32.mlp.experts.85.gate_proj.weight": "model-00034-of-00101.safetensors", + "model.layers.32.mlp.experts.85.up_proj.weight": "model-00034-of-00101.safetensors", + "model.layers.32.mlp.experts.85.down_proj.weight": "model-00034-of-00101.safetensors", + "model.layers.32.mlp.experts.86.gate_proj.weight": "model-00034-of-00101.safetensors", + "model.layers.32.mlp.experts.86.up_proj.weight": "model-00034-of-00101.safetensors", + "model.layers.32.mlp.experts.86.down_proj.weight": "model-00034-of-00101.safetensors", + "model.layers.32.mlp.experts.87.gate_proj.weight": "model-00034-of-00101.safetensors", + "model.layers.32.mlp.experts.87.up_proj.weight": "model-00034-of-00101.safetensors", + "model.layers.32.mlp.experts.87.down_proj.weight": "model-00034-of-00101.safetensors", + "model.layers.32.mlp.experts.88.gate_proj.weight": "model-00034-of-00101.safetensors", + "model.layers.32.mlp.experts.88.up_proj.weight": "model-00034-of-00101.safetensors", + "model.layers.32.mlp.experts.88.down_proj.weight": "model-00034-of-00101.safetensors", + "model.layers.32.mlp.experts.89.gate_proj.weight": "model-00034-of-00101.safetensors", + "model.layers.32.mlp.experts.89.up_proj.weight": "model-00034-of-00101.safetensors", + "model.layers.32.mlp.experts.89.down_proj.weight": "model-00034-of-00101.safetensors", + "model.layers.32.mlp.experts.90.gate_proj.weight": "model-00034-of-00101.safetensors", + "model.layers.32.mlp.experts.90.up_proj.weight": "model-00034-of-00101.safetensors", + "model.layers.32.mlp.experts.90.down_proj.weight": "model-00034-of-00101.safetensors", + "model.layers.32.mlp.experts.91.gate_proj.weight": "model-00034-of-00101.safetensors", + "model.layers.32.mlp.experts.91.up_proj.weight": "model-00034-of-00101.safetensors", + "model.layers.32.mlp.experts.91.down_proj.weight": "model-00034-of-00101.safetensors", + "model.layers.32.mlp.experts.92.gate_proj.weight": "model-00034-of-00101.safetensors", + "model.layers.32.mlp.experts.92.up_proj.weight": "model-00034-of-00101.safetensors", + "model.layers.32.mlp.experts.92.down_proj.weight": "model-00034-of-00101.safetensors", + "model.layers.32.mlp.experts.93.gate_proj.weight": "model-00034-of-00101.safetensors", + "model.layers.32.mlp.experts.93.up_proj.weight": "model-00034-of-00101.safetensors", + "model.layers.32.mlp.experts.93.down_proj.weight": "model-00034-of-00101.safetensors", + "model.layers.32.mlp.experts.94.gate_proj.weight": "model-00034-of-00101.safetensors", + "model.layers.32.mlp.experts.94.up_proj.weight": "model-00034-of-00101.safetensors", + "model.layers.32.mlp.experts.94.down_proj.weight": "model-00034-of-00101.safetensors", + "model.layers.32.mlp.experts.95.gate_proj.weight": "model-00034-of-00101.safetensors", + "model.layers.32.mlp.experts.95.up_proj.weight": "model-00034-of-00101.safetensors", + "model.layers.32.mlp.experts.95.down_proj.weight": "model-00034-of-00101.safetensors", + "model.layers.32.mlp.experts.96.gate_proj.weight": "model-00034-of-00101.safetensors", + "model.layers.32.mlp.experts.96.up_proj.weight": "model-00034-of-00101.safetensors", + "model.layers.32.mlp.experts.96.down_proj.weight": "model-00034-of-00101.safetensors", + "model.layers.32.mlp.experts.97.gate_proj.weight": "model-00034-of-00101.safetensors", + "model.layers.32.mlp.experts.97.up_proj.weight": "model-00034-of-00101.safetensors", + "model.layers.32.mlp.experts.97.down_proj.weight": "model-00034-of-00101.safetensors", + "model.layers.32.mlp.experts.98.gate_proj.weight": "model-00034-of-00101.safetensors", + "model.layers.32.mlp.experts.98.up_proj.weight": "model-00034-of-00101.safetensors", + "model.layers.32.mlp.experts.98.down_proj.weight": "model-00034-of-00101.safetensors", + "model.layers.32.mlp.experts.99.gate_proj.weight": "model-00034-of-00101.safetensors", + "model.layers.32.mlp.experts.99.up_proj.weight": "model-00034-of-00101.safetensors", + "model.layers.32.mlp.experts.99.down_proj.weight": "model-00034-of-00101.safetensors", + "model.layers.32.mlp.experts.100.gate_proj.weight": "model-00034-of-00101.safetensors", + "model.layers.32.mlp.experts.100.up_proj.weight": "model-00034-of-00101.safetensors", + "model.layers.32.mlp.experts.100.down_proj.weight": "model-00034-of-00101.safetensors", + "model.layers.32.mlp.experts.101.gate_proj.weight": "model-00034-of-00101.safetensors", + "model.layers.32.mlp.experts.101.up_proj.weight": "model-00035-of-00101.safetensors", + "model.layers.32.mlp.experts.101.down_proj.weight": "model-00035-of-00101.safetensors", + "model.layers.32.mlp.experts.102.gate_proj.weight": "model-00035-of-00101.safetensors", + "model.layers.32.mlp.experts.102.up_proj.weight": "model-00035-of-00101.safetensors", + "model.layers.32.mlp.experts.102.down_proj.weight": "model-00035-of-00101.safetensors", + "model.layers.32.mlp.experts.103.gate_proj.weight": "model-00035-of-00101.safetensors", + "model.layers.32.mlp.experts.103.up_proj.weight": "model-00035-of-00101.safetensors", + "model.layers.32.mlp.experts.103.down_proj.weight": "model-00035-of-00101.safetensors", + "model.layers.32.mlp.experts.104.gate_proj.weight": "model-00035-of-00101.safetensors", + "model.layers.32.mlp.experts.104.up_proj.weight": "model-00035-of-00101.safetensors", + "model.layers.32.mlp.experts.104.down_proj.weight": "model-00035-of-00101.safetensors", + "model.layers.32.mlp.experts.105.gate_proj.weight": "model-00035-of-00101.safetensors", + "model.layers.32.mlp.experts.105.up_proj.weight": "model-00035-of-00101.safetensors", + "model.layers.32.mlp.experts.105.down_proj.weight": "model-00035-of-00101.safetensors", + "model.layers.32.mlp.experts.106.gate_proj.weight": "model-00035-of-00101.safetensors", + "model.layers.32.mlp.experts.106.up_proj.weight": "model-00035-of-00101.safetensors", + "model.layers.32.mlp.experts.106.down_proj.weight": "model-00035-of-00101.safetensors", + "model.layers.32.mlp.experts.107.gate_proj.weight": "model-00035-of-00101.safetensors", + "model.layers.32.mlp.experts.107.up_proj.weight": "model-00035-of-00101.safetensors", + "model.layers.32.mlp.experts.107.down_proj.weight": "model-00035-of-00101.safetensors", + "model.layers.32.mlp.experts.108.gate_proj.weight": "model-00035-of-00101.safetensors", + "model.layers.32.mlp.experts.108.up_proj.weight": "model-00035-of-00101.safetensors", + "model.layers.32.mlp.experts.108.down_proj.weight": "model-00035-of-00101.safetensors", + "model.layers.32.mlp.experts.109.gate_proj.weight": "model-00035-of-00101.safetensors", + "model.layers.32.mlp.experts.109.up_proj.weight": "model-00035-of-00101.safetensors", + "model.layers.32.mlp.experts.109.down_proj.weight": "model-00035-of-00101.safetensors", + "model.layers.32.mlp.experts.110.gate_proj.weight": "model-00035-of-00101.safetensors", + "model.layers.32.mlp.experts.110.up_proj.weight": "model-00035-of-00101.safetensors", + "model.layers.32.mlp.experts.110.down_proj.weight": "model-00035-of-00101.safetensors", + "model.layers.32.mlp.experts.111.gate_proj.weight": "model-00035-of-00101.safetensors", + "model.layers.32.mlp.experts.111.up_proj.weight": "model-00035-of-00101.safetensors", + "model.layers.32.mlp.experts.111.down_proj.weight": "model-00035-of-00101.safetensors", + "model.layers.32.mlp.experts.112.gate_proj.weight": "model-00035-of-00101.safetensors", + "model.layers.32.mlp.experts.112.up_proj.weight": "model-00035-of-00101.safetensors", + "model.layers.32.mlp.experts.112.down_proj.weight": "model-00035-of-00101.safetensors", + "model.layers.32.mlp.experts.113.gate_proj.weight": "model-00035-of-00101.safetensors", + "model.layers.32.mlp.experts.113.up_proj.weight": "model-00035-of-00101.safetensors", + "model.layers.32.mlp.experts.113.down_proj.weight": "model-00035-of-00101.safetensors", + "model.layers.32.mlp.experts.114.gate_proj.weight": "model-00035-of-00101.safetensors", + "model.layers.32.mlp.experts.114.up_proj.weight": "model-00035-of-00101.safetensors", + "model.layers.32.mlp.experts.114.down_proj.weight": "model-00035-of-00101.safetensors", + "model.layers.32.mlp.experts.115.gate_proj.weight": "model-00035-of-00101.safetensors", + "model.layers.32.mlp.experts.115.up_proj.weight": "model-00035-of-00101.safetensors", + "model.layers.32.mlp.experts.115.down_proj.weight": "model-00035-of-00101.safetensors", + "model.layers.32.mlp.experts.116.gate_proj.weight": "model-00035-of-00101.safetensors", + "model.layers.32.mlp.experts.116.up_proj.weight": "model-00035-of-00101.safetensors", + "model.layers.32.mlp.experts.116.down_proj.weight": "model-00035-of-00101.safetensors", + "model.layers.32.mlp.experts.117.gate_proj.weight": "model-00035-of-00101.safetensors", + "model.layers.32.mlp.experts.117.up_proj.weight": "model-00035-of-00101.safetensors", + "model.layers.32.mlp.experts.117.down_proj.weight": "model-00035-of-00101.safetensors", + "model.layers.32.mlp.experts.118.gate_proj.weight": "model-00035-of-00101.safetensors", + "model.layers.32.mlp.experts.118.up_proj.weight": "model-00035-of-00101.safetensors", + "model.layers.32.mlp.experts.118.down_proj.weight": "model-00035-of-00101.safetensors", + "model.layers.32.mlp.experts.119.gate_proj.weight": "model-00035-of-00101.safetensors", + "model.layers.32.mlp.experts.119.up_proj.weight": "model-00035-of-00101.safetensors", + "model.layers.32.mlp.experts.119.down_proj.weight": "model-00035-of-00101.safetensors", + "model.layers.32.mlp.gate.weight": "model-00035-of-00101.safetensors", + "model.layers.32.mlp.gate.e_score_correction_bias": "model-00035-of-00101.safetensors", + "model.layers.32.mlp.shared_experts.gate_proj.weight": "model-00035-of-00101.safetensors", + "model.layers.32.mlp.shared_experts.up_proj.weight": "model-00035-of-00101.safetensors", + "model.layers.32.mlp.shared_experts.down_proj.weight": "model-00035-of-00101.safetensors", + "model.layers.32.input_layernorm.weight": "model-00035-of-00101.safetensors", + "model.layers.32.post_attention_layernorm.weight": "model-00035-of-00101.safetensors", + "model.layers.33.self_attn.q_proj.weight": "model-00035-of-00101.safetensors", + "model.layers.33.self_attn.q_proj.bias": "model-00035-of-00101.safetensors", + "model.layers.33.self_attn.k_proj.weight": "model-00035-of-00101.safetensors", + "model.layers.33.self_attn.k_proj.bias": "model-00035-of-00101.safetensors", + "model.layers.33.self_attn.v_proj.weight": "model-00035-of-00101.safetensors", + "model.layers.33.self_attn.v_proj.bias": "model-00035-of-00101.safetensors", + "model.layers.33.self_attn.o_proj.weight": "model-00035-of-00101.safetensors", + "model.layers.33.self_attn.q_norm.weight": "model-00035-of-00101.safetensors", + "model.layers.33.self_attn.k_norm.weight": "model-00035-of-00101.safetensors", + "model.layers.33.mlp.experts.0.gate_proj.weight": "model-00035-of-00101.safetensors", + "model.layers.33.mlp.experts.0.up_proj.weight": "model-00035-of-00101.safetensors", + "model.layers.33.mlp.experts.0.down_proj.weight": "model-00035-of-00101.safetensors", + "model.layers.33.mlp.experts.1.gate_proj.weight": "model-00035-of-00101.safetensors", + "model.layers.33.mlp.experts.1.up_proj.weight": "model-00035-of-00101.safetensors", + "model.layers.33.mlp.experts.1.down_proj.weight": "model-00035-of-00101.safetensors", + "model.layers.33.mlp.experts.2.gate_proj.weight": "model-00035-of-00101.safetensors", + "model.layers.33.mlp.experts.2.up_proj.weight": "model-00035-of-00101.safetensors", + "model.layers.33.mlp.experts.2.down_proj.weight": "model-00035-of-00101.safetensors", + "model.layers.33.mlp.experts.3.gate_proj.weight": "model-00035-of-00101.safetensors", + "model.layers.33.mlp.experts.3.up_proj.weight": "model-00035-of-00101.safetensors", + "model.layers.33.mlp.experts.3.down_proj.weight": "model-00035-of-00101.safetensors", + "model.layers.33.mlp.experts.4.gate_proj.weight": "model-00035-of-00101.safetensors", + "model.layers.33.mlp.experts.4.up_proj.weight": "model-00035-of-00101.safetensors", + "model.layers.33.mlp.experts.4.down_proj.weight": "model-00035-of-00101.safetensors", + "model.layers.33.mlp.experts.5.gate_proj.weight": "model-00035-of-00101.safetensors", + "model.layers.33.mlp.experts.5.up_proj.weight": "model-00035-of-00101.safetensors", + "model.layers.33.mlp.experts.5.down_proj.weight": "model-00035-of-00101.safetensors", + "model.layers.33.mlp.experts.6.gate_proj.weight": "model-00035-of-00101.safetensors", + "model.layers.33.mlp.experts.6.up_proj.weight": "model-00035-of-00101.safetensors", + "model.layers.33.mlp.experts.6.down_proj.weight": "model-00035-of-00101.safetensors", + "model.layers.33.mlp.experts.7.gate_proj.weight": "model-00035-of-00101.safetensors", + "model.layers.33.mlp.experts.7.up_proj.weight": "model-00035-of-00101.safetensors", + "model.layers.33.mlp.experts.7.down_proj.weight": "model-00035-of-00101.safetensors", + "model.layers.33.mlp.experts.8.gate_proj.weight": "model-00035-of-00101.safetensors", + "model.layers.33.mlp.experts.8.up_proj.weight": "model-00035-of-00101.safetensors", + "model.layers.33.mlp.experts.8.down_proj.weight": "model-00035-of-00101.safetensors", + "model.layers.33.mlp.experts.9.gate_proj.weight": "model-00035-of-00101.safetensors", + "model.layers.33.mlp.experts.9.up_proj.weight": "model-00035-of-00101.safetensors", + "model.layers.33.mlp.experts.9.down_proj.weight": "model-00035-of-00101.safetensors", + "model.layers.33.mlp.experts.10.gate_proj.weight": "model-00035-of-00101.safetensors", + "model.layers.33.mlp.experts.10.up_proj.weight": "model-00035-of-00101.safetensors", + "model.layers.33.mlp.experts.10.down_proj.weight": "model-00035-of-00101.safetensors", + "model.layers.33.mlp.experts.11.gate_proj.weight": "model-00035-of-00101.safetensors", + "model.layers.33.mlp.experts.11.up_proj.weight": "model-00035-of-00101.safetensors", + "model.layers.33.mlp.experts.11.down_proj.weight": "model-00035-of-00101.safetensors", + "model.layers.33.mlp.experts.12.gate_proj.weight": "model-00035-of-00101.safetensors", + "model.layers.33.mlp.experts.12.up_proj.weight": "model-00035-of-00101.safetensors", + "model.layers.33.mlp.experts.12.down_proj.weight": "model-00035-of-00101.safetensors", + "model.layers.33.mlp.experts.13.gate_proj.weight": "model-00035-of-00101.safetensors", + "model.layers.33.mlp.experts.13.up_proj.weight": "model-00035-of-00101.safetensors", + "model.layers.33.mlp.experts.13.down_proj.weight": "model-00035-of-00101.safetensors", + "model.layers.33.mlp.experts.14.gate_proj.weight": "model-00035-of-00101.safetensors", + "model.layers.33.mlp.experts.14.up_proj.weight": "model-00035-of-00101.safetensors", + "model.layers.33.mlp.experts.14.down_proj.weight": "model-00035-of-00101.safetensors", + "model.layers.33.mlp.experts.15.gate_proj.weight": "model-00035-of-00101.safetensors", + "model.layers.33.mlp.experts.15.up_proj.weight": "model-00035-of-00101.safetensors", + "model.layers.33.mlp.experts.15.down_proj.weight": "model-00035-of-00101.safetensors", + "model.layers.33.mlp.experts.16.gate_proj.weight": "model-00035-of-00101.safetensors", + "model.layers.33.mlp.experts.16.up_proj.weight": "model-00035-of-00101.safetensors", + "model.layers.33.mlp.experts.16.down_proj.weight": "model-00035-of-00101.safetensors", + "model.layers.33.mlp.experts.17.gate_proj.weight": "model-00035-of-00101.safetensors", + "model.layers.33.mlp.experts.17.up_proj.weight": "model-00035-of-00101.safetensors", + "model.layers.33.mlp.experts.17.down_proj.weight": "model-00035-of-00101.safetensors", + "model.layers.33.mlp.experts.18.gate_proj.weight": "model-00035-of-00101.safetensors", + "model.layers.33.mlp.experts.18.up_proj.weight": "model-00035-of-00101.safetensors", + "model.layers.33.mlp.experts.18.down_proj.weight": "model-00035-of-00101.safetensors", + "model.layers.33.mlp.experts.19.gate_proj.weight": "model-00035-of-00101.safetensors", + "model.layers.33.mlp.experts.19.up_proj.weight": "model-00035-of-00101.safetensors", + "model.layers.33.mlp.experts.19.down_proj.weight": "model-00035-of-00101.safetensors", + "model.layers.33.mlp.experts.20.gate_proj.weight": "model-00035-of-00101.safetensors", + "model.layers.33.mlp.experts.20.up_proj.weight": "model-00035-of-00101.safetensors", + "model.layers.33.mlp.experts.20.down_proj.weight": "model-00035-of-00101.safetensors", + "model.layers.33.mlp.experts.21.gate_proj.weight": "model-00035-of-00101.safetensors", + "model.layers.33.mlp.experts.21.up_proj.weight": "model-00035-of-00101.safetensors", + "model.layers.33.mlp.experts.21.down_proj.weight": "model-00035-of-00101.safetensors", + "model.layers.33.mlp.experts.22.gate_proj.weight": "model-00035-of-00101.safetensors", + "model.layers.33.mlp.experts.22.up_proj.weight": "model-00035-of-00101.safetensors", + "model.layers.33.mlp.experts.22.down_proj.weight": "model-00035-of-00101.safetensors", + "model.layers.33.mlp.experts.23.gate_proj.weight": "model-00035-of-00101.safetensors", + "model.layers.33.mlp.experts.23.up_proj.weight": "model-00035-of-00101.safetensors", + "model.layers.33.mlp.experts.23.down_proj.weight": "model-00035-of-00101.safetensors", + "model.layers.33.mlp.experts.24.gate_proj.weight": "model-00035-of-00101.safetensors", + "model.layers.33.mlp.experts.24.up_proj.weight": "model-00035-of-00101.safetensors", + "model.layers.33.mlp.experts.24.down_proj.weight": "model-00035-of-00101.safetensors", + "model.layers.33.mlp.experts.25.gate_proj.weight": "model-00035-of-00101.safetensors", + "model.layers.33.mlp.experts.25.up_proj.weight": "model-00035-of-00101.safetensors", + "model.layers.33.mlp.experts.25.down_proj.weight": "model-00035-of-00101.safetensors", + "model.layers.33.mlp.experts.26.gate_proj.weight": "model-00035-of-00101.safetensors", + "model.layers.33.mlp.experts.26.up_proj.weight": "model-00035-of-00101.safetensors", + "model.layers.33.mlp.experts.26.down_proj.weight": "model-00035-of-00101.safetensors", + "model.layers.33.mlp.experts.27.gate_proj.weight": "model-00035-of-00101.safetensors", + "model.layers.33.mlp.experts.27.up_proj.weight": "model-00035-of-00101.safetensors", + "model.layers.33.mlp.experts.27.down_proj.weight": "model-00035-of-00101.safetensors", + "model.layers.33.mlp.experts.28.gate_proj.weight": "model-00035-of-00101.safetensors", + "model.layers.33.mlp.experts.28.up_proj.weight": "model-00035-of-00101.safetensors", + "model.layers.33.mlp.experts.28.down_proj.weight": "model-00035-of-00101.safetensors", + "model.layers.33.mlp.experts.29.gate_proj.weight": "model-00035-of-00101.safetensors", + "model.layers.33.mlp.experts.29.up_proj.weight": "model-00035-of-00101.safetensors", + "model.layers.33.mlp.experts.29.down_proj.weight": "model-00035-of-00101.safetensors", + "model.layers.33.mlp.experts.30.gate_proj.weight": "model-00035-of-00101.safetensors", + "model.layers.33.mlp.experts.30.up_proj.weight": "model-00035-of-00101.safetensors", + "model.layers.33.mlp.experts.30.down_proj.weight": "model-00035-of-00101.safetensors", + "model.layers.33.mlp.experts.31.gate_proj.weight": "model-00035-of-00101.safetensors", + "model.layers.33.mlp.experts.31.up_proj.weight": "model-00035-of-00101.safetensors", + "model.layers.33.mlp.experts.31.down_proj.weight": "model-00035-of-00101.safetensors", + "model.layers.33.mlp.experts.32.gate_proj.weight": "model-00035-of-00101.safetensors", + "model.layers.33.mlp.experts.32.up_proj.weight": "model-00035-of-00101.safetensors", + "model.layers.33.mlp.experts.32.down_proj.weight": "model-00035-of-00101.safetensors", + "model.layers.33.mlp.experts.33.gate_proj.weight": "model-00035-of-00101.safetensors", + "model.layers.33.mlp.experts.33.up_proj.weight": "model-00035-of-00101.safetensors", + "model.layers.33.mlp.experts.33.down_proj.weight": "model-00035-of-00101.safetensors", + "model.layers.33.mlp.experts.34.gate_proj.weight": "model-00035-of-00101.safetensors", + "model.layers.33.mlp.experts.34.up_proj.weight": "model-00035-of-00101.safetensors", + "model.layers.33.mlp.experts.34.down_proj.weight": "model-00035-of-00101.safetensors", + "model.layers.33.mlp.experts.35.gate_proj.weight": "model-00035-of-00101.safetensors", + "model.layers.33.mlp.experts.35.up_proj.weight": "model-00035-of-00101.safetensors", + "model.layers.33.mlp.experts.35.down_proj.weight": "model-00035-of-00101.safetensors", + "model.layers.33.mlp.experts.36.gate_proj.weight": "model-00035-of-00101.safetensors", + "model.layers.33.mlp.experts.36.up_proj.weight": "model-00035-of-00101.safetensors", + "model.layers.33.mlp.experts.36.down_proj.weight": "model-00035-of-00101.safetensors", + "model.layers.33.mlp.experts.37.gate_proj.weight": "model-00035-of-00101.safetensors", + "model.layers.33.mlp.experts.37.up_proj.weight": "model-00035-of-00101.safetensors", + "model.layers.33.mlp.experts.37.down_proj.weight": "model-00035-of-00101.safetensors", + "model.layers.33.mlp.experts.38.gate_proj.weight": "model-00035-of-00101.safetensors", + "model.layers.33.mlp.experts.38.up_proj.weight": "model-00035-of-00101.safetensors", + "model.layers.33.mlp.experts.38.down_proj.weight": "model-00035-of-00101.safetensors", + "model.layers.33.mlp.experts.39.gate_proj.weight": "model-00035-of-00101.safetensors", + "model.layers.33.mlp.experts.39.up_proj.weight": "model-00035-of-00101.safetensors", + "model.layers.33.mlp.experts.39.down_proj.weight": "model-00035-of-00101.safetensors", + "model.layers.33.mlp.experts.40.gate_proj.weight": "model-00035-of-00101.safetensors", + "model.layers.33.mlp.experts.40.up_proj.weight": "model-00035-of-00101.safetensors", + "model.layers.33.mlp.experts.40.down_proj.weight": "model-00035-of-00101.safetensors", + "model.layers.33.mlp.experts.41.gate_proj.weight": "model-00035-of-00101.safetensors", + "model.layers.33.mlp.experts.41.up_proj.weight": "model-00035-of-00101.safetensors", + "model.layers.33.mlp.experts.41.down_proj.weight": "model-00035-of-00101.safetensors", + "model.layers.33.mlp.experts.42.gate_proj.weight": "model-00035-of-00101.safetensors", + "model.layers.33.mlp.experts.42.up_proj.weight": "model-00035-of-00101.safetensors", + "model.layers.33.mlp.experts.42.down_proj.weight": "model-00035-of-00101.safetensors", + "model.layers.33.mlp.experts.43.gate_proj.weight": "model-00035-of-00101.safetensors", + "model.layers.33.mlp.experts.43.up_proj.weight": "model-00035-of-00101.safetensors", + "model.layers.33.mlp.experts.43.down_proj.weight": "model-00035-of-00101.safetensors", + "model.layers.33.mlp.experts.44.gate_proj.weight": "model-00035-of-00101.safetensors", + "model.layers.33.mlp.experts.44.up_proj.weight": "model-00035-of-00101.safetensors", + "model.layers.33.mlp.experts.44.down_proj.weight": "model-00035-of-00101.safetensors", + "model.layers.33.mlp.experts.45.gate_proj.weight": "model-00035-of-00101.safetensors", + "model.layers.33.mlp.experts.45.up_proj.weight": "model-00035-of-00101.safetensors", + "model.layers.33.mlp.experts.45.down_proj.weight": "model-00035-of-00101.safetensors", + "model.layers.33.mlp.experts.46.gate_proj.weight": "model-00035-of-00101.safetensors", + "model.layers.33.mlp.experts.46.up_proj.weight": "model-00035-of-00101.safetensors", + "model.layers.33.mlp.experts.46.down_proj.weight": "model-00035-of-00101.safetensors", + "model.layers.33.mlp.experts.47.gate_proj.weight": "model-00035-of-00101.safetensors", + "model.layers.33.mlp.experts.47.up_proj.weight": "model-00035-of-00101.safetensors", + "model.layers.33.mlp.experts.47.down_proj.weight": "model-00035-of-00101.safetensors", + "model.layers.33.mlp.experts.48.gate_proj.weight": "model-00035-of-00101.safetensors", + "model.layers.33.mlp.experts.48.up_proj.weight": "model-00035-of-00101.safetensors", + "model.layers.33.mlp.experts.48.down_proj.weight": "model-00035-of-00101.safetensors", + "model.layers.33.mlp.experts.49.gate_proj.weight": "model-00035-of-00101.safetensors", + "model.layers.33.mlp.experts.49.up_proj.weight": "model-00035-of-00101.safetensors", + "model.layers.33.mlp.experts.49.down_proj.weight": "model-00035-of-00101.safetensors", + "model.layers.33.mlp.experts.50.gate_proj.weight": "model-00035-of-00101.safetensors", + "model.layers.33.mlp.experts.50.up_proj.weight": "model-00035-of-00101.safetensors", + "model.layers.33.mlp.experts.50.down_proj.weight": "model-00035-of-00101.safetensors", + "model.layers.33.mlp.experts.51.gate_proj.weight": "model-00035-of-00101.safetensors", + "model.layers.33.mlp.experts.51.up_proj.weight": "model-00035-of-00101.safetensors", + "model.layers.33.mlp.experts.51.down_proj.weight": "model-00035-of-00101.safetensors", + "model.layers.33.mlp.experts.52.gate_proj.weight": "model-00035-of-00101.safetensors", + "model.layers.33.mlp.experts.52.up_proj.weight": "model-00035-of-00101.safetensors", + "model.layers.33.mlp.experts.52.down_proj.weight": "model-00035-of-00101.safetensors", + "model.layers.33.mlp.experts.53.gate_proj.weight": "model-00035-of-00101.safetensors", + "model.layers.33.mlp.experts.53.up_proj.weight": "model-00035-of-00101.safetensors", + "model.layers.33.mlp.experts.53.down_proj.weight": "model-00035-of-00101.safetensors", + "model.layers.33.mlp.experts.54.gate_proj.weight": "model-00035-of-00101.safetensors", + "model.layers.33.mlp.experts.54.up_proj.weight": "model-00035-of-00101.safetensors", + "model.layers.33.mlp.experts.54.down_proj.weight": "model-00035-of-00101.safetensors", + "model.layers.33.mlp.experts.55.gate_proj.weight": "model-00035-of-00101.safetensors", + "model.layers.33.mlp.experts.55.up_proj.weight": "model-00035-of-00101.safetensors", + "model.layers.33.mlp.experts.55.down_proj.weight": "model-00035-of-00101.safetensors", + "model.layers.33.mlp.experts.56.gate_proj.weight": "model-00035-of-00101.safetensors", + "model.layers.33.mlp.experts.56.up_proj.weight": "model-00035-of-00101.safetensors", + "model.layers.33.mlp.experts.56.down_proj.weight": "model-00035-of-00101.safetensors", + "model.layers.33.mlp.experts.57.gate_proj.weight": "model-00035-of-00101.safetensors", + "model.layers.33.mlp.experts.57.up_proj.weight": "model-00035-of-00101.safetensors", + "model.layers.33.mlp.experts.57.down_proj.weight": "model-00035-of-00101.safetensors", + "model.layers.33.mlp.experts.58.gate_proj.weight": "model-00035-of-00101.safetensors", + "model.layers.33.mlp.experts.58.up_proj.weight": "model-00035-of-00101.safetensors", + "model.layers.33.mlp.experts.58.down_proj.weight": "model-00035-of-00101.safetensors", + "model.layers.33.mlp.experts.59.gate_proj.weight": "model-00035-of-00101.safetensors", + "model.layers.33.mlp.experts.59.up_proj.weight": "model-00035-of-00101.safetensors", + "model.layers.33.mlp.experts.59.down_proj.weight": "model-00035-of-00101.safetensors", + "model.layers.33.mlp.experts.60.gate_proj.weight": "model-00035-of-00101.safetensors", + "model.layers.33.mlp.experts.60.up_proj.weight": "model-00035-of-00101.safetensors", + "model.layers.33.mlp.experts.60.down_proj.weight": "model-00035-of-00101.safetensors", + "model.layers.33.mlp.experts.61.gate_proj.weight": "model-00035-of-00101.safetensors", + "model.layers.33.mlp.experts.61.up_proj.weight": "model-00035-of-00101.safetensors", + "model.layers.33.mlp.experts.61.down_proj.weight": "model-00035-of-00101.safetensors", + "model.layers.33.mlp.experts.62.gate_proj.weight": "model-00035-of-00101.safetensors", + "model.layers.33.mlp.experts.62.up_proj.weight": "model-00035-of-00101.safetensors", + "model.layers.33.mlp.experts.62.down_proj.weight": "model-00035-of-00101.safetensors", + "model.layers.33.mlp.experts.63.gate_proj.weight": "model-00035-of-00101.safetensors", + "model.layers.33.mlp.experts.63.up_proj.weight": "model-00035-of-00101.safetensors", + "model.layers.33.mlp.experts.63.down_proj.weight": "model-00035-of-00101.safetensors", + "model.layers.33.mlp.experts.64.gate_proj.weight": "model-00035-of-00101.safetensors", + "model.layers.33.mlp.experts.64.up_proj.weight": "model-00035-of-00101.safetensors", + "model.layers.33.mlp.experts.64.down_proj.weight": "model-00035-of-00101.safetensors", + "model.layers.33.mlp.experts.65.gate_proj.weight": "model-00035-of-00101.safetensors", + "model.layers.33.mlp.experts.65.up_proj.weight": "model-00035-of-00101.safetensors", + "model.layers.33.mlp.experts.65.down_proj.weight": "model-00035-of-00101.safetensors", + "model.layers.33.mlp.experts.66.gate_proj.weight": "model-00035-of-00101.safetensors", + "model.layers.33.mlp.experts.66.up_proj.weight": "model-00035-of-00101.safetensors", + "model.layers.33.mlp.experts.66.down_proj.weight": "model-00035-of-00101.safetensors", + "model.layers.33.mlp.experts.67.gate_proj.weight": "model-00035-of-00101.safetensors", + "model.layers.33.mlp.experts.67.up_proj.weight": "model-00035-of-00101.safetensors", + "model.layers.33.mlp.experts.67.down_proj.weight": "model-00035-of-00101.safetensors", + "model.layers.33.mlp.experts.68.gate_proj.weight": "model-00035-of-00101.safetensors", + "model.layers.33.mlp.experts.68.up_proj.weight": "model-00035-of-00101.safetensors", + "model.layers.33.mlp.experts.68.down_proj.weight": "model-00035-of-00101.safetensors", + "model.layers.33.mlp.experts.69.gate_proj.weight": "model-00035-of-00101.safetensors", + "model.layers.33.mlp.experts.69.up_proj.weight": "model-00035-of-00101.safetensors", + "model.layers.33.mlp.experts.69.down_proj.weight": "model-00035-of-00101.safetensors", + "model.layers.33.mlp.experts.70.gate_proj.weight": "model-00035-of-00101.safetensors", + "model.layers.33.mlp.experts.70.up_proj.weight": "model-00035-of-00101.safetensors", + "model.layers.33.mlp.experts.70.down_proj.weight": "model-00035-of-00101.safetensors", + "model.layers.33.mlp.experts.71.gate_proj.weight": "model-00035-of-00101.safetensors", + "model.layers.33.mlp.experts.71.up_proj.weight": "model-00035-of-00101.safetensors", + "model.layers.33.mlp.experts.71.down_proj.weight": "model-00035-of-00101.safetensors", + "model.layers.33.mlp.experts.72.gate_proj.weight": "model-00035-of-00101.safetensors", + "model.layers.33.mlp.experts.72.up_proj.weight": "model-00035-of-00101.safetensors", + "model.layers.33.mlp.experts.72.down_proj.weight": "model-00035-of-00101.safetensors", + "model.layers.33.mlp.experts.73.gate_proj.weight": "model-00035-of-00101.safetensors", + "model.layers.33.mlp.experts.73.up_proj.weight": "model-00035-of-00101.safetensors", + "model.layers.33.mlp.experts.73.down_proj.weight": "model-00035-of-00101.safetensors", + "model.layers.33.mlp.experts.74.gate_proj.weight": "model-00035-of-00101.safetensors", + "model.layers.33.mlp.experts.74.up_proj.weight": "model-00035-of-00101.safetensors", + "model.layers.33.mlp.experts.74.down_proj.weight": "model-00035-of-00101.safetensors", + "model.layers.33.mlp.experts.75.gate_proj.weight": "model-00035-of-00101.safetensors", + "model.layers.33.mlp.experts.75.up_proj.weight": "model-00035-of-00101.safetensors", + "model.layers.33.mlp.experts.75.down_proj.weight": "model-00035-of-00101.safetensors", + "model.layers.33.mlp.experts.76.gate_proj.weight": "model-00035-of-00101.safetensors", + "model.layers.33.mlp.experts.76.up_proj.weight": "model-00035-of-00101.safetensors", + "model.layers.33.mlp.experts.76.down_proj.weight": "model-00035-of-00101.safetensors", + "model.layers.33.mlp.experts.77.gate_proj.weight": "model-00035-of-00101.safetensors", + "model.layers.33.mlp.experts.77.up_proj.weight": "model-00035-of-00101.safetensors", + "model.layers.33.mlp.experts.77.down_proj.weight": "model-00035-of-00101.safetensors", + "model.layers.33.mlp.experts.78.gate_proj.weight": "model-00035-of-00101.safetensors", + "model.layers.33.mlp.experts.78.up_proj.weight": "model-00035-of-00101.safetensors", + "model.layers.33.mlp.experts.78.down_proj.weight": "model-00035-of-00101.safetensors", + "model.layers.33.mlp.experts.79.gate_proj.weight": "model-00035-of-00101.safetensors", + "model.layers.33.mlp.experts.79.up_proj.weight": "model-00035-of-00101.safetensors", + "model.layers.33.mlp.experts.79.down_proj.weight": "model-00035-of-00101.safetensors", + "model.layers.33.mlp.experts.80.gate_proj.weight": "model-00035-of-00101.safetensors", + "model.layers.33.mlp.experts.80.up_proj.weight": "model-00035-of-00101.safetensors", + "model.layers.33.mlp.experts.80.down_proj.weight": "model-00035-of-00101.safetensors", + "model.layers.33.mlp.experts.81.gate_proj.weight": "model-00035-of-00101.safetensors", + "model.layers.33.mlp.experts.81.up_proj.weight": "model-00035-of-00101.safetensors", + "model.layers.33.mlp.experts.81.down_proj.weight": "model-00035-of-00101.safetensors", + "model.layers.33.mlp.experts.82.gate_proj.weight": "model-00035-of-00101.safetensors", + "model.layers.33.mlp.experts.82.up_proj.weight": "model-00035-of-00101.safetensors", + "model.layers.33.mlp.experts.82.down_proj.weight": "model-00035-of-00101.safetensors", + "model.layers.33.mlp.experts.83.gate_proj.weight": "model-00035-of-00101.safetensors", + "model.layers.33.mlp.experts.83.up_proj.weight": "model-00035-of-00101.safetensors", + "model.layers.33.mlp.experts.83.down_proj.weight": "model-00035-of-00101.safetensors", + "model.layers.33.mlp.experts.84.gate_proj.weight": "model-00035-of-00101.safetensors", + "model.layers.33.mlp.experts.84.up_proj.weight": "model-00035-of-00101.safetensors", + "model.layers.33.mlp.experts.84.down_proj.weight": "model-00035-of-00101.safetensors", + "model.layers.33.mlp.experts.85.gate_proj.weight": "model-00035-of-00101.safetensors", + "model.layers.33.mlp.experts.85.up_proj.weight": "model-00035-of-00101.safetensors", + "model.layers.33.mlp.experts.85.down_proj.weight": "model-00035-of-00101.safetensors", + "model.layers.33.mlp.experts.86.gate_proj.weight": "model-00035-of-00101.safetensors", + "model.layers.33.mlp.experts.86.up_proj.weight": "model-00035-of-00101.safetensors", + "model.layers.33.mlp.experts.86.down_proj.weight": "model-00035-of-00101.safetensors", + "model.layers.33.mlp.experts.87.gate_proj.weight": "model-00035-of-00101.safetensors", + "model.layers.33.mlp.experts.87.up_proj.weight": "model-00035-of-00101.safetensors", + "model.layers.33.mlp.experts.87.down_proj.weight": "model-00035-of-00101.safetensors", + "model.layers.33.mlp.experts.88.gate_proj.weight": "model-00036-of-00101.safetensors", + "model.layers.33.mlp.experts.88.up_proj.weight": "model-00036-of-00101.safetensors", + "model.layers.33.mlp.experts.88.down_proj.weight": "model-00036-of-00101.safetensors", + "model.layers.33.mlp.experts.89.gate_proj.weight": "model-00036-of-00101.safetensors", + "model.layers.33.mlp.experts.89.up_proj.weight": "model-00036-of-00101.safetensors", + "model.layers.33.mlp.experts.89.down_proj.weight": "model-00036-of-00101.safetensors", + "model.layers.33.mlp.experts.90.gate_proj.weight": "model-00036-of-00101.safetensors", + "model.layers.33.mlp.experts.90.up_proj.weight": "model-00036-of-00101.safetensors", + "model.layers.33.mlp.experts.90.down_proj.weight": "model-00036-of-00101.safetensors", + "model.layers.33.mlp.experts.91.gate_proj.weight": "model-00036-of-00101.safetensors", + "model.layers.33.mlp.experts.91.up_proj.weight": "model-00036-of-00101.safetensors", + "model.layers.33.mlp.experts.91.down_proj.weight": "model-00036-of-00101.safetensors", + "model.layers.33.mlp.experts.92.gate_proj.weight": "model-00036-of-00101.safetensors", + "model.layers.33.mlp.experts.92.up_proj.weight": "model-00036-of-00101.safetensors", + "model.layers.33.mlp.experts.92.down_proj.weight": "model-00036-of-00101.safetensors", + "model.layers.33.mlp.experts.93.gate_proj.weight": "model-00036-of-00101.safetensors", + "model.layers.33.mlp.experts.93.up_proj.weight": "model-00036-of-00101.safetensors", + "model.layers.33.mlp.experts.93.down_proj.weight": "model-00036-of-00101.safetensors", + "model.layers.33.mlp.experts.94.gate_proj.weight": "model-00036-of-00101.safetensors", + "model.layers.33.mlp.experts.94.up_proj.weight": "model-00036-of-00101.safetensors", + "model.layers.33.mlp.experts.94.down_proj.weight": "model-00036-of-00101.safetensors", + "model.layers.33.mlp.experts.95.gate_proj.weight": "model-00036-of-00101.safetensors", + "model.layers.33.mlp.experts.95.up_proj.weight": "model-00036-of-00101.safetensors", + "model.layers.33.mlp.experts.95.down_proj.weight": "model-00036-of-00101.safetensors", + "model.layers.33.mlp.experts.96.gate_proj.weight": "model-00036-of-00101.safetensors", + "model.layers.33.mlp.experts.96.up_proj.weight": "model-00036-of-00101.safetensors", + "model.layers.33.mlp.experts.96.down_proj.weight": "model-00036-of-00101.safetensors", + "model.layers.33.mlp.experts.97.gate_proj.weight": "model-00036-of-00101.safetensors", + "model.layers.33.mlp.experts.97.up_proj.weight": "model-00036-of-00101.safetensors", + "model.layers.33.mlp.experts.97.down_proj.weight": "model-00036-of-00101.safetensors", + "model.layers.33.mlp.experts.98.gate_proj.weight": "model-00036-of-00101.safetensors", + "model.layers.33.mlp.experts.98.up_proj.weight": "model-00036-of-00101.safetensors", + "model.layers.33.mlp.experts.98.down_proj.weight": "model-00036-of-00101.safetensors", + "model.layers.33.mlp.experts.99.gate_proj.weight": "model-00036-of-00101.safetensors", + "model.layers.33.mlp.experts.99.up_proj.weight": "model-00036-of-00101.safetensors", + "model.layers.33.mlp.experts.99.down_proj.weight": "model-00036-of-00101.safetensors", + "model.layers.33.mlp.experts.100.gate_proj.weight": "model-00036-of-00101.safetensors", + "model.layers.33.mlp.experts.100.up_proj.weight": "model-00036-of-00101.safetensors", + "model.layers.33.mlp.experts.100.down_proj.weight": "model-00036-of-00101.safetensors", + "model.layers.33.mlp.experts.101.gate_proj.weight": "model-00036-of-00101.safetensors", + "model.layers.33.mlp.experts.101.up_proj.weight": "model-00036-of-00101.safetensors", + "model.layers.33.mlp.experts.101.down_proj.weight": "model-00036-of-00101.safetensors", + "model.layers.33.mlp.experts.102.gate_proj.weight": "model-00036-of-00101.safetensors", + "model.layers.33.mlp.experts.102.up_proj.weight": "model-00036-of-00101.safetensors", + "model.layers.33.mlp.experts.102.down_proj.weight": "model-00036-of-00101.safetensors", + "model.layers.33.mlp.experts.103.gate_proj.weight": "model-00036-of-00101.safetensors", + "model.layers.33.mlp.experts.103.up_proj.weight": "model-00036-of-00101.safetensors", + "model.layers.33.mlp.experts.103.down_proj.weight": "model-00036-of-00101.safetensors", + "model.layers.33.mlp.experts.104.gate_proj.weight": "model-00036-of-00101.safetensors", + "model.layers.33.mlp.experts.104.up_proj.weight": "model-00036-of-00101.safetensors", + "model.layers.33.mlp.experts.104.down_proj.weight": "model-00036-of-00101.safetensors", + "model.layers.33.mlp.experts.105.gate_proj.weight": "model-00036-of-00101.safetensors", + "model.layers.33.mlp.experts.105.up_proj.weight": "model-00036-of-00101.safetensors", + "model.layers.33.mlp.experts.105.down_proj.weight": "model-00036-of-00101.safetensors", + "model.layers.33.mlp.experts.106.gate_proj.weight": "model-00036-of-00101.safetensors", + "model.layers.33.mlp.experts.106.up_proj.weight": "model-00036-of-00101.safetensors", + "model.layers.33.mlp.experts.106.down_proj.weight": "model-00036-of-00101.safetensors", + "model.layers.33.mlp.experts.107.gate_proj.weight": "model-00036-of-00101.safetensors", + "model.layers.33.mlp.experts.107.up_proj.weight": "model-00036-of-00101.safetensors", + "model.layers.33.mlp.experts.107.down_proj.weight": "model-00036-of-00101.safetensors", + "model.layers.33.mlp.experts.108.gate_proj.weight": "model-00036-of-00101.safetensors", + "model.layers.33.mlp.experts.108.up_proj.weight": "model-00036-of-00101.safetensors", + "model.layers.33.mlp.experts.108.down_proj.weight": "model-00036-of-00101.safetensors", + "model.layers.33.mlp.experts.109.gate_proj.weight": "model-00036-of-00101.safetensors", + "model.layers.33.mlp.experts.109.up_proj.weight": "model-00036-of-00101.safetensors", + "model.layers.33.mlp.experts.109.down_proj.weight": "model-00036-of-00101.safetensors", + "model.layers.33.mlp.experts.110.gate_proj.weight": "model-00036-of-00101.safetensors", + "model.layers.33.mlp.experts.110.up_proj.weight": "model-00036-of-00101.safetensors", + "model.layers.33.mlp.experts.110.down_proj.weight": "model-00036-of-00101.safetensors", + "model.layers.33.mlp.experts.111.gate_proj.weight": "model-00036-of-00101.safetensors", + "model.layers.33.mlp.experts.111.up_proj.weight": "model-00036-of-00101.safetensors", + "model.layers.33.mlp.experts.111.down_proj.weight": "model-00036-of-00101.safetensors", + "model.layers.33.mlp.experts.112.gate_proj.weight": "model-00036-of-00101.safetensors", + "model.layers.33.mlp.experts.112.up_proj.weight": "model-00036-of-00101.safetensors", + "model.layers.33.mlp.experts.112.down_proj.weight": "model-00036-of-00101.safetensors", + "model.layers.33.mlp.experts.113.gate_proj.weight": "model-00036-of-00101.safetensors", + "model.layers.33.mlp.experts.113.up_proj.weight": "model-00036-of-00101.safetensors", + "model.layers.33.mlp.experts.113.down_proj.weight": "model-00036-of-00101.safetensors", + "model.layers.33.mlp.experts.114.gate_proj.weight": "model-00036-of-00101.safetensors", + "model.layers.33.mlp.experts.114.up_proj.weight": "model-00036-of-00101.safetensors", + "model.layers.33.mlp.experts.114.down_proj.weight": "model-00036-of-00101.safetensors", + "model.layers.33.mlp.experts.115.gate_proj.weight": "model-00036-of-00101.safetensors", + "model.layers.33.mlp.experts.115.up_proj.weight": "model-00036-of-00101.safetensors", + "model.layers.33.mlp.experts.115.down_proj.weight": "model-00036-of-00101.safetensors", + "model.layers.33.mlp.experts.116.gate_proj.weight": "model-00036-of-00101.safetensors", + "model.layers.33.mlp.experts.116.up_proj.weight": "model-00036-of-00101.safetensors", + "model.layers.33.mlp.experts.116.down_proj.weight": "model-00036-of-00101.safetensors", + "model.layers.33.mlp.experts.117.gate_proj.weight": "model-00036-of-00101.safetensors", + "model.layers.33.mlp.experts.117.up_proj.weight": "model-00036-of-00101.safetensors", + "model.layers.33.mlp.experts.117.down_proj.weight": "model-00036-of-00101.safetensors", + "model.layers.33.mlp.experts.118.gate_proj.weight": "model-00036-of-00101.safetensors", + "model.layers.33.mlp.experts.118.up_proj.weight": "model-00036-of-00101.safetensors", + "model.layers.33.mlp.experts.118.down_proj.weight": "model-00036-of-00101.safetensors", + "model.layers.33.mlp.experts.119.gate_proj.weight": "model-00036-of-00101.safetensors", + "model.layers.33.mlp.experts.119.up_proj.weight": "model-00036-of-00101.safetensors", + "model.layers.33.mlp.experts.119.down_proj.weight": "model-00036-of-00101.safetensors", + "model.layers.33.mlp.gate.weight": "model-00036-of-00101.safetensors", + "model.layers.33.mlp.gate.e_score_correction_bias": "model-00036-of-00101.safetensors", + "model.layers.33.mlp.shared_experts.gate_proj.weight": "model-00036-of-00101.safetensors", + "model.layers.33.mlp.shared_experts.up_proj.weight": "model-00036-of-00101.safetensors", + "model.layers.33.mlp.shared_experts.down_proj.weight": "model-00036-of-00101.safetensors", + "model.layers.33.input_layernorm.weight": "model-00036-of-00101.safetensors", + "model.layers.33.post_attention_layernorm.weight": "model-00036-of-00101.safetensors", + "model.layers.34.self_attn.q_proj.weight": "model-00036-of-00101.safetensors", + "model.layers.34.self_attn.q_proj.bias": "model-00036-of-00101.safetensors", + "model.layers.34.self_attn.k_proj.weight": "model-00036-of-00101.safetensors", + "model.layers.34.self_attn.k_proj.bias": "model-00036-of-00101.safetensors", + "model.layers.34.self_attn.v_proj.weight": "model-00036-of-00101.safetensors", + "model.layers.34.self_attn.v_proj.bias": "model-00036-of-00101.safetensors", + "model.layers.34.self_attn.o_proj.weight": "model-00036-of-00101.safetensors", + "model.layers.34.self_attn.q_norm.weight": "model-00036-of-00101.safetensors", + "model.layers.34.self_attn.k_norm.weight": "model-00036-of-00101.safetensors", + "model.layers.34.mlp.experts.0.gate_proj.weight": "model-00036-of-00101.safetensors", + "model.layers.34.mlp.experts.0.up_proj.weight": "model-00036-of-00101.safetensors", + "model.layers.34.mlp.experts.0.down_proj.weight": "model-00036-of-00101.safetensors", + "model.layers.34.mlp.experts.1.gate_proj.weight": "model-00036-of-00101.safetensors", + "model.layers.34.mlp.experts.1.up_proj.weight": "model-00036-of-00101.safetensors", + "model.layers.34.mlp.experts.1.down_proj.weight": "model-00036-of-00101.safetensors", + "model.layers.34.mlp.experts.2.gate_proj.weight": "model-00036-of-00101.safetensors", + "model.layers.34.mlp.experts.2.up_proj.weight": "model-00036-of-00101.safetensors", + "model.layers.34.mlp.experts.2.down_proj.weight": "model-00036-of-00101.safetensors", + "model.layers.34.mlp.experts.3.gate_proj.weight": "model-00036-of-00101.safetensors", + "model.layers.34.mlp.experts.3.up_proj.weight": "model-00036-of-00101.safetensors", + "model.layers.34.mlp.experts.3.down_proj.weight": "model-00036-of-00101.safetensors", + "model.layers.34.mlp.experts.4.gate_proj.weight": "model-00036-of-00101.safetensors", + "model.layers.34.mlp.experts.4.up_proj.weight": "model-00036-of-00101.safetensors", + "model.layers.34.mlp.experts.4.down_proj.weight": "model-00036-of-00101.safetensors", + "model.layers.34.mlp.experts.5.gate_proj.weight": "model-00036-of-00101.safetensors", + "model.layers.34.mlp.experts.5.up_proj.weight": "model-00036-of-00101.safetensors", + "model.layers.34.mlp.experts.5.down_proj.weight": "model-00036-of-00101.safetensors", + "model.layers.34.mlp.experts.6.gate_proj.weight": "model-00036-of-00101.safetensors", + "model.layers.34.mlp.experts.6.up_proj.weight": "model-00036-of-00101.safetensors", + "model.layers.34.mlp.experts.6.down_proj.weight": "model-00036-of-00101.safetensors", + "model.layers.34.mlp.experts.7.gate_proj.weight": "model-00036-of-00101.safetensors", + "model.layers.34.mlp.experts.7.up_proj.weight": "model-00036-of-00101.safetensors", + "model.layers.34.mlp.experts.7.down_proj.weight": "model-00036-of-00101.safetensors", + "model.layers.34.mlp.experts.8.gate_proj.weight": "model-00036-of-00101.safetensors", + "model.layers.34.mlp.experts.8.up_proj.weight": "model-00036-of-00101.safetensors", + "model.layers.34.mlp.experts.8.down_proj.weight": "model-00036-of-00101.safetensors", + "model.layers.34.mlp.experts.9.gate_proj.weight": "model-00036-of-00101.safetensors", + "model.layers.34.mlp.experts.9.up_proj.weight": "model-00036-of-00101.safetensors", + "model.layers.34.mlp.experts.9.down_proj.weight": "model-00036-of-00101.safetensors", + "model.layers.34.mlp.experts.10.gate_proj.weight": "model-00036-of-00101.safetensors", + "model.layers.34.mlp.experts.10.up_proj.weight": "model-00036-of-00101.safetensors", + "model.layers.34.mlp.experts.10.down_proj.weight": "model-00036-of-00101.safetensors", + "model.layers.34.mlp.experts.11.gate_proj.weight": "model-00036-of-00101.safetensors", + "model.layers.34.mlp.experts.11.up_proj.weight": "model-00036-of-00101.safetensors", + "model.layers.34.mlp.experts.11.down_proj.weight": "model-00036-of-00101.safetensors", + "model.layers.34.mlp.experts.12.gate_proj.weight": "model-00036-of-00101.safetensors", + "model.layers.34.mlp.experts.12.up_proj.weight": "model-00036-of-00101.safetensors", + "model.layers.34.mlp.experts.12.down_proj.weight": "model-00036-of-00101.safetensors", + "model.layers.34.mlp.experts.13.gate_proj.weight": "model-00036-of-00101.safetensors", + "model.layers.34.mlp.experts.13.up_proj.weight": "model-00036-of-00101.safetensors", + "model.layers.34.mlp.experts.13.down_proj.weight": "model-00036-of-00101.safetensors", + "model.layers.34.mlp.experts.14.gate_proj.weight": "model-00036-of-00101.safetensors", + "model.layers.34.mlp.experts.14.up_proj.weight": "model-00036-of-00101.safetensors", + "model.layers.34.mlp.experts.14.down_proj.weight": "model-00036-of-00101.safetensors", + "model.layers.34.mlp.experts.15.gate_proj.weight": "model-00036-of-00101.safetensors", + "model.layers.34.mlp.experts.15.up_proj.weight": "model-00036-of-00101.safetensors", + "model.layers.34.mlp.experts.15.down_proj.weight": "model-00036-of-00101.safetensors", + "model.layers.34.mlp.experts.16.gate_proj.weight": "model-00036-of-00101.safetensors", + "model.layers.34.mlp.experts.16.up_proj.weight": "model-00036-of-00101.safetensors", + "model.layers.34.mlp.experts.16.down_proj.weight": "model-00036-of-00101.safetensors", + "model.layers.34.mlp.experts.17.gate_proj.weight": "model-00036-of-00101.safetensors", + "model.layers.34.mlp.experts.17.up_proj.weight": "model-00036-of-00101.safetensors", + "model.layers.34.mlp.experts.17.down_proj.weight": "model-00036-of-00101.safetensors", + "model.layers.34.mlp.experts.18.gate_proj.weight": "model-00036-of-00101.safetensors", + "model.layers.34.mlp.experts.18.up_proj.weight": "model-00036-of-00101.safetensors", + "model.layers.34.mlp.experts.18.down_proj.weight": "model-00036-of-00101.safetensors", + "model.layers.34.mlp.experts.19.gate_proj.weight": "model-00036-of-00101.safetensors", + "model.layers.34.mlp.experts.19.up_proj.weight": "model-00036-of-00101.safetensors", + "model.layers.34.mlp.experts.19.down_proj.weight": "model-00036-of-00101.safetensors", + "model.layers.34.mlp.experts.20.gate_proj.weight": "model-00036-of-00101.safetensors", + "model.layers.34.mlp.experts.20.up_proj.weight": "model-00036-of-00101.safetensors", + "model.layers.34.mlp.experts.20.down_proj.weight": "model-00036-of-00101.safetensors", + "model.layers.34.mlp.experts.21.gate_proj.weight": "model-00036-of-00101.safetensors", + "model.layers.34.mlp.experts.21.up_proj.weight": "model-00036-of-00101.safetensors", + "model.layers.34.mlp.experts.21.down_proj.weight": "model-00036-of-00101.safetensors", + "model.layers.34.mlp.experts.22.gate_proj.weight": "model-00036-of-00101.safetensors", + "model.layers.34.mlp.experts.22.up_proj.weight": "model-00036-of-00101.safetensors", + "model.layers.34.mlp.experts.22.down_proj.weight": "model-00036-of-00101.safetensors", + "model.layers.34.mlp.experts.23.gate_proj.weight": "model-00036-of-00101.safetensors", + "model.layers.34.mlp.experts.23.up_proj.weight": "model-00036-of-00101.safetensors", + "model.layers.34.mlp.experts.23.down_proj.weight": "model-00036-of-00101.safetensors", + "model.layers.34.mlp.experts.24.gate_proj.weight": "model-00036-of-00101.safetensors", + "model.layers.34.mlp.experts.24.up_proj.weight": "model-00036-of-00101.safetensors", + "model.layers.34.mlp.experts.24.down_proj.weight": "model-00036-of-00101.safetensors", + "model.layers.34.mlp.experts.25.gate_proj.weight": "model-00036-of-00101.safetensors", + "model.layers.34.mlp.experts.25.up_proj.weight": "model-00036-of-00101.safetensors", + "model.layers.34.mlp.experts.25.down_proj.weight": "model-00036-of-00101.safetensors", + "model.layers.34.mlp.experts.26.gate_proj.weight": "model-00036-of-00101.safetensors", + "model.layers.34.mlp.experts.26.up_proj.weight": "model-00036-of-00101.safetensors", + "model.layers.34.mlp.experts.26.down_proj.weight": "model-00036-of-00101.safetensors", + "model.layers.34.mlp.experts.27.gate_proj.weight": "model-00036-of-00101.safetensors", + "model.layers.34.mlp.experts.27.up_proj.weight": "model-00036-of-00101.safetensors", + "model.layers.34.mlp.experts.27.down_proj.weight": "model-00036-of-00101.safetensors", + "model.layers.34.mlp.experts.28.gate_proj.weight": "model-00036-of-00101.safetensors", + "model.layers.34.mlp.experts.28.up_proj.weight": "model-00036-of-00101.safetensors", + "model.layers.34.mlp.experts.28.down_proj.weight": "model-00036-of-00101.safetensors", + "model.layers.34.mlp.experts.29.gate_proj.weight": "model-00036-of-00101.safetensors", + "model.layers.34.mlp.experts.29.up_proj.weight": "model-00036-of-00101.safetensors", + "model.layers.34.mlp.experts.29.down_proj.weight": "model-00036-of-00101.safetensors", + "model.layers.34.mlp.experts.30.gate_proj.weight": "model-00036-of-00101.safetensors", + "model.layers.34.mlp.experts.30.up_proj.weight": "model-00036-of-00101.safetensors", + "model.layers.34.mlp.experts.30.down_proj.weight": "model-00036-of-00101.safetensors", + "model.layers.34.mlp.experts.31.gate_proj.weight": "model-00036-of-00101.safetensors", + "model.layers.34.mlp.experts.31.up_proj.weight": "model-00036-of-00101.safetensors", + "model.layers.34.mlp.experts.31.down_proj.weight": "model-00036-of-00101.safetensors", + "model.layers.34.mlp.experts.32.gate_proj.weight": "model-00036-of-00101.safetensors", + "model.layers.34.mlp.experts.32.up_proj.weight": "model-00036-of-00101.safetensors", + "model.layers.34.mlp.experts.32.down_proj.weight": "model-00036-of-00101.safetensors", + "model.layers.34.mlp.experts.33.gate_proj.weight": "model-00036-of-00101.safetensors", + "model.layers.34.mlp.experts.33.up_proj.weight": "model-00036-of-00101.safetensors", + "model.layers.34.mlp.experts.33.down_proj.weight": "model-00036-of-00101.safetensors", + "model.layers.34.mlp.experts.34.gate_proj.weight": "model-00036-of-00101.safetensors", + "model.layers.34.mlp.experts.34.up_proj.weight": "model-00036-of-00101.safetensors", + "model.layers.34.mlp.experts.34.down_proj.weight": "model-00036-of-00101.safetensors", + "model.layers.34.mlp.experts.35.gate_proj.weight": "model-00036-of-00101.safetensors", + "model.layers.34.mlp.experts.35.up_proj.weight": "model-00036-of-00101.safetensors", + "model.layers.34.mlp.experts.35.down_proj.weight": "model-00036-of-00101.safetensors", + "model.layers.34.mlp.experts.36.gate_proj.weight": "model-00036-of-00101.safetensors", + "model.layers.34.mlp.experts.36.up_proj.weight": "model-00036-of-00101.safetensors", + "model.layers.34.mlp.experts.36.down_proj.weight": "model-00036-of-00101.safetensors", + "model.layers.34.mlp.experts.37.gate_proj.weight": "model-00036-of-00101.safetensors", + "model.layers.34.mlp.experts.37.up_proj.weight": "model-00036-of-00101.safetensors", + "model.layers.34.mlp.experts.37.down_proj.weight": "model-00036-of-00101.safetensors", + "model.layers.34.mlp.experts.38.gate_proj.weight": "model-00036-of-00101.safetensors", + "model.layers.34.mlp.experts.38.up_proj.weight": "model-00036-of-00101.safetensors", + "model.layers.34.mlp.experts.38.down_proj.weight": "model-00036-of-00101.safetensors", + "model.layers.34.mlp.experts.39.gate_proj.weight": "model-00036-of-00101.safetensors", + "model.layers.34.mlp.experts.39.up_proj.weight": "model-00036-of-00101.safetensors", + "model.layers.34.mlp.experts.39.down_proj.weight": "model-00036-of-00101.safetensors", + "model.layers.34.mlp.experts.40.gate_proj.weight": "model-00036-of-00101.safetensors", + "model.layers.34.mlp.experts.40.up_proj.weight": "model-00036-of-00101.safetensors", + "model.layers.34.mlp.experts.40.down_proj.weight": "model-00036-of-00101.safetensors", + "model.layers.34.mlp.experts.41.gate_proj.weight": "model-00036-of-00101.safetensors", + "model.layers.34.mlp.experts.41.up_proj.weight": "model-00036-of-00101.safetensors", + "model.layers.34.mlp.experts.41.down_proj.weight": "model-00036-of-00101.safetensors", + "model.layers.34.mlp.experts.42.gate_proj.weight": "model-00036-of-00101.safetensors", + "model.layers.34.mlp.experts.42.up_proj.weight": "model-00036-of-00101.safetensors", + "model.layers.34.mlp.experts.42.down_proj.weight": "model-00036-of-00101.safetensors", + "model.layers.34.mlp.experts.43.gate_proj.weight": "model-00036-of-00101.safetensors", + "model.layers.34.mlp.experts.43.up_proj.weight": "model-00036-of-00101.safetensors", + "model.layers.34.mlp.experts.43.down_proj.weight": "model-00036-of-00101.safetensors", + "model.layers.34.mlp.experts.44.gate_proj.weight": "model-00036-of-00101.safetensors", + "model.layers.34.mlp.experts.44.up_proj.weight": "model-00036-of-00101.safetensors", + "model.layers.34.mlp.experts.44.down_proj.weight": "model-00036-of-00101.safetensors", + "model.layers.34.mlp.experts.45.gate_proj.weight": "model-00036-of-00101.safetensors", + "model.layers.34.mlp.experts.45.up_proj.weight": "model-00036-of-00101.safetensors", + "model.layers.34.mlp.experts.45.down_proj.weight": "model-00036-of-00101.safetensors", + "model.layers.34.mlp.experts.46.gate_proj.weight": "model-00036-of-00101.safetensors", + "model.layers.34.mlp.experts.46.up_proj.weight": "model-00036-of-00101.safetensors", + "model.layers.34.mlp.experts.46.down_proj.weight": "model-00036-of-00101.safetensors", + "model.layers.34.mlp.experts.47.gate_proj.weight": "model-00036-of-00101.safetensors", + "model.layers.34.mlp.experts.47.up_proj.weight": "model-00036-of-00101.safetensors", + "model.layers.34.mlp.experts.47.down_proj.weight": "model-00036-of-00101.safetensors", + "model.layers.34.mlp.experts.48.gate_proj.weight": "model-00036-of-00101.safetensors", + "model.layers.34.mlp.experts.48.up_proj.weight": "model-00036-of-00101.safetensors", + "model.layers.34.mlp.experts.48.down_proj.weight": "model-00036-of-00101.safetensors", + "model.layers.34.mlp.experts.49.gate_proj.weight": "model-00036-of-00101.safetensors", + "model.layers.34.mlp.experts.49.up_proj.weight": "model-00036-of-00101.safetensors", + "model.layers.34.mlp.experts.49.down_proj.weight": "model-00036-of-00101.safetensors", + "model.layers.34.mlp.experts.50.gate_proj.weight": "model-00036-of-00101.safetensors", + "model.layers.34.mlp.experts.50.up_proj.weight": "model-00036-of-00101.safetensors", + "model.layers.34.mlp.experts.50.down_proj.weight": "model-00036-of-00101.safetensors", + "model.layers.34.mlp.experts.51.gate_proj.weight": "model-00036-of-00101.safetensors", + "model.layers.34.mlp.experts.51.up_proj.weight": "model-00036-of-00101.safetensors", + "model.layers.34.mlp.experts.51.down_proj.weight": "model-00036-of-00101.safetensors", + "model.layers.34.mlp.experts.52.gate_proj.weight": "model-00036-of-00101.safetensors", + "model.layers.34.mlp.experts.52.up_proj.weight": "model-00036-of-00101.safetensors", + "model.layers.34.mlp.experts.52.down_proj.weight": "model-00036-of-00101.safetensors", + "model.layers.34.mlp.experts.53.gate_proj.weight": "model-00036-of-00101.safetensors", + "model.layers.34.mlp.experts.53.up_proj.weight": "model-00036-of-00101.safetensors", + "model.layers.34.mlp.experts.53.down_proj.weight": "model-00036-of-00101.safetensors", + "model.layers.34.mlp.experts.54.gate_proj.weight": "model-00036-of-00101.safetensors", + "model.layers.34.mlp.experts.54.up_proj.weight": "model-00036-of-00101.safetensors", + "model.layers.34.mlp.experts.54.down_proj.weight": "model-00036-of-00101.safetensors", + "model.layers.34.mlp.experts.55.gate_proj.weight": "model-00036-of-00101.safetensors", + "model.layers.34.mlp.experts.55.up_proj.weight": "model-00036-of-00101.safetensors", + "model.layers.34.mlp.experts.55.down_proj.weight": "model-00036-of-00101.safetensors", + "model.layers.34.mlp.experts.56.gate_proj.weight": "model-00036-of-00101.safetensors", + "model.layers.34.mlp.experts.56.up_proj.weight": "model-00036-of-00101.safetensors", + "model.layers.34.mlp.experts.56.down_proj.weight": "model-00036-of-00101.safetensors", + "model.layers.34.mlp.experts.57.gate_proj.weight": "model-00036-of-00101.safetensors", + "model.layers.34.mlp.experts.57.up_proj.weight": "model-00036-of-00101.safetensors", + "model.layers.34.mlp.experts.57.down_proj.weight": "model-00036-of-00101.safetensors", + "model.layers.34.mlp.experts.58.gate_proj.weight": "model-00036-of-00101.safetensors", + "model.layers.34.mlp.experts.58.up_proj.weight": "model-00036-of-00101.safetensors", + "model.layers.34.mlp.experts.58.down_proj.weight": "model-00036-of-00101.safetensors", + "model.layers.34.mlp.experts.59.gate_proj.weight": "model-00036-of-00101.safetensors", + "model.layers.34.mlp.experts.59.up_proj.weight": "model-00036-of-00101.safetensors", + "model.layers.34.mlp.experts.59.down_proj.weight": "model-00036-of-00101.safetensors", + "model.layers.34.mlp.experts.60.gate_proj.weight": "model-00036-of-00101.safetensors", + "model.layers.34.mlp.experts.60.up_proj.weight": "model-00036-of-00101.safetensors", + "model.layers.34.mlp.experts.60.down_proj.weight": "model-00036-of-00101.safetensors", + "model.layers.34.mlp.experts.61.gate_proj.weight": "model-00036-of-00101.safetensors", + "model.layers.34.mlp.experts.61.up_proj.weight": "model-00036-of-00101.safetensors", + "model.layers.34.mlp.experts.61.down_proj.weight": "model-00036-of-00101.safetensors", + "model.layers.34.mlp.experts.62.gate_proj.weight": "model-00036-of-00101.safetensors", + "model.layers.34.mlp.experts.62.up_proj.weight": "model-00036-of-00101.safetensors", + "model.layers.34.mlp.experts.62.down_proj.weight": "model-00036-of-00101.safetensors", + "model.layers.34.mlp.experts.63.gate_proj.weight": "model-00036-of-00101.safetensors", + "model.layers.34.mlp.experts.63.up_proj.weight": "model-00036-of-00101.safetensors", + "model.layers.34.mlp.experts.63.down_proj.weight": "model-00036-of-00101.safetensors", + "model.layers.34.mlp.experts.64.gate_proj.weight": "model-00036-of-00101.safetensors", + "model.layers.34.mlp.experts.64.up_proj.weight": "model-00036-of-00101.safetensors", + "model.layers.34.mlp.experts.64.down_proj.weight": "model-00036-of-00101.safetensors", + "model.layers.34.mlp.experts.65.gate_proj.weight": "model-00036-of-00101.safetensors", + "model.layers.34.mlp.experts.65.up_proj.weight": "model-00036-of-00101.safetensors", + "model.layers.34.mlp.experts.65.down_proj.weight": "model-00036-of-00101.safetensors", + "model.layers.34.mlp.experts.66.gate_proj.weight": "model-00036-of-00101.safetensors", + "model.layers.34.mlp.experts.66.up_proj.weight": "model-00036-of-00101.safetensors", + "model.layers.34.mlp.experts.66.down_proj.weight": "model-00036-of-00101.safetensors", + "model.layers.34.mlp.experts.67.gate_proj.weight": "model-00036-of-00101.safetensors", + "model.layers.34.mlp.experts.67.up_proj.weight": "model-00036-of-00101.safetensors", + "model.layers.34.mlp.experts.67.down_proj.weight": "model-00036-of-00101.safetensors", + "model.layers.34.mlp.experts.68.gate_proj.weight": "model-00036-of-00101.safetensors", + "model.layers.34.mlp.experts.68.up_proj.weight": "model-00036-of-00101.safetensors", + "model.layers.34.mlp.experts.68.down_proj.weight": "model-00036-of-00101.safetensors", + "model.layers.34.mlp.experts.69.gate_proj.weight": "model-00036-of-00101.safetensors", + "model.layers.34.mlp.experts.69.up_proj.weight": "model-00036-of-00101.safetensors", + "model.layers.34.mlp.experts.69.down_proj.weight": "model-00036-of-00101.safetensors", + "model.layers.34.mlp.experts.70.gate_proj.weight": "model-00036-of-00101.safetensors", + "model.layers.34.mlp.experts.70.up_proj.weight": "model-00036-of-00101.safetensors", + "model.layers.34.mlp.experts.70.down_proj.weight": "model-00036-of-00101.safetensors", + "model.layers.34.mlp.experts.71.gate_proj.weight": "model-00036-of-00101.safetensors", + "model.layers.34.mlp.experts.71.up_proj.weight": "model-00036-of-00101.safetensors", + "model.layers.34.mlp.experts.71.down_proj.weight": "model-00036-of-00101.safetensors", + "model.layers.34.mlp.experts.72.gate_proj.weight": "model-00036-of-00101.safetensors", + "model.layers.34.mlp.experts.72.up_proj.weight": "model-00036-of-00101.safetensors", + "model.layers.34.mlp.experts.72.down_proj.weight": "model-00036-of-00101.safetensors", + "model.layers.34.mlp.experts.73.gate_proj.weight": "model-00036-of-00101.safetensors", + "model.layers.34.mlp.experts.73.up_proj.weight": "model-00036-of-00101.safetensors", + "model.layers.34.mlp.experts.73.down_proj.weight": "model-00036-of-00101.safetensors", + "model.layers.34.mlp.experts.74.gate_proj.weight": "model-00036-of-00101.safetensors", + "model.layers.34.mlp.experts.74.up_proj.weight": "model-00036-of-00101.safetensors", + "model.layers.34.mlp.experts.74.down_proj.weight": "model-00037-of-00101.safetensors", + "model.layers.34.mlp.experts.75.gate_proj.weight": "model-00037-of-00101.safetensors", + "model.layers.34.mlp.experts.75.up_proj.weight": "model-00037-of-00101.safetensors", + "model.layers.34.mlp.experts.75.down_proj.weight": "model-00037-of-00101.safetensors", + "model.layers.34.mlp.experts.76.gate_proj.weight": "model-00037-of-00101.safetensors", + "model.layers.34.mlp.experts.76.up_proj.weight": "model-00037-of-00101.safetensors", + "model.layers.34.mlp.experts.76.down_proj.weight": "model-00037-of-00101.safetensors", + "model.layers.34.mlp.experts.77.gate_proj.weight": "model-00037-of-00101.safetensors", + "model.layers.34.mlp.experts.77.up_proj.weight": "model-00037-of-00101.safetensors", + "model.layers.34.mlp.experts.77.down_proj.weight": "model-00037-of-00101.safetensors", + "model.layers.34.mlp.experts.78.gate_proj.weight": "model-00037-of-00101.safetensors", + "model.layers.34.mlp.experts.78.up_proj.weight": "model-00037-of-00101.safetensors", + "model.layers.34.mlp.experts.78.down_proj.weight": "model-00037-of-00101.safetensors", + "model.layers.34.mlp.experts.79.gate_proj.weight": "model-00037-of-00101.safetensors", + "model.layers.34.mlp.experts.79.up_proj.weight": "model-00037-of-00101.safetensors", + "model.layers.34.mlp.experts.79.down_proj.weight": "model-00037-of-00101.safetensors", + "model.layers.34.mlp.experts.80.gate_proj.weight": "model-00037-of-00101.safetensors", + "model.layers.34.mlp.experts.80.up_proj.weight": "model-00037-of-00101.safetensors", + "model.layers.34.mlp.experts.80.down_proj.weight": "model-00037-of-00101.safetensors", + "model.layers.34.mlp.experts.81.gate_proj.weight": "model-00037-of-00101.safetensors", + "model.layers.34.mlp.experts.81.up_proj.weight": "model-00037-of-00101.safetensors", + "model.layers.34.mlp.experts.81.down_proj.weight": "model-00037-of-00101.safetensors", + "model.layers.34.mlp.experts.82.gate_proj.weight": "model-00037-of-00101.safetensors", + "model.layers.34.mlp.experts.82.up_proj.weight": "model-00037-of-00101.safetensors", + "model.layers.34.mlp.experts.82.down_proj.weight": "model-00037-of-00101.safetensors", + "model.layers.34.mlp.experts.83.gate_proj.weight": "model-00037-of-00101.safetensors", + "model.layers.34.mlp.experts.83.up_proj.weight": "model-00037-of-00101.safetensors", + "model.layers.34.mlp.experts.83.down_proj.weight": "model-00037-of-00101.safetensors", + "model.layers.34.mlp.experts.84.gate_proj.weight": "model-00037-of-00101.safetensors", + "model.layers.34.mlp.experts.84.up_proj.weight": "model-00037-of-00101.safetensors", + "model.layers.34.mlp.experts.84.down_proj.weight": "model-00037-of-00101.safetensors", + "model.layers.34.mlp.experts.85.gate_proj.weight": "model-00037-of-00101.safetensors", + "model.layers.34.mlp.experts.85.up_proj.weight": "model-00037-of-00101.safetensors", + "model.layers.34.mlp.experts.85.down_proj.weight": "model-00037-of-00101.safetensors", + "model.layers.34.mlp.experts.86.gate_proj.weight": "model-00037-of-00101.safetensors", + "model.layers.34.mlp.experts.86.up_proj.weight": "model-00037-of-00101.safetensors", + "model.layers.34.mlp.experts.86.down_proj.weight": "model-00037-of-00101.safetensors", + "model.layers.34.mlp.experts.87.gate_proj.weight": "model-00037-of-00101.safetensors", + "model.layers.34.mlp.experts.87.up_proj.weight": "model-00037-of-00101.safetensors", + "model.layers.34.mlp.experts.87.down_proj.weight": "model-00037-of-00101.safetensors", + "model.layers.34.mlp.experts.88.gate_proj.weight": "model-00037-of-00101.safetensors", + "model.layers.34.mlp.experts.88.up_proj.weight": "model-00037-of-00101.safetensors", + "model.layers.34.mlp.experts.88.down_proj.weight": "model-00037-of-00101.safetensors", + "model.layers.34.mlp.experts.89.gate_proj.weight": "model-00037-of-00101.safetensors", + "model.layers.34.mlp.experts.89.up_proj.weight": "model-00037-of-00101.safetensors", + "model.layers.34.mlp.experts.89.down_proj.weight": "model-00037-of-00101.safetensors", + "model.layers.34.mlp.experts.90.gate_proj.weight": "model-00037-of-00101.safetensors", + "model.layers.34.mlp.experts.90.up_proj.weight": "model-00037-of-00101.safetensors", + "model.layers.34.mlp.experts.90.down_proj.weight": "model-00037-of-00101.safetensors", + "model.layers.34.mlp.experts.91.gate_proj.weight": "model-00037-of-00101.safetensors", + "model.layers.34.mlp.experts.91.up_proj.weight": "model-00037-of-00101.safetensors", + "model.layers.34.mlp.experts.91.down_proj.weight": "model-00037-of-00101.safetensors", + "model.layers.34.mlp.experts.92.gate_proj.weight": "model-00037-of-00101.safetensors", + "model.layers.34.mlp.experts.92.up_proj.weight": "model-00037-of-00101.safetensors", + "model.layers.34.mlp.experts.92.down_proj.weight": "model-00037-of-00101.safetensors", + "model.layers.34.mlp.experts.93.gate_proj.weight": "model-00037-of-00101.safetensors", + "model.layers.34.mlp.experts.93.up_proj.weight": "model-00037-of-00101.safetensors", + "model.layers.34.mlp.experts.93.down_proj.weight": "model-00037-of-00101.safetensors", + "model.layers.34.mlp.experts.94.gate_proj.weight": "model-00037-of-00101.safetensors", + "model.layers.34.mlp.experts.94.up_proj.weight": "model-00037-of-00101.safetensors", + "model.layers.34.mlp.experts.94.down_proj.weight": "model-00037-of-00101.safetensors", + "model.layers.34.mlp.experts.95.gate_proj.weight": "model-00037-of-00101.safetensors", + "model.layers.34.mlp.experts.95.up_proj.weight": "model-00037-of-00101.safetensors", + "model.layers.34.mlp.experts.95.down_proj.weight": "model-00037-of-00101.safetensors", + "model.layers.34.mlp.experts.96.gate_proj.weight": "model-00037-of-00101.safetensors", + "model.layers.34.mlp.experts.96.up_proj.weight": "model-00037-of-00101.safetensors", + "model.layers.34.mlp.experts.96.down_proj.weight": "model-00037-of-00101.safetensors", + "model.layers.34.mlp.experts.97.gate_proj.weight": "model-00037-of-00101.safetensors", + "model.layers.34.mlp.experts.97.up_proj.weight": "model-00037-of-00101.safetensors", + "model.layers.34.mlp.experts.97.down_proj.weight": "model-00037-of-00101.safetensors", + "model.layers.34.mlp.experts.98.gate_proj.weight": "model-00037-of-00101.safetensors", + "model.layers.34.mlp.experts.98.up_proj.weight": "model-00037-of-00101.safetensors", + "model.layers.34.mlp.experts.98.down_proj.weight": "model-00037-of-00101.safetensors", + "model.layers.34.mlp.experts.99.gate_proj.weight": "model-00037-of-00101.safetensors", + "model.layers.34.mlp.experts.99.up_proj.weight": "model-00037-of-00101.safetensors", + "model.layers.34.mlp.experts.99.down_proj.weight": "model-00037-of-00101.safetensors", + "model.layers.34.mlp.experts.100.gate_proj.weight": "model-00037-of-00101.safetensors", + "model.layers.34.mlp.experts.100.up_proj.weight": "model-00037-of-00101.safetensors", + "model.layers.34.mlp.experts.100.down_proj.weight": "model-00037-of-00101.safetensors", + "model.layers.34.mlp.experts.101.gate_proj.weight": "model-00037-of-00101.safetensors", + "model.layers.34.mlp.experts.101.up_proj.weight": "model-00037-of-00101.safetensors", + "model.layers.34.mlp.experts.101.down_proj.weight": "model-00037-of-00101.safetensors", + "model.layers.34.mlp.experts.102.gate_proj.weight": "model-00037-of-00101.safetensors", + "model.layers.34.mlp.experts.102.up_proj.weight": "model-00037-of-00101.safetensors", + "model.layers.34.mlp.experts.102.down_proj.weight": "model-00037-of-00101.safetensors", + "model.layers.34.mlp.experts.103.gate_proj.weight": "model-00037-of-00101.safetensors", + "model.layers.34.mlp.experts.103.up_proj.weight": "model-00037-of-00101.safetensors", + "model.layers.34.mlp.experts.103.down_proj.weight": "model-00037-of-00101.safetensors", + "model.layers.34.mlp.experts.104.gate_proj.weight": "model-00037-of-00101.safetensors", + "model.layers.34.mlp.experts.104.up_proj.weight": "model-00037-of-00101.safetensors", + "model.layers.34.mlp.experts.104.down_proj.weight": "model-00037-of-00101.safetensors", + "model.layers.34.mlp.experts.105.gate_proj.weight": "model-00037-of-00101.safetensors", + "model.layers.34.mlp.experts.105.up_proj.weight": "model-00037-of-00101.safetensors", + "model.layers.34.mlp.experts.105.down_proj.weight": "model-00037-of-00101.safetensors", + "model.layers.34.mlp.experts.106.gate_proj.weight": "model-00037-of-00101.safetensors", + "model.layers.34.mlp.experts.106.up_proj.weight": "model-00037-of-00101.safetensors", + "model.layers.34.mlp.experts.106.down_proj.weight": "model-00037-of-00101.safetensors", + "model.layers.34.mlp.experts.107.gate_proj.weight": "model-00037-of-00101.safetensors", + "model.layers.34.mlp.experts.107.up_proj.weight": "model-00037-of-00101.safetensors", + "model.layers.34.mlp.experts.107.down_proj.weight": "model-00037-of-00101.safetensors", + "model.layers.34.mlp.experts.108.gate_proj.weight": "model-00037-of-00101.safetensors", + "model.layers.34.mlp.experts.108.up_proj.weight": "model-00037-of-00101.safetensors", + "model.layers.34.mlp.experts.108.down_proj.weight": "model-00037-of-00101.safetensors", + "model.layers.34.mlp.experts.109.gate_proj.weight": "model-00037-of-00101.safetensors", + "model.layers.34.mlp.experts.109.up_proj.weight": "model-00037-of-00101.safetensors", + "model.layers.34.mlp.experts.109.down_proj.weight": "model-00037-of-00101.safetensors", + "model.layers.34.mlp.experts.110.gate_proj.weight": "model-00037-of-00101.safetensors", + "model.layers.34.mlp.experts.110.up_proj.weight": "model-00037-of-00101.safetensors", + "model.layers.34.mlp.experts.110.down_proj.weight": "model-00037-of-00101.safetensors", + "model.layers.34.mlp.experts.111.gate_proj.weight": "model-00037-of-00101.safetensors", + "model.layers.34.mlp.experts.111.up_proj.weight": "model-00037-of-00101.safetensors", + "model.layers.34.mlp.experts.111.down_proj.weight": "model-00037-of-00101.safetensors", + "model.layers.34.mlp.experts.112.gate_proj.weight": "model-00037-of-00101.safetensors", + "model.layers.34.mlp.experts.112.up_proj.weight": "model-00037-of-00101.safetensors", + "model.layers.34.mlp.experts.112.down_proj.weight": "model-00037-of-00101.safetensors", + "model.layers.34.mlp.experts.113.gate_proj.weight": "model-00037-of-00101.safetensors", + "model.layers.34.mlp.experts.113.up_proj.weight": "model-00037-of-00101.safetensors", + "model.layers.34.mlp.experts.113.down_proj.weight": "model-00037-of-00101.safetensors", + "model.layers.34.mlp.experts.114.gate_proj.weight": "model-00037-of-00101.safetensors", + "model.layers.34.mlp.experts.114.up_proj.weight": "model-00037-of-00101.safetensors", + "model.layers.34.mlp.experts.114.down_proj.weight": "model-00037-of-00101.safetensors", + "model.layers.34.mlp.experts.115.gate_proj.weight": "model-00037-of-00101.safetensors", + "model.layers.34.mlp.experts.115.up_proj.weight": "model-00037-of-00101.safetensors", + "model.layers.34.mlp.experts.115.down_proj.weight": "model-00037-of-00101.safetensors", + "model.layers.34.mlp.experts.116.gate_proj.weight": "model-00037-of-00101.safetensors", + "model.layers.34.mlp.experts.116.up_proj.weight": "model-00037-of-00101.safetensors", + "model.layers.34.mlp.experts.116.down_proj.weight": "model-00037-of-00101.safetensors", + "model.layers.34.mlp.experts.117.gate_proj.weight": "model-00037-of-00101.safetensors", + "model.layers.34.mlp.experts.117.up_proj.weight": "model-00037-of-00101.safetensors", + "model.layers.34.mlp.experts.117.down_proj.weight": "model-00037-of-00101.safetensors", + "model.layers.34.mlp.experts.118.gate_proj.weight": "model-00037-of-00101.safetensors", + "model.layers.34.mlp.experts.118.up_proj.weight": "model-00037-of-00101.safetensors", + "model.layers.34.mlp.experts.118.down_proj.weight": "model-00037-of-00101.safetensors", + "model.layers.34.mlp.experts.119.gate_proj.weight": "model-00037-of-00101.safetensors", + "model.layers.34.mlp.experts.119.up_proj.weight": "model-00037-of-00101.safetensors", + "model.layers.34.mlp.experts.119.down_proj.weight": "model-00037-of-00101.safetensors", + "model.layers.34.mlp.gate.weight": "model-00037-of-00101.safetensors", + "model.layers.34.mlp.gate.e_score_correction_bias": "model-00037-of-00101.safetensors", + "model.layers.34.mlp.shared_experts.gate_proj.weight": "model-00037-of-00101.safetensors", + "model.layers.34.mlp.shared_experts.up_proj.weight": "model-00037-of-00101.safetensors", + "model.layers.34.mlp.shared_experts.down_proj.weight": "model-00037-of-00101.safetensors", + "model.layers.34.input_layernorm.weight": "model-00037-of-00101.safetensors", + "model.layers.34.post_attention_layernorm.weight": "model-00037-of-00101.safetensors", + "model.layers.35.self_attn.q_proj.weight": "model-00037-of-00101.safetensors", + "model.layers.35.self_attn.q_proj.bias": "model-00037-of-00101.safetensors", + "model.layers.35.self_attn.k_proj.weight": "model-00037-of-00101.safetensors", + "model.layers.35.self_attn.k_proj.bias": "model-00037-of-00101.safetensors", + "model.layers.35.self_attn.v_proj.weight": "model-00037-of-00101.safetensors", + "model.layers.35.self_attn.v_proj.bias": "model-00037-of-00101.safetensors", + "model.layers.35.self_attn.o_proj.weight": "model-00037-of-00101.safetensors", + "model.layers.35.self_attn.q_norm.weight": "model-00037-of-00101.safetensors", + "model.layers.35.self_attn.k_norm.weight": "model-00037-of-00101.safetensors", + "model.layers.35.mlp.experts.0.gate_proj.weight": "model-00037-of-00101.safetensors", + "model.layers.35.mlp.experts.0.up_proj.weight": "model-00037-of-00101.safetensors", + "model.layers.35.mlp.experts.0.down_proj.weight": "model-00037-of-00101.safetensors", + "model.layers.35.mlp.experts.1.gate_proj.weight": "model-00037-of-00101.safetensors", + "model.layers.35.mlp.experts.1.up_proj.weight": "model-00037-of-00101.safetensors", + "model.layers.35.mlp.experts.1.down_proj.weight": "model-00037-of-00101.safetensors", + "model.layers.35.mlp.experts.2.gate_proj.weight": "model-00037-of-00101.safetensors", + "model.layers.35.mlp.experts.2.up_proj.weight": "model-00037-of-00101.safetensors", + "model.layers.35.mlp.experts.2.down_proj.weight": "model-00037-of-00101.safetensors", + "model.layers.35.mlp.experts.3.gate_proj.weight": "model-00037-of-00101.safetensors", + "model.layers.35.mlp.experts.3.up_proj.weight": "model-00037-of-00101.safetensors", + "model.layers.35.mlp.experts.3.down_proj.weight": "model-00037-of-00101.safetensors", + "model.layers.35.mlp.experts.4.gate_proj.weight": "model-00037-of-00101.safetensors", + "model.layers.35.mlp.experts.4.up_proj.weight": "model-00037-of-00101.safetensors", + "model.layers.35.mlp.experts.4.down_proj.weight": "model-00037-of-00101.safetensors", + "model.layers.35.mlp.experts.5.gate_proj.weight": "model-00037-of-00101.safetensors", + "model.layers.35.mlp.experts.5.up_proj.weight": "model-00037-of-00101.safetensors", + "model.layers.35.mlp.experts.5.down_proj.weight": "model-00037-of-00101.safetensors", + "model.layers.35.mlp.experts.6.gate_proj.weight": "model-00037-of-00101.safetensors", + "model.layers.35.mlp.experts.6.up_proj.weight": "model-00037-of-00101.safetensors", + "model.layers.35.mlp.experts.6.down_proj.weight": "model-00037-of-00101.safetensors", + "model.layers.35.mlp.experts.7.gate_proj.weight": "model-00037-of-00101.safetensors", + "model.layers.35.mlp.experts.7.up_proj.weight": "model-00037-of-00101.safetensors", + "model.layers.35.mlp.experts.7.down_proj.weight": "model-00037-of-00101.safetensors", + "model.layers.35.mlp.experts.8.gate_proj.weight": "model-00037-of-00101.safetensors", + "model.layers.35.mlp.experts.8.up_proj.weight": "model-00037-of-00101.safetensors", + "model.layers.35.mlp.experts.8.down_proj.weight": "model-00037-of-00101.safetensors", + "model.layers.35.mlp.experts.9.gate_proj.weight": "model-00037-of-00101.safetensors", + "model.layers.35.mlp.experts.9.up_proj.weight": "model-00037-of-00101.safetensors", + "model.layers.35.mlp.experts.9.down_proj.weight": "model-00037-of-00101.safetensors", + "model.layers.35.mlp.experts.10.gate_proj.weight": "model-00037-of-00101.safetensors", + "model.layers.35.mlp.experts.10.up_proj.weight": "model-00037-of-00101.safetensors", + "model.layers.35.mlp.experts.10.down_proj.weight": "model-00037-of-00101.safetensors", + "model.layers.35.mlp.experts.11.gate_proj.weight": "model-00037-of-00101.safetensors", + "model.layers.35.mlp.experts.11.up_proj.weight": "model-00037-of-00101.safetensors", + "model.layers.35.mlp.experts.11.down_proj.weight": "model-00037-of-00101.safetensors", + "model.layers.35.mlp.experts.12.gate_proj.weight": "model-00037-of-00101.safetensors", + "model.layers.35.mlp.experts.12.up_proj.weight": "model-00037-of-00101.safetensors", + "model.layers.35.mlp.experts.12.down_proj.weight": "model-00037-of-00101.safetensors", + "model.layers.35.mlp.experts.13.gate_proj.weight": "model-00037-of-00101.safetensors", + "model.layers.35.mlp.experts.13.up_proj.weight": "model-00037-of-00101.safetensors", + "model.layers.35.mlp.experts.13.down_proj.weight": "model-00037-of-00101.safetensors", + "model.layers.35.mlp.experts.14.gate_proj.weight": "model-00037-of-00101.safetensors", + "model.layers.35.mlp.experts.14.up_proj.weight": "model-00037-of-00101.safetensors", + "model.layers.35.mlp.experts.14.down_proj.weight": "model-00037-of-00101.safetensors", + "model.layers.35.mlp.experts.15.gate_proj.weight": "model-00037-of-00101.safetensors", + "model.layers.35.mlp.experts.15.up_proj.weight": "model-00037-of-00101.safetensors", + "model.layers.35.mlp.experts.15.down_proj.weight": "model-00037-of-00101.safetensors", + "model.layers.35.mlp.experts.16.gate_proj.weight": "model-00037-of-00101.safetensors", + "model.layers.35.mlp.experts.16.up_proj.weight": "model-00037-of-00101.safetensors", + "model.layers.35.mlp.experts.16.down_proj.weight": "model-00037-of-00101.safetensors", + "model.layers.35.mlp.experts.17.gate_proj.weight": "model-00037-of-00101.safetensors", + "model.layers.35.mlp.experts.17.up_proj.weight": "model-00037-of-00101.safetensors", + "model.layers.35.mlp.experts.17.down_proj.weight": "model-00037-of-00101.safetensors", + "model.layers.35.mlp.experts.18.gate_proj.weight": "model-00037-of-00101.safetensors", + "model.layers.35.mlp.experts.18.up_proj.weight": "model-00037-of-00101.safetensors", + "model.layers.35.mlp.experts.18.down_proj.weight": "model-00037-of-00101.safetensors", + "model.layers.35.mlp.experts.19.gate_proj.weight": "model-00037-of-00101.safetensors", + "model.layers.35.mlp.experts.19.up_proj.weight": "model-00037-of-00101.safetensors", + "model.layers.35.mlp.experts.19.down_proj.weight": "model-00037-of-00101.safetensors", + "model.layers.35.mlp.experts.20.gate_proj.weight": "model-00037-of-00101.safetensors", + "model.layers.35.mlp.experts.20.up_proj.weight": "model-00037-of-00101.safetensors", + "model.layers.35.mlp.experts.20.down_proj.weight": "model-00037-of-00101.safetensors", + "model.layers.35.mlp.experts.21.gate_proj.weight": "model-00037-of-00101.safetensors", + "model.layers.35.mlp.experts.21.up_proj.weight": "model-00037-of-00101.safetensors", + "model.layers.35.mlp.experts.21.down_proj.weight": "model-00037-of-00101.safetensors", + "model.layers.35.mlp.experts.22.gate_proj.weight": "model-00037-of-00101.safetensors", + "model.layers.35.mlp.experts.22.up_proj.weight": "model-00037-of-00101.safetensors", + "model.layers.35.mlp.experts.22.down_proj.weight": "model-00037-of-00101.safetensors", + "model.layers.35.mlp.experts.23.gate_proj.weight": "model-00037-of-00101.safetensors", + "model.layers.35.mlp.experts.23.up_proj.weight": "model-00037-of-00101.safetensors", + "model.layers.35.mlp.experts.23.down_proj.weight": "model-00037-of-00101.safetensors", + "model.layers.35.mlp.experts.24.gate_proj.weight": "model-00037-of-00101.safetensors", + "model.layers.35.mlp.experts.24.up_proj.weight": "model-00037-of-00101.safetensors", + "model.layers.35.mlp.experts.24.down_proj.weight": "model-00037-of-00101.safetensors", + "model.layers.35.mlp.experts.25.gate_proj.weight": "model-00037-of-00101.safetensors", + "model.layers.35.mlp.experts.25.up_proj.weight": "model-00037-of-00101.safetensors", + "model.layers.35.mlp.experts.25.down_proj.weight": "model-00037-of-00101.safetensors", + "model.layers.35.mlp.experts.26.gate_proj.weight": "model-00037-of-00101.safetensors", + "model.layers.35.mlp.experts.26.up_proj.weight": "model-00037-of-00101.safetensors", + "model.layers.35.mlp.experts.26.down_proj.weight": "model-00037-of-00101.safetensors", + "model.layers.35.mlp.experts.27.gate_proj.weight": "model-00037-of-00101.safetensors", + "model.layers.35.mlp.experts.27.up_proj.weight": "model-00037-of-00101.safetensors", + "model.layers.35.mlp.experts.27.down_proj.weight": "model-00037-of-00101.safetensors", + "model.layers.35.mlp.experts.28.gate_proj.weight": "model-00037-of-00101.safetensors", + "model.layers.35.mlp.experts.28.up_proj.weight": "model-00037-of-00101.safetensors", + "model.layers.35.mlp.experts.28.down_proj.weight": "model-00037-of-00101.safetensors", + "model.layers.35.mlp.experts.29.gate_proj.weight": "model-00037-of-00101.safetensors", + "model.layers.35.mlp.experts.29.up_proj.weight": "model-00037-of-00101.safetensors", + "model.layers.35.mlp.experts.29.down_proj.weight": "model-00037-of-00101.safetensors", + "model.layers.35.mlp.experts.30.gate_proj.weight": "model-00037-of-00101.safetensors", + "model.layers.35.mlp.experts.30.up_proj.weight": "model-00037-of-00101.safetensors", + "model.layers.35.mlp.experts.30.down_proj.weight": "model-00037-of-00101.safetensors", + "model.layers.35.mlp.experts.31.gate_proj.weight": "model-00037-of-00101.safetensors", + "model.layers.35.mlp.experts.31.up_proj.weight": "model-00037-of-00101.safetensors", + "model.layers.35.mlp.experts.31.down_proj.weight": "model-00037-of-00101.safetensors", + "model.layers.35.mlp.experts.32.gate_proj.weight": "model-00037-of-00101.safetensors", + "model.layers.35.mlp.experts.32.up_proj.weight": "model-00037-of-00101.safetensors", + "model.layers.35.mlp.experts.32.down_proj.weight": "model-00037-of-00101.safetensors", + "model.layers.35.mlp.experts.33.gate_proj.weight": "model-00037-of-00101.safetensors", + "model.layers.35.mlp.experts.33.up_proj.weight": "model-00037-of-00101.safetensors", + "model.layers.35.mlp.experts.33.down_proj.weight": "model-00037-of-00101.safetensors", + "model.layers.35.mlp.experts.34.gate_proj.weight": "model-00037-of-00101.safetensors", + "model.layers.35.mlp.experts.34.up_proj.weight": "model-00037-of-00101.safetensors", + "model.layers.35.mlp.experts.34.down_proj.weight": "model-00037-of-00101.safetensors", + "model.layers.35.mlp.experts.35.gate_proj.weight": "model-00037-of-00101.safetensors", + "model.layers.35.mlp.experts.35.up_proj.weight": "model-00037-of-00101.safetensors", + "model.layers.35.mlp.experts.35.down_proj.weight": "model-00037-of-00101.safetensors", + "model.layers.35.mlp.experts.36.gate_proj.weight": "model-00037-of-00101.safetensors", + "model.layers.35.mlp.experts.36.up_proj.weight": "model-00037-of-00101.safetensors", + "model.layers.35.mlp.experts.36.down_proj.weight": "model-00037-of-00101.safetensors", + "model.layers.35.mlp.experts.37.gate_proj.weight": "model-00037-of-00101.safetensors", + "model.layers.35.mlp.experts.37.up_proj.weight": "model-00037-of-00101.safetensors", + "model.layers.35.mlp.experts.37.down_proj.weight": "model-00037-of-00101.safetensors", + "model.layers.35.mlp.experts.38.gate_proj.weight": "model-00037-of-00101.safetensors", + "model.layers.35.mlp.experts.38.up_proj.weight": "model-00037-of-00101.safetensors", + "model.layers.35.mlp.experts.38.down_proj.weight": "model-00037-of-00101.safetensors", + "model.layers.35.mlp.experts.39.gate_proj.weight": "model-00037-of-00101.safetensors", + "model.layers.35.mlp.experts.39.up_proj.weight": "model-00037-of-00101.safetensors", + "model.layers.35.mlp.experts.39.down_proj.weight": "model-00037-of-00101.safetensors", + "model.layers.35.mlp.experts.40.gate_proj.weight": "model-00037-of-00101.safetensors", + "model.layers.35.mlp.experts.40.up_proj.weight": "model-00037-of-00101.safetensors", + "model.layers.35.mlp.experts.40.down_proj.weight": "model-00037-of-00101.safetensors", + "model.layers.35.mlp.experts.41.gate_proj.weight": "model-00037-of-00101.safetensors", + "model.layers.35.mlp.experts.41.up_proj.weight": "model-00037-of-00101.safetensors", + "model.layers.35.mlp.experts.41.down_proj.weight": "model-00037-of-00101.safetensors", + "model.layers.35.mlp.experts.42.gate_proj.weight": "model-00037-of-00101.safetensors", + "model.layers.35.mlp.experts.42.up_proj.weight": "model-00037-of-00101.safetensors", + "model.layers.35.mlp.experts.42.down_proj.weight": "model-00037-of-00101.safetensors", + "model.layers.35.mlp.experts.43.gate_proj.weight": "model-00037-of-00101.safetensors", + "model.layers.35.mlp.experts.43.up_proj.weight": "model-00037-of-00101.safetensors", + "model.layers.35.mlp.experts.43.down_proj.weight": "model-00037-of-00101.safetensors", + "model.layers.35.mlp.experts.44.gate_proj.weight": "model-00037-of-00101.safetensors", + "model.layers.35.mlp.experts.44.up_proj.weight": "model-00037-of-00101.safetensors", + "model.layers.35.mlp.experts.44.down_proj.weight": "model-00037-of-00101.safetensors", + "model.layers.35.mlp.experts.45.gate_proj.weight": "model-00037-of-00101.safetensors", + "model.layers.35.mlp.experts.45.up_proj.weight": "model-00037-of-00101.safetensors", + "model.layers.35.mlp.experts.45.down_proj.weight": "model-00037-of-00101.safetensors", + "model.layers.35.mlp.experts.46.gate_proj.weight": "model-00037-of-00101.safetensors", + "model.layers.35.mlp.experts.46.up_proj.weight": "model-00037-of-00101.safetensors", + "model.layers.35.mlp.experts.46.down_proj.weight": "model-00037-of-00101.safetensors", + "model.layers.35.mlp.experts.47.gate_proj.weight": "model-00037-of-00101.safetensors", + "model.layers.35.mlp.experts.47.up_proj.weight": "model-00037-of-00101.safetensors", + "model.layers.35.mlp.experts.47.down_proj.weight": "model-00037-of-00101.safetensors", + "model.layers.35.mlp.experts.48.gate_proj.weight": "model-00037-of-00101.safetensors", + "model.layers.35.mlp.experts.48.up_proj.weight": "model-00037-of-00101.safetensors", + "model.layers.35.mlp.experts.48.down_proj.weight": "model-00037-of-00101.safetensors", + "model.layers.35.mlp.experts.49.gate_proj.weight": "model-00037-of-00101.safetensors", + "model.layers.35.mlp.experts.49.up_proj.weight": "model-00037-of-00101.safetensors", + "model.layers.35.mlp.experts.49.down_proj.weight": "model-00037-of-00101.safetensors", + "model.layers.35.mlp.experts.50.gate_proj.weight": "model-00037-of-00101.safetensors", + "model.layers.35.mlp.experts.50.up_proj.weight": "model-00037-of-00101.safetensors", + "model.layers.35.mlp.experts.50.down_proj.weight": "model-00037-of-00101.safetensors", + "model.layers.35.mlp.experts.51.gate_proj.weight": "model-00037-of-00101.safetensors", + "model.layers.35.mlp.experts.51.up_proj.weight": "model-00037-of-00101.safetensors", + "model.layers.35.mlp.experts.51.down_proj.weight": "model-00037-of-00101.safetensors", + "model.layers.35.mlp.experts.52.gate_proj.weight": "model-00037-of-00101.safetensors", + "model.layers.35.mlp.experts.52.up_proj.weight": "model-00037-of-00101.safetensors", + "model.layers.35.mlp.experts.52.down_proj.weight": "model-00037-of-00101.safetensors", + "model.layers.35.mlp.experts.53.gate_proj.weight": "model-00037-of-00101.safetensors", + "model.layers.35.mlp.experts.53.up_proj.weight": "model-00037-of-00101.safetensors", + "model.layers.35.mlp.experts.53.down_proj.weight": "model-00037-of-00101.safetensors", + "model.layers.35.mlp.experts.54.gate_proj.weight": "model-00037-of-00101.safetensors", + "model.layers.35.mlp.experts.54.up_proj.weight": "model-00037-of-00101.safetensors", + "model.layers.35.mlp.experts.54.down_proj.weight": "model-00037-of-00101.safetensors", + "model.layers.35.mlp.experts.55.gate_proj.weight": "model-00037-of-00101.safetensors", + "model.layers.35.mlp.experts.55.up_proj.weight": "model-00037-of-00101.safetensors", + "model.layers.35.mlp.experts.55.down_proj.weight": "model-00037-of-00101.safetensors", + "model.layers.35.mlp.experts.56.gate_proj.weight": "model-00037-of-00101.safetensors", + "model.layers.35.mlp.experts.56.up_proj.weight": "model-00037-of-00101.safetensors", + "model.layers.35.mlp.experts.56.down_proj.weight": "model-00037-of-00101.safetensors", + "model.layers.35.mlp.experts.57.gate_proj.weight": "model-00037-of-00101.safetensors", + "model.layers.35.mlp.experts.57.up_proj.weight": "model-00037-of-00101.safetensors", + "model.layers.35.mlp.experts.57.down_proj.weight": "model-00037-of-00101.safetensors", + "model.layers.35.mlp.experts.58.gate_proj.weight": "model-00037-of-00101.safetensors", + "model.layers.35.mlp.experts.58.up_proj.weight": "model-00037-of-00101.safetensors", + "model.layers.35.mlp.experts.58.down_proj.weight": "model-00037-of-00101.safetensors", + "model.layers.35.mlp.experts.59.gate_proj.weight": "model-00037-of-00101.safetensors", + "model.layers.35.mlp.experts.59.up_proj.weight": "model-00037-of-00101.safetensors", + "model.layers.35.mlp.experts.59.down_proj.weight": "model-00037-of-00101.safetensors", + "model.layers.35.mlp.experts.60.gate_proj.weight": "model-00037-of-00101.safetensors", + "model.layers.35.mlp.experts.60.up_proj.weight": "model-00037-of-00101.safetensors", + "model.layers.35.mlp.experts.60.down_proj.weight": "model-00037-of-00101.safetensors", + "model.layers.35.mlp.experts.61.gate_proj.weight": "model-00037-of-00101.safetensors", + "model.layers.35.mlp.experts.61.up_proj.weight": "model-00038-of-00101.safetensors", + "model.layers.35.mlp.experts.61.down_proj.weight": "model-00038-of-00101.safetensors", + "model.layers.35.mlp.experts.62.gate_proj.weight": "model-00038-of-00101.safetensors", + "model.layers.35.mlp.experts.62.up_proj.weight": "model-00038-of-00101.safetensors", + "model.layers.35.mlp.experts.62.down_proj.weight": "model-00038-of-00101.safetensors", + "model.layers.35.mlp.experts.63.gate_proj.weight": "model-00038-of-00101.safetensors", + "model.layers.35.mlp.experts.63.up_proj.weight": "model-00038-of-00101.safetensors", + "model.layers.35.mlp.experts.63.down_proj.weight": "model-00038-of-00101.safetensors", + "model.layers.35.mlp.experts.64.gate_proj.weight": "model-00038-of-00101.safetensors", + "model.layers.35.mlp.experts.64.up_proj.weight": "model-00038-of-00101.safetensors", + "model.layers.35.mlp.experts.64.down_proj.weight": "model-00038-of-00101.safetensors", + "model.layers.35.mlp.experts.65.gate_proj.weight": "model-00038-of-00101.safetensors", + "model.layers.35.mlp.experts.65.up_proj.weight": "model-00038-of-00101.safetensors", + "model.layers.35.mlp.experts.65.down_proj.weight": "model-00038-of-00101.safetensors", + "model.layers.35.mlp.experts.66.gate_proj.weight": "model-00038-of-00101.safetensors", + "model.layers.35.mlp.experts.66.up_proj.weight": "model-00038-of-00101.safetensors", + "model.layers.35.mlp.experts.66.down_proj.weight": "model-00038-of-00101.safetensors", + "model.layers.35.mlp.experts.67.gate_proj.weight": "model-00038-of-00101.safetensors", + "model.layers.35.mlp.experts.67.up_proj.weight": "model-00038-of-00101.safetensors", + "model.layers.35.mlp.experts.67.down_proj.weight": "model-00038-of-00101.safetensors", + "model.layers.35.mlp.experts.68.gate_proj.weight": "model-00038-of-00101.safetensors", + "model.layers.35.mlp.experts.68.up_proj.weight": "model-00038-of-00101.safetensors", + "model.layers.35.mlp.experts.68.down_proj.weight": "model-00038-of-00101.safetensors", + "model.layers.35.mlp.experts.69.gate_proj.weight": "model-00038-of-00101.safetensors", + "model.layers.35.mlp.experts.69.up_proj.weight": "model-00038-of-00101.safetensors", + "model.layers.35.mlp.experts.69.down_proj.weight": "model-00038-of-00101.safetensors", + "model.layers.35.mlp.experts.70.gate_proj.weight": "model-00038-of-00101.safetensors", + "model.layers.35.mlp.experts.70.up_proj.weight": "model-00038-of-00101.safetensors", + "model.layers.35.mlp.experts.70.down_proj.weight": "model-00038-of-00101.safetensors", + "model.layers.35.mlp.experts.71.gate_proj.weight": "model-00038-of-00101.safetensors", + "model.layers.35.mlp.experts.71.up_proj.weight": "model-00038-of-00101.safetensors", + "model.layers.35.mlp.experts.71.down_proj.weight": "model-00038-of-00101.safetensors", + "model.layers.35.mlp.experts.72.gate_proj.weight": "model-00038-of-00101.safetensors", + "model.layers.35.mlp.experts.72.up_proj.weight": "model-00038-of-00101.safetensors", + "model.layers.35.mlp.experts.72.down_proj.weight": "model-00038-of-00101.safetensors", + "model.layers.35.mlp.experts.73.gate_proj.weight": "model-00038-of-00101.safetensors", + "model.layers.35.mlp.experts.73.up_proj.weight": "model-00038-of-00101.safetensors", + "model.layers.35.mlp.experts.73.down_proj.weight": "model-00038-of-00101.safetensors", + "model.layers.35.mlp.experts.74.gate_proj.weight": "model-00038-of-00101.safetensors", + "model.layers.35.mlp.experts.74.up_proj.weight": "model-00038-of-00101.safetensors", + "model.layers.35.mlp.experts.74.down_proj.weight": "model-00038-of-00101.safetensors", + "model.layers.35.mlp.experts.75.gate_proj.weight": "model-00038-of-00101.safetensors", + "model.layers.35.mlp.experts.75.up_proj.weight": "model-00038-of-00101.safetensors", + "model.layers.35.mlp.experts.75.down_proj.weight": "model-00038-of-00101.safetensors", + "model.layers.35.mlp.experts.76.gate_proj.weight": "model-00038-of-00101.safetensors", + "model.layers.35.mlp.experts.76.up_proj.weight": "model-00038-of-00101.safetensors", + "model.layers.35.mlp.experts.76.down_proj.weight": "model-00038-of-00101.safetensors", + "model.layers.35.mlp.experts.77.gate_proj.weight": "model-00038-of-00101.safetensors", + "model.layers.35.mlp.experts.77.up_proj.weight": "model-00038-of-00101.safetensors", + "model.layers.35.mlp.experts.77.down_proj.weight": "model-00038-of-00101.safetensors", + "model.layers.35.mlp.experts.78.gate_proj.weight": "model-00038-of-00101.safetensors", + "model.layers.35.mlp.experts.78.up_proj.weight": "model-00038-of-00101.safetensors", + "model.layers.35.mlp.experts.78.down_proj.weight": "model-00038-of-00101.safetensors", + "model.layers.35.mlp.experts.79.gate_proj.weight": "model-00038-of-00101.safetensors", + "model.layers.35.mlp.experts.79.up_proj.weight": "model-00038-of-00101.safetensors", + "model.layers.35.mlp.experts.79.down_proj.weight": "model-00038-of-00101.safetensors", + "model.layers.35.mlp.experts.80.gate_proj.weight": "model-00038-of-00101.safetensors", + "model.layers.35.mlp.experts.80.up_proj.weight": "model-00038-of-00101.safetensors", + "model.layers.35.mlp.experts.80.down_proj.weight": "model-00038-of-00101.safetensors", + "model.layers.35.mlp.experts.81.gate_proj.weight": "model-00038-of-00101.safetensors", + "model.layers.35.mlp.experts.81.up_proj.weight": "model-00038-of-00101.safetensors", + "model.layers.35.mlp.experts.81.down_proj.weight": "model-00038-of-00101.safetensors", + "model.layers.35.mlp.experts.82.gate_proj.weight": "model-00038-of-00101.safetensors", + "model.layers.35.mlp.experts.82.up_proj.weight": "model-00038-of-00101.safetensors", + "model.layers.35.mlp.experts.82.down_proj.weight": "model-00038-of-00101.safetensors", + "model.layers.35.mlp.experts.83.gate_proj.weight": "model-00038-of-00101.safetensors", + "model.layers.35.mlp.experts.83.up_proj.weight": "model-00038-of-00101.safetensors", + "model.layers.35.mlp.experts.83.down_proj.weight": "model-00038-of-00101.safetensors", + "model.layers.35.mlp.experts.84.gate_proj.weight": "model-00038-of-00101.safetensors", + "model.layers.35.mlp.experts.84.up_proj.weight": "model-00038-of-00101.safetensors", + "model.layers.35.mlp.experts.84.down_proj.weight": "model-00038-of-00101.safetensors", + "model.layers.35.mlp.experts.85.gate_proj.weight": "model-00038-of-00101.safetensors", + "model.layers.35.mlp.experts.85.up_proj.weight": "model-00038-of-00101.safetensors", + "model.layers.35.mlp.experts.85.down_proj.weight": "model-00038-of-00101.safetensors", + "model.layers.35.mlp.experts.86.gate_proj.weight": "model-00038-of-00101.safetensors", + "model.layers.35.mlp.experts.86.up_proj.weight": "model-00038-of-00101.safetensors", + "model.layers.35.mlp.experts.86.down_proj.weight": "model-00038-of-00101.safetensors", + "model.layers.35.mlp.experts.87.gate_proj.weight": "model-00038-of-00101.safetensors", + "model.layers.35.mlp.experts.87.up_proj.weight": "model-00038-of-00101.safetensors", + "model.layers.35.mlp.experts.87.down_proj.weight": "model-00038-of-00101.safetensors", + "model.layers.35.mlp.experts.88.gate_proj.weight": "model-00038-of-00101.safetensors", + "model.layers.35.mlp.experts.88.up_proj.weight": "model-00038-of-00101.safetensors", + "model.layers.35.mlp.experts.88.down_proj.weight": "model-00038-of-00101.safetensors", + "model.layers.35.mlp.experts.89.gate_proj.weight": "model-00038-of-00101.safetensors", + "model.layers.35.mlp.experts.89.up_proj.weight": "model-00038-of-00101.safetensors", + "model.layers.35.mlp.experts.89.down_proj.weight": "model-00038-of-00101.safetensors", + "model.layers.35.mlp.experts.90.gate_proj.weight": "model-00038-of-00101.safetensors", + "model.layers.35.mlp.experts.90.up_proj.weight": "model-00038-of-00101.safetensors", + "model.layers.35.mlp.experts.90.down_proj.weight": "model-00038-of-00101.safetensors", + "model.layers.35.mlp.experts.91.gate_proj.weight": "model-00038-of-00101.safetensors", + "model.layers.35.mlp.experts.91.up_proj.weight": "model-00038-of-00101.safetensors", + "model.layers.35.mlp.experts.91.down_proj.weight": "model-00038-of-00101.safetensors", + "model.layers.35.mlp.experts.92.gate_proj.weight": "model-00038-of-00101.safetensors", + "model.layers.35.mlp.experts.92.up_proj.weight": "model-00038-of-00101.safetensors", + "model.layers.35.mlp.experts.92.down_proj.weight": "model-00038-of-00101.safetensors", + "model.layers.35.mlp.experts.93.gate_proj.weight": "model-00038-of-00101.safetensors", + "model.layers.35.mlp.experts.93.up_proj.weight": "model-00038-of-00101.safetensors", + "model.layers.35.mlp.experts.93.down_proj.weight": "model-00038-of-00101.safetensors", + "model.layers.35.mlp.experts.94.gate_proj.weight": "model-00038-of-00101.safetensors", + "model.layers.35.mlp.experts.94.up_proj.weight": "model-00038-of-00101.safetensors", + "model.layers.35.mlp.experts.94.down_proj.weight": "model-00038-of-00101.safetensors", + "model.layers.35.mlp.experts.95.gate_proj.weight": "model-00038-of-00101.safetensors", + "model.layers.35.mlp.experts.95.up_proj.weight": "model-00038-of-00101.safetensors", + "model.layers.35.mlp.experts.95.down_proj.weight": "model-00038-of-00101.safetensors", + "model.layers.35.mlp.experts.96.gate_proj.weight": "model-00038-of-00101.safetensors", + "model.layers.35.mlp.experts.96.up_proj.weight": "model-00038-of-00101.safetensors", + "model.layers.35.mlp.experts.96.down_proj.weight": "model-00038-of-00101.safetensors", + "model.layers.35.mlp.experts.97.gate_proj.weight": "model-00038-of-00101.safetensors", + "model.layers.35.mlp.experts.97.up_proj.weight": "model-00038-of-00101.safetensors", + "model.layers.35.mlp.experts.97.down_proj.weight": "model-00038-of-00101.safetensors", + "model.layers.35.mlp.experts.98.gate_proj.weight": "model-00038-of-00101.safetensors", + "model.layers.35.mlp.experts.98.up_proj.weight": "model-00038-of-00101.safetensors", + "model.layers.35.mlp.experts.98.down_proj.weight": "model-00038-of-00101.safetensors", + "model.layers.35.mlp.experts.99.gate_proj.weight": "model-00038-of-00101.safetensors", + "model.layers.35.mlp.experts.99.up_proj.weight": "model-00038-of-00101.safetensors", + "model.layers.35.mlp.experts.99.down_proj.weight": "model-00038-of-00101.safetensors", + "model.layers.35.mlp.experts.100.gate_proj.weight": "model-00038-of-00101.safetensors", + "model.layers.35.mlp.experts.100.up_proj.weight": "model-00038-of-00101.safetensors", + "model.layers.35.mlp.experts.100.down_proj.weight": "model-00038-of-00101.safetensors", + "model.layers.35.mlp.experts.101.gate_proj.weight": "model-00038-of-00101.safetensors", + "model.layers.35.mlp.experts.101.up_proj.weight": "model-00038-of-00101.safetensors", + "model.layers.35.mlp.experts.101.down_proj.weight": "model-00038-of-00101.safetensors", + "model.layers.35.mlp.experts.102.gate_proj.weight": "model-00038-of-00101.safetensors", + "model.layers.35.mlp.experts.102.up_proj.weight": "model-00038-of-00101.safetensors", + "model.layers.35.mlp.experts.102.down_proj.weight": "model-00038-of-00101.safetensors", + "model.layers.35.mlp.experts.103.gate_proj.weight": "model-00038-of-00101.safetensors", + "model.layers.35.mlp.experts.103.up_proj.weight": "model-00038-of-00101.safetensors", + "model.layers.35.mlp.experts.103.down_proj.weight": "model-00038-of-00101.safetensors", + "model.layers.35.mlp.experts.104.gate_proj.weight": "model-00038-of-00101.safetensors", + "model.layers.35.mlp.experts.104.up_proj.weight": "model-00038-of-00101.safetensors", + "model.layers.35.mlp.experts.104.down_proj.weight": "model-00038-of-00101.safetensors", + "model.layers.35.mlp.experts.105.gate_proj.weight": "model-00038-of-00101.safetensors", + "model.layers.35.mlp.experts.105.up_proj.weight": "model-00038-of-00101.safetensors", + "model.layers.35.mlp.experts.105.down_proj.weight": "model-00038-of-00101.safetensors", + "model.layers.35.mlp.experts.106.gate_proj.weight": "model-00038-of-00101.safetensors", + "model.layers.35.mlp.experts.106.up_proj.weight": "model-00038-of-00101.safetensors", + "model.layers.35.mlp.experts.106.down_proj.weight": "model-00038-of-00101.safetensors", + "model.layers.35.mlp.experts.107.gate_proj.weight": "model-00038-of-00101.safetensors", + "model.layers.35.mlp.experts.107.up_proj.weight": "model-00038-of-00101.safetensors", + "model.layers.35.mlp.experts.107.down_proj.weight": "model-00038-of-00101.safetensors", + "model.layers.35.mlp.experts.108.gate_proj.weight": "model-00038-of-00101.safetensors", + "model.layers.35.mlp.experts.108.up_proj.weight": "model-00038-of-00101.safetensors", + "model.layers.35.mlp.experts.108.down_proj.weight": "model-00038-of-00101.safetensors", + "model.layers.35.mlp.experts.109.gate_proj.weight": "model-00038-of-00101.safetensors", + "model.layers.35.mlp.experts.109.up_proj.weight": "model-00038-of-00101.safetensors", + "model.layers.35.mlp.experts.109.down_proj.weight": "model-00038-of-00101.safetensors", + "model.layers.35.mlp.experts.110.gate_proj.weight": "model-00038-of-00101.safetensors", + "model.layers.35.mlp.experts.110.up_proj.weight": "model-00038-of-00101.safetensors", + "model.layers.35.mlp.experts.110.down_proj.weight": "model-00038-of-00101.safetensors", + "model.layers.35.mlp.experts.111.gate_proj.weight": "model-00038-of-00101.safetensors", + "model.layers.35.mlp.experts.111.up_proj.weight": "model-00038-of-00101.safetensors", + "model.layers.35.mlp.experts.111.down_proj.weight": "model-00038-of-00101.safetensors", + "model.layers.35.mlp.experts.112.gate_proj.weight": "model-00038-of-00101.safetensors", + "model.layers.35.mlp.experts.112.up_proj.weight": "model-00038-of-00101.safetensors", + "model.layers.35.mlp.experts.112.down_proj.weight": "model-00038-of-00101.safetensors", + "model.layers.35.mlp.experts.113.gate_proj.weight": "model-00038-of-00101.safetensors", + "model.layers.35.mlp.experts.113.up_proj.weight": "model-00038-of-00101.safetensors", + "model.layers.35.mlp.experts.113.down_proj.weight": "model-00038-of-00101.safetensors", + "model.layers.35.mlp.experts.114.gate_proj.weight": "model-00038-of-00101.safetensors", + "model.layers.35.mlp.experts.114.up_proj.weight": "model-00038-of-00101.safetensors", + "model.layers.35.mlp.experts.114.down_proj.weight": "model-00038-of-00101.safetensors", + "model.layers.35.mlp.experts.115.gate_proj.weight": "model-00038-of-00101.safetensors", + "model.layers.35.mlp.experts.115.up_proj.weight": "model-00038-of-00101.safetensors", + "model.layers.35.mlp.experts.115.down_proj.weight": "model-00038-of-00101.safetensors", + "model.layers.35.mlp.experts.116.gate_proj.weight": "model-00038-of-00101.safetensors", + "model.layers.35.mlp.experts.116.up_proj.weight": "model-00038-of-00101.safetensors", + "model.layers.35.mlp.experts.116.down_proj.weight": "model-00038-of-00101.safetensors", + "model.layers.35.mlp.experts.117.gate_proj.weight": "model-00038-of-00101.safetensors", + "model.layers.35.mlp.experts.117.up_proj.weight": "model-00038-of-00101.safetensors", + "model.layers.35.mlp.experts.117.down_proj.weight": "model-00038-of-00101.safetensors", + "model.layers.35.mlp.experts.118.gate_proj.weight": "model-00038-of-00101.safetensors", + "model.layers.35.mlp.experts.118.up_proj.weight": "model-00038-of-00101.safetensors", + "model.layers.35.mlp.experts.118.down_proj.weight": "model-00038-of-00101.safetensors", + "model.layers.35.mlp.experts.119.gate_proj.weight": "model-00038-of-00101.safetensors", + "model.layers.35.mlp.experts.119.up_proj.weight": "model-00038-of-00101.safetensors", + "model.layers.35.mlp.experts.119.down_proj.weight": "model-00038-of-00101.safetensors", + "model.layers.35.mlp.gate.weight": "model-00038-of-00101.safetensors", + "model.layers.35.mlp.gate.e_score_correction_bias": "model-00038-of-00101.safetensors", + "model.layers.35.mlp.shared_experts.gate_proj.weight": "model-00038-of-00101.safetensors", + "model.layers.35.mlp.shared_experts.up_proj.weight": "model-00038-of-00101.safetensors", + "model.layers.35.mlp.shared_experts.down_proj.weight": "model-00038-of-00101.safetensors", + "model.layers.35.input_layernorm.weight": "model-00038-of-00101.safetensors", + "model.layers.35.post_attention_layernorm.weight": "model-00038-of-00101.safetensors", + "model.layers.36.self_attn.q_proj.weight": "model-00038-of-00101.safetensors", + "model.layers.36.self_attn.q_proj.bias": "model-00038-of-00101.safetensors", + "model.layers.36.self_attn.k_proj.weight": "model-00038-of-00101.safetensors", + "model.layers.36.self_attn.k_proj.bias": "model-00038-of-00101.safetensors", + "model.layers.36.self_attn.v_proj.weight": "model-00038-of-00101.safetensors", + "model.layers.36.self_attn.v_proj.bias": "model-00038-of-00101.safetensors", + "model.layers.36.self_attn.o_proj.weight": "model-00038-of-00101.safetensors", + "model.layers.36.self_attn.q_norm.weight": "model-00038-of-00101.safetensors", + "model.layers.36.self_attn.k_norm.weight": "model-00038-of-00101.safetensors", + "model.layers.36.mlp.experts.0.gate_proj.weight": "model-00038-of-00101.safetensors", + "model.layers.36.mlp.experts.0.up_proj.weight": "model-00038-of-00101.safetensors", + "model.layers.36.mlp.experts.0.down_proj.weight": "model-00038-of-00101.safetensors", + "model.layers.36.mlp.experts.1.gate_proj.weight": "model-00038-of-00101.safetensors", + "model.layers.36.mlp.experts.1.up_proj.weight": "model-00038-of-00101.safetensors", + "model.layers.36.mlp.experts.1.down_proj.weight": "model-00038-of-00101.safetensors", + "model.layers.36.mlp.experts.2.gate_proj.weight": "model-00038-of-00101.safetensors", + "model.layers.36.mlp.experts.2.up_proj.weight": "model-00038-of-00101.safetensors", + "model.layers.36.mlp.experts.2.down_proj.weight": "model-00038-of-00101.safetensors", + "model.layers.36.mlp.experts.3.gate_proj.weight": "model-00038-of-00101.safetensors", + "model.layers.36.mlp.experts.3.up_proj.weight": "model-00038-of-00101.safetensors", + "model.layers.36.mlp.experts.3.down_proj.weight": "model-00038-of-00101.safetensors", + "model.layers.36.mlp.experts.4.gate_proj.weight": "model-00038-of-00101.safetensors", + "model.layers.36.mlp.experts.4.up_proj.weight": "model-00038-of-00101.safetensors", + "model.layers.36.mlp.experts.4.down_proj.weight": "model-00038-of-00101.safetensors", + "model.layers.36.mlp.experts.5.gate_proj.weight": "model-00038-of-00101.safetensors", + "model.layers.36.mlp.experts.5.up_proj.weight": "model-00038-of-00101.safetensors", + "model.layers.36.mlp.experts.5.down_proj.weight": "model-00038-of-00101.safetensors", + "model.layers.36.mlp.experts.6.gate_proj.weight": "model-00038-of-00101.safetensors", + "model.layers.36.mlp.experts.6.up_proj.weight": "model-00038-of-00101.safetensors", + "model.layers.36.mlp.experts.6.down_proj.weight": "model-00038-of-00101.safetensors", + "model.layers.36.mlp.experts.7.gate_proj.weight": "model-00038-of-00101.safetensors", + "model.layers.36.mlp.experts.7.up_proj.weight": "model-00038-of-00101.safetensors", + "model.layers.36.mlp.experts.7.down_proj.weight": "model-00038-of-00101.safetensors", + "model.layers.36.mlp.experts.8.gate_proj.weight": "model-00038-of-00101.safetensors", + "model.layers.36.mlp.experts.8.up_proj.weight": "model-00038-of-00101.safetensors", + "model.layers.36.mlp.experts.8.down_proj.weight": "model-00038-of-00101.safetensors", + "model.layers.36.mlp.experts.9.gate_proj.weight": "model-00038-of-00101.safetensors", + "model.layers.36.mlp.experts.9.up_proj.weight": "model-00038-of-00101.safetensors", + "model.layers.36.mlp.experts.9.down_proj.weight": "model-00038-of-00101.safetensors", + "model.layers.36.mlp.experts.10.gate_proj.weight": "model-00038-of-00101.safetensors", + "model.layers.36.mlp.experts.10.up_proj.weight": "model-00038-of-00101.safetensors", + "model.layers.36.mlp.experts.10.down_proj.weight": "model-00038-of-00101.safetensors", + "model.layers.36.mlp.experts.11.gate_proj.weight": "model-00038-of-00101.safetensors", + "model.layers.36.mlp.experts.11.up_proj.weight": "model-00038-of-00101.safetensors", + "model.layers.36.mlp.experts.11.down_proj.weight": "model-00038-of-00101.safetensors", + "model.layers.36.mlp.experts.12.gate_proj.weight": "model-00038-of-00101.safetensors", + "model.layers.36.mlp.experts.12.up_proj.weight": "model-00038-of-00101.safetensors", + "model.layers.36.mlp.experts.12.down_proj.weight": "model-00038-of-00101.safetensors", + "model.layers.36.mlp.experts.13.gate_proj.weight": "model-00038-of-00101.safetensors", + "model.layers.36.mlp.experts.13.up_proj.weight": "model-00038-of-00101.safetensors", + "model.layers.36.mlp.experts.13.down_proj.weight": "model-00038-of-00101.safetensors", + "model.layers.36.mlp.experts.14.gate_proj.weight": "model-00038-of-00101.safetensors", + "model.layers.36.mlp.experts.14.up_proj.weight": "model-00038-of-00101.safetensors", + "model.layers.36.mlp.experts.14.down_proj.weight": "model-00038-of-00101.safetensors", + "model.layers.36.mlp.experts.15.gate_proj.weight": "model-00038-of-00101.safetensors", + "model.layers.36.mlp.experts.15.up_proj.weight": "model-00038-of-00101.safetensors", + "model.layers.36.mlp.experts.15.down_proj.weight": "model-00038-of-00101.safetensors", + "model.layers.36.mlp.experts.16.gate_proj.weight": "model-00038-of-00101.safetensors", + "model.layers.36.mlp.experts.16.up_proj.weight": "model-00038-of-00101.safetensors", + "model.layers.36.mlp.experts.16.down_proj.weight": "model-00038-of-00101.safetensors", + "model.layers.36.mlp.experts.17.gate_proj.weight": "model-00038-of-00101.safetensors", + "model.layers.36.mlp.experts.17.up_proj.weight": "model-00038-of-00101.safetensors", + "model.layers.36.mlp.experts.17.down_proj.weight": "model-00038-of-00101.safetensors", + "model.layers.36.mlp.experts.18.gate_proj.weight": "model-00038-of-00101.safetensors", + "model.layers.36.mlp.experts.18.up_proj.weight": "model-00038-of-00101.safetensors", + "model.layers.36.mlp.experts.18.down_proj.weight": "model-00038-of-00101.safetensors", + "model.layers.36.mlp.experts.19.gate_proj.weight": "model-00038-of-00101.safetensors", + "model.layers.36.mlp.experts.19.up_proj.weight": "model-00038-of-00101.safetensors", + "model.layers.36.mlp.experts.19.down_proj.weight": "model-00038-of-00101.safetensors", + "model.layers.36.mlp.experts.20.gate_proj.weight": "model-00038-of-00101.safetensors", + "model.layers.36.mlp.experts.20.up_proj.weight": "model-00038-of-00101.safetensors", + "model.layers.36.mlp.experts.20.down_proj.weight": "model-00038-of-00101.safetensors", + "model.layers.36.mlp.experts.21.gate_proj.weight": "model-00038-of-00101.safetensors", + "model.layers.36.mlp.experts.21.up_proj.weight": "model-00038-of-00101.safetensors", + "model.layers.36.mlp.experts.21.down_proj.weight": "model-00038-of-00101.safetensors", + "model.layers.36.mlp.experts.22.gate_proj.weight": "model-00038-of-00101.safetensors", + "model.layers.36.mlp.experts.22.up_proj.weight": "model-00038-of-00101.safetensors", + "model.layers.36.mlp.experts.22.down_proj.weight": "model-00038-of-00101.safetensors", + "model.layers.36.mlp.experts.23.gate_proj.weight": "model-00038-of-00101.safetensors", + "model.layers.36.mlp.experts.23.up_proj.weight": "model-00038-of-00101.safetensors", + "model.layers.36.mlp.experts.23.down_proj.weight": "model-00038-of-00101.safetensors", + "model.layers.36.mlp.experts.24.gate_proj.weight": "model-00038-of-00101.safetensors", + "model.layers.36.mlp.experts.24.up_proj.weight": "model-00038-of-00101.safetensors", + "model.layers.36.mlp.experts.24.down_proj.weight": "model-00038-of-00101.safetensors", + "model.layers.36.mlp.experts.25.gate_proj.weight": "model-00038-of-00101.safetensors", + "model.layers.36.mlp.experts.25.up_proj.weight": "model-00038-of-00101.safetensors", + "model.layers.36.mlp.experts.25.down_proj.weight": "model-00038-of-00101.safetensors", + "model.layers.36.mlp.experts.26.gate_proj.weight": "model-00038-of-00101.safetensors", + "model.layers.36.mlp.experts.26.up_proj.weight": "model-00038-of-00101.safetensors", + "model.layers.36.mlp.experts.26.down_proj.weight": "model-00038-of-00101.safetensors", + "model.layers.36.mlp.experts.27.gate_proj.weight": "model-00038-of-00101.safetensors", + "model.layers.36.mlp.experts.27.up_proj.weight": "model-00038-of-00101.safetensors", + "model.layers.36.mlp.experts.27.down_proj.weight": "model-00038-of-00101.safetensors", + "model.layers.36.mlp.experts.28.gate_proj.weight": "model-00038-of-00101.safetensors", + "model.layers.36.mlp.experts.28.up_proj.weight": "model-00038-of-00101.safetensors", + "model.layers.36.mlp.experts.28.down_proj.weight": "model-00038-of-00101.safetensors", + "model.layers.36.mlp.experts.29.gate_proj.weight": "model-00038-of-00101.safetensors", + "model.layers.36.mlp.experts.29.up_proj.weight": "model-00038-of-00101.safetensors", + "model.layers.36.mlp.experts.29.down_proj.weight": "model-00038-of-00101.safetensors", + "model.layers.36.mlp.experts.30.gate_proj.weight": "model-00038-of-00101.safetensors", + "model.layers.36.mlp.experts.30.up_proj.weight": "model-00038-of-00101.safetensors", + "model.layers.36.mlp.experts.30.down_proj.weight": "model-00038-of-00101.safetensors", + "model.layers.36.mlp.experts.31.gate_proj.weight": "model-00038-of-00101.safetensors", + "model.layers.36.mlp.experts.31.up_proj.weight": "model-00038-of-00101.safetensors", + "model.layers.36.mlp.experts.31.down_proj.weight": "model-00038-of-00101.safetensors", + "model.layers.36.mlp.experts.32.gate_proj.weight": "model-00038-of-00101.safetensors", + "model.layers.36.mlp.experts.32.up_proj.weight": "model-00038-of-00101.safetensors", + "model.layers.36.mlp.experts.32.down_proj.weight": "model-00038-of-00101.safetensors", + "model.layers.36.mlp.experts.33.gate_proj.weight": "model-00038-of-00101.safetensors", + "model.layers.36.mlp.experts.33.up_proj.weight": "model-00038-of-00101.safetensors", + "model.layers.36.mlp.experts.33.down_proj.weight": "model-00038-of-00101.safetensors", + "model.layers.36.mlp.experts.34.gate_proj.weight": "model-00038-of-00101.safetensors", + "model.layers.36.mlp.experts.34.up_proj.weight": "model-00038-of-00101.safetensors", + "model.layers.36.mlp.experts.34.down_proj.weight": "model-00038-of-00101.safetensors", + "model.layers.36.mlp.experts.35.gate_proj.weight": "model-00038-of-00101.safetensors", + "model.layers.36.mlp.experts.35.up_proj.weight": "model-00038-of-00101.safetensors", + "model.layers.36.mlp.experts.35.down_proj.weight": "model-00038-of-00101.safetensors", + "model.layers.36.mlp.experts.36.gate_proj.weight": "model-00038-of-00101.safetensors", + "model.layers.36.mlp.experts.36.up_proj.weight": "model-00038-of-00101.safetensors", + "model.layers.36.mlp.experts.36.down_proj.weight": "model-00038-of-00101.safetensors", + "model.layers.36.mlp.experts.37.gate_proj.weight": "model-00038-of-00101.safetensors", + "model.layers.36.mlp.experts.37.up_proj.weight": "model-00038-of-00101.safetensors", + "model.layers.36.mlp.experts.37.down_proj.weight": "model-00038-of-00101.safetensors", + "model.layers.36.mlp.experts.38.gate_proj.weight": "model-00038-of-00101.safetensors", + "model.layers.36.mlp.experts.38.up_proj.weight": "model-00038-of-00101.safetensors", + "model.layers.36.mlp.experts.38.down_proj.weight": "model-00038-of-00101.safetensors", + "model.layers.36.mlp.experts.39.gate_proj.weight": "model-00038-of-00101.safetensors", + "model.layers.36.mlp.experts.39.up_proj.weight": "model-00038-of-00101.safetensors", + "model.layers.36.mlp.experts.39.down_proj.weight": "model-00038-of-00101.safetensors", + "model.layers.36.mlp.experts.40.gate_proj.weight": "model-00038-of-00101.safetensors", + "model.layers.36.mlp.experts.40.up_proj.weight": "model-00038-of-00101.safetensors", + "model.layers.36.mlp.experts.40.down_proj.weight": "model-00038-of-00101.safetensors", + "model.layers.36.mlp.experts.41.gate_proj.weight": "model-00038-of-00101.safetensors", + "model.layers.36.mlp.experts.41.up_proj.weight": "model-00038-of-00101.safetensors", + "model.layers.36.mlp.experts.41.down_proj.weight": "model-00038-of-00101.safetensors", + "model.layers.36.mlp.experts.42.gate_proj.weight": "model-00038-of-00101.safetensors", + "model.layers.36.mlp.experts.42.up_proj.weight": "model-00038-of-00101.safetensors", + "model.layers.36.mlp.experts.42.down_proj.weight": "model-00038-of-00101.safetensors", + "model.layers.36.mlp.experts.43.gate_proj.weight": "model-00038-of-00101.safetensors", + "model.layers.36.mlp.experts.43.up_proj.weight": "model-00038-of-00101.safetensors", + "model.layers.36.mlp.experts.43.down_proj.weight": "model-00038-of-00101.safetensors", + "model.layers.36.mlp.experts.44.gate_proj.weight": "model-00038-of-00101.safetensors", + "model.layers.36.mlp.experts.44.up_proj.weight": "model-00038-of-00101.safetensors", + "model.layers.36.mlp.experts.44.down_proj.weight": "model-00038-of-00101.safetensors", + "model.layers.36.mlp.experts.45.gate_proj.weight": "model-00038-of-00101.safetensors", + "model.layers.36.mlp.experts.45.up_proj.weight": "model-00038-of-00101.safetensors", + "model.layers.36.mlp.experts.45.down_proj.weight": "model-00038-of-00101.safetensors", + "model.layers.36.mlp.experts.46.gate_proj.weight": "model-00038-of-00101.safetensors", + "model.layers.36.mlp.experts.46.up_proj.weight": "model-00038-of-00101.safetensors", + "model.layers.36.mlp.experts.46.down_proj.weight": "model-00038-of-00101.safetensors", + "model.layers.36.mlp.experts.47.gate_proj.weight": "model-00038-of-00101.safetensors", + "model.layers.36.mlp.experts.47.up_proj.weight": "model-00038-of-00101.safetensors", + "model.layers.36.mlp.experts.47.down_proj.weight": "model-00038-of-00101.safetensors", + "model.layers.36.mlp.experts.48.gate_proj.weight": "model-00039-of-00101.safetensors", + "model.layers.36.mlp.experts.48.up_proj.weight": "model-00039-of-00101.safetensors", + "model.layers.36.mlp.experts.48.down_proj.weight": "model-00039-of-00101.safetensors", + "model.layers.36.mlp.experts.49.gate_proj.weight": "model-00039-of-00101.safetensors", + "model.layers.36.mlp.experts.49.up_proj.weight": "model-00039-of-00101.safetensors", + "model.layers.36.mlp.experts.49.down_proj.weight": "model-00039-of-00101.safetensors", + "model.layers.36.mlp.experts.50.gate_proj.weight": "model-00039-of-00101.safetensors", + "model.layers.36.mlp.experts.50.up_proj.weight": "model-00039-of-00101.safetensors", + "model.layers.36.mlp.experts.50.down_proj.weight": "model-00039-of-00101.safetensors", + "model.layers.36.mlp.experts.51.gate_proj.weight": "model-00039-of-00101.safetensors", + "model.layers.36.mlp.experts.51.up_proj.weight": "model-00039-of-00101.safetensors", + "model.layers.36.mlp.experts.51.down_proj.weight": "model-00039-of-00101.safetensors", + "model.layers.36.mlp.experts.52.gate_proj.weight": "model-00039-of-00101.safetensors", + "model.layers.36.mlp.experts.52.up_proj.weight": "model-00039-of-00101.safetensors", + "model.layers.36.mlp.experts.52.down_proj.weight": "model-00039-of-00101.safetensors", + "model.layers.36.mlp.experts.53.gate_proj.weight": "model-00039-of-00101.safetensors", + "model.layers.36.mlp.experts.53.up_proj.weight": "model-00039-of-00101.safetensors", + "model.layers.36.mlp.experts.53.down_proj.weight": "model-00039-of-00101.safetensors", + "model.layers.36.mlp.experts.54.gate_proj.weight": "model-00039-of-00101.safetensors", + "model.layers.36.mlp.experts.54.up_proj.weight": "model-00039-of-00101.safetensors", + "model.layers.36.mlp.experts.54.down_proj.weight": "model-00039-of-00101.safetensors", + "model.layers.36.mlp.experts.55.gate_proj.weight": "model-00039-of-00101.safetensors", + "model.layers.36.mlp.experts.55.up_proj.weight": "model-00039-of-00101.safetensors", + "model.layers.36.mlp.experts.55.down_proj.weight": "model-00039-of-00101.safetensors", + "model.layers.36.mlp.experts.56.gate_proj.weight": "model-00039-of-00101.safetensors", + "model.layers.36.mlp.experts.56.up_proj.weight": "model-00039-of-00101.safetensors", + "model.layers.36.mlp.experts.56.down_proj.weight": "model-00039-of-00101.safetensors", + "model.layers.36.mlp.experts.57.gate_proj.weight": "model-00039-of-00101.safetensors", + "model.layers.36.mlp.experts.57.up_proj.weight": "model-00039-of-00101.safetensors", + "model.layers.36.mlp.experts.57.down_proj.weight": "model-00039-of-00101.safetensors", + "model.layers.36.mlp.experts.58.gate_proj.weight": "model-00039-of-00101.safetensors", + "model.layers.36.mlp.experts.58.up_proj.weight": "model-00039-of-00101.safetensors", + "model.layers.36.mlp.experts.58.down_proj.weight": "model-00039-of-00101.safetensors", + "model.layers.36.mlp.experts.59.gate_proj.weight": "model-00039-of-00101.safetensors", + "model.layers.36.mlp.experts.59.up_proj.weight": "model-00039-of-00101.safetensors", + "model.layers.36.mlp.experts.59.down_proj.weight": "model-00039-of-00101.safetensors", + "model.layers.36.mlp.experts.60.gate_proj.weight": "model-00039-of-00101.safetensors", + "model.layers.36.mlp.experts.60.up_proj.weight": "model-00039-of-00101.safetensors", + "model.layers.36.mlp.experts.60.down_proj.weight": "model-00039-of-00101.safetensors", + "model.layers.36.mlp.experts.61.gate_proj.weight": "model-00039-of-00101.safetensors", + "model.layers.36.mlp.experts.61.up_proj.weight": "model-00039-of-00101.safetensors", + "model.layers.36.mlp.experts.61.down_proj.weight": "model-00039-of-00101.safetensors", + "model.layers.36.mlp.experts.62.gate_proj.weight": "model-00039-of-00101.safetensors", + "model.layers.36.mlp.experts.62.up_proj.weight": "model-00039-of-00101.safetensors", + "model.layers.36.mlp.experts.62.down_proj.weight": "model-00039-of-00101.safetensors", + "model.layers.36.mlp.experts.63.gate_proj.weight": "model-00039-of-00101.safetensors", + "model.layers.36.mlp.experts.63.up_proj.weight": "model-00039-of-00101.safetensors", + "model.layers.36.mlp.experts.63.down_proj.weight": "model-00039-of-00101.safetensors", + "model.layers.36.mlp.experts.64.gate_proj.weight": "model-00039-of-00101.safetensors", + "model.layers.36.mlp.experts.64.up_proj.weight": "model-00039-of-00101.safetensors", + "model.layers.36.mlp.experts.64.down_proj.weight": "model-00039-of-00101.safetensors", + "model.layers.36.mlp.experts.65.gate_proj.weight": "model-00039-of-00101.safetensors", + "model.layers.36.mlp.experts.65.up_proj.weight": "model-00039-of-00101.safetensors", + "model.layers.36.mlp.experts.65.down_proj.weight": "model-00039-of-00101.safetensors", + "model.layers.36.mlp.experts.66.gate_proj.weight": "model-00039-of-00101.safetensors", + "model.layers.36.mlp.experts.66.up_proj.weight": "model-00039-of-00101.safetensors", + "model.layers.36.mlp.experts.66.down_proj.weight": "model-00039-of-00101.safetensors", + "model.layers.36.mlp.experts.67.gate_proj.weight": "model-00039-of-00101.safetensors", + "model.layers.36.mlp.experts.67.up_proj.weight": "model-00039-of-00101.safetensors", + "model.layers.36.mlp.experts.67.down_proj.weight": "model-00039-of-00101.safetensors", + "model.layers.36.mlp.experts.68.gate_proj.weight": "model-00039-of-00101.safetensors", + "model.layers.36.mlp.experts.68.up_proj.weight": "model-00039-of-00101.safetensors", + "model.layers.36.mlp.experts.68.down_proj.weight": "model-00039-of-00101.safetensors", + "model.layers.36.mlp.experts.69.gate_proj.weight": "model-00039-of-00101.safetensors", + "model.layers.36.mlp.experts.69.up_proj.weight": "model-00039-of-00101.safetensors", + "model.layers.36.mlp.experts.69.down_proj.weight": "model-00039-of-00101.safetensors", + "model.layers.36.mlp.experts.70.gate_proj.weight": "model-00039-of-00101.safetensors", + "model.layers.36.mlp.experts.70.up_proj.weight": "model-00039-of-00101.safetensors", + "model.layers.36.mlp.experts.70.down_proj.weight": "model-00039-of-00101.safetensors", + "model.layers.36.mlp.experts.71.gate_proj.weight": "model-00039-of-00101.safetensors", + "model.layers.36.mlp.experts.71.up_proj.weight": "model-00039-of-00101.safetensors", + "model.layers.36.mlp.experts.71.down_proj.weight": "model-00039-of-00101.safetensors", + "model.layers.36.mlp.experts.72.gate_proj.weight": "model-00039-of-00101.safetensors", + "model.layers.36.mlp.experts.72.up_proj.weight": "model-00039-of-00101.safetensors", + "model.layers.36.mlp.experts.72.down_proj.weight": "model-00039-of-00101.safetensors", + "model.layers.36.mlp.experts.73.gate_proj.weight": "model-00039-of-00101.safetensors", + "model.layers.36.mlp.experts.73.up_proj.weight": "model-00039-of-00101.safetensors", + "model.layers.36.mlp.experts.73.down_proj.weight": "model-00039-of-00101.safetensors", + "model.layers.36.mlp.experts.74.gate_proj.weight": "model-00039-of-00101.safetensors", + "model.layers.36.mlp.experts.74.up_proj.weight": "model-00039-of-00101.safetensors", + "model.layers.36.mlp.experts.74.down_proj.weight": "model-00039-of-00101.safetensors", + "model.layers.36.mlp.experts.75.gate_proj.weight": "model-00039-of-00101.safetensors", + "model.layers.36.mlp.experts.75.up_proj.weight": "model-00039-of-00101.safetensors", + "model.layers.36.mlp.experts.75.down_proj.weight": "model-00039-of-00101.safetensors", + "model.layers.36.mlp.experts.76.gate_proj.weight": "model-00039-of-00101.safetensors", + "model.layers.36.mlp.experts.76.up_proj.weight": "model-00039-of-00101.safetensors", + "model.layers.36.mlp.experts.76.down_proj.weight": "model-00039-of-00101.safetensors", + "model.layers.36.mlp.experts.77.gate_proj.weight": "model-00039-of-00101.safetensors", + "model.layers.36.mlp.experts.77.up_proj.weight": "model-00039-of-00101.safetensors", + "model.layers.36.mlp.experts.77.down_proj.weight": "model-00039-of-00101.safetensors", + "model.layers.36.mlp.experts.78.gate_proj.weight": "model-00039-of-00101.safetensors", + "model.layers.36.mlp.experts.78.up_proj.weight": "model-00039-of-00101.safetensors", + "model.layers.36.mlp.experts.78.down_proj.weight": "model-00039-of-00101.safetensors", + "model.layers.36.mlp.experts.79.gate_proj.weight": "model-00039-of-00101.safetensors", + "model.layers.36.mlp.experts.79.up_proj.weight": "model-00039-of-00101.safetensors", + "model.layers.36.mlp.experts.79.down_proj.weight": "model-00039-of-00101.safetensors", + "model.layers.36.mlp.experts.80.gate_proj.weight": "model-00039-of-00101.safetensors", + "model.layers.36.mlp.experts.80.up_proj.weight": "model-00039-of-00101.safetensors", + "model.layers.36.mlp.experts.80.down_proj.weight": "model-00039-of-00101.safetensors", + "model.layers.36.mlp.experts.81.gate_proj.weight": "model-00039-of-00101.safetensors", + "model.layers.36.mlp.experts.81.up_proj.weight": "model-00039-of-00101.safetensors", + "model.layers.36.mlp.experts.81.down_proj.weight": "model-00039-of-00101.safetensors", + "model.layers.36.mlp.experts.82.gate_proj.weight": "model-00039-of-00101.safetensors", + "model.layers.36.mlp.experts.82.up_proj.weight": "model-00039-of-00101.safetensors", + "model.layers.36.mlp.experts.82.down_proj.weight": "model-00039-of-00101.safetensors", + "model.layers.36.mlp.experts.83.gate_proj.weight": "model-00039-of-00101.safetensors", + "model.layers.36.mlp.experts.83.up_proj.weight": "model-00039-of-00101.safetensors", + "model.layers.36.mlp.experts.83.down_proj.weight": "model-00039-of-00101.safetensors", + "model.layers.36.mlp.experts.84.gate_proj.weight": "model-00039-of-00101.safetensors", + "model.layers.36.mlp.experts.84.up_proj.weight": "model-00039-of-00101.safetensors", + "model.layers.36.mlp.experts.84.down_proj.weight": "model-00039-of-00101.safetensors", + "model.layers.36.mlp.experts.85.gate_proj.weight": "model-00039-of-00101.safetensors", + "model.layers.36.mlp.experts.85.up_proj.weight": "model-00039-of-00101.safetensors", + "model.layers.36.mlp.experts.85.down_proj.weight": "model-00039-of-00101.safetensors", + "model.layers.36.mlp.experts.86.gate_proj.weight": "model-00039-of-00101.safetensors", + "model.layers.36.mlp.experts.86.up_proj.weight": "model-00039-of-00101.safetensors", + "model.layers.36.mlp.experts.86.down_proj.weight": "model-00039-of-00101.safetensors", + "model.layers.36.mlp.experts.87.gate_proj.weight": "model-00039-of-00101.safetensors", + "model.layers.36.mlp.experts.87.up_proj.weight": "model-00039-of-00101.safetensors", + "model.layers.36.mlp.experts.87.down_proj.weight": "model-00039-of-00101.safetensors", + "model.layers.36.mlp.experts.88.gate_proj.weight": "model-00039-of-00101.safetensors", + "model.layers.36.mlp.experts.88.up_proj.weight": "model-00039-of-00101.safetensors", + "model.layers.36.mlp.experts.88.down_proj.weight": "model-00039-of-00101.safetensors", + "model.layers.36.mlp.experts.89.gate_proj.weight": "model-00039-of-00101.safetensors", + "model.layers.36.mlp.experts.89.up_proj.weight": "model-00039-of-00101.safetensors", + "model.layers.36.mlp.experts.89.down_proj.weight": "model-00039-of-00101.safetensors", + "model.layers.36.mlp.experts.90.gate_proj.weight": "model-00039-of-00101.safetensors", + "model.layers.36.mlp.experts.90.up_proj.weight": "model-00039-of-00101.safetensors", + "model.layers.36.mlp.experts.90.down_proj.weight": "model-00039-of-00101.safetensors", + "model.layers.36.mlp.experts.91.gate_proj.weight": "model-00039-of-00101.safetensors", + "model.layers.36.mlp.experts.91.up_proj.weight": "model-00039-of-00101.safetensors", + "model.layers.36.mlp.experts.91.down_proj.weight": "model-00039-of-00101.safetensors", + "model.layers.36.mlp.experts.92.gate_proj.weight": "model-00039-of-00101.safetensors", + "model.layers.36.mlp.experts.92.up_proj.weight": "model-00039-of-00101.safetensors", + "model.layers.36.mlp.experts.92.down_proj.weight": "model-00039-of-00101.safetensors", + "model.layers.36.mlp.experts.93.gate_proj.weight": "model-00039-of-00101.safetensors", + "model.layers.36.mlp.experts.93.up_proj.weight": "model-00039-of-00101.safetensors", + "model.layers.36.mlp.experts.93.down_proj.weight": "model-00039-of-00101.safetensors", + "model.layers.36.mlp.experts.94.gate_proj.weight": "model-00039-of-00101.safetensors", + "model.layers.36.mlp.experts.94.up_proj.weight": "model-00039-of-00101.safetensors", + "model.layers.36.mlp.experts.94.down_proj.weight": "model-00039-of-00101.safetensors", + "model.layers.36.mlp.experts.95.gate_proj.weight": "model-00039-of-00101.safetensors", + "model.layers.36.mlp.experts.95.up_proj.weight": "model-00039-of-00101.safetensors", + "model.layers.36.mlp.experts.95.down_proj.weight": "model-00039-of-00101.safetensors", + "model.layers.36.mlp.experts.96.gate_proj.weight": "model-00039-of-00101.safetensors", + "model.layers.36.mlp.experts.96.up_proj.weight": "model-00039-of-00101.safetensors", + "model.layers.36.mlp.experts.96.down_proj.weight": "model-00039-of-00101.safetensors", + "model.layers.36.mlp.experts.97.gate_proj.weight": "model-00039-of-00101.safetensors", + "model.layers.36.mlp.experts.97.up_proj.weight": "model-00039-of-00101.safetensors", + "model.layers.36.mlp.experts.97.down_proj.weight": "model-00039-of-00101.safetensors", + "model.layers.36.mlp.experts.98.gate_proj.weight": "model-00039-of-00101.safetensors", + "model.layers.36.mlp.experts.98.up_proj.weight": "model-00039-of-00101.safetensors", + "model.layers.36.mlp.experts.98.down_proj.weight": "model-00039-of-00101.safetensors", + "model.layers.36.mlp.experts.99.gate_proj.weight": "model-00039-of-00101.safetensors", + "model.layers.36.mlp.experts.99.up_proj.weight": "model-00039-of-00101.safetensors", + "model.layers.36.mlp.experts.99.down_proj.weight": "model-00039-of-00101.safetensors", + "model.layers.36.mlp.experts.100.gate_proj.weight": "model-00039-of-00101.safetensors", + "model.layers.36.mlp.experts.100.up_proj.weight": "model-00039-of-00101.safetensors", + "model.layers.36.mlp.experts.100.down_proj.weight": "model-00039-of-00101.safetensors", + "model.layers.36.mlp.experts.101.gate_proj.weight": "model-00039-of-00101.safetensors", + "model.layers.36.mlp.experts.101.up_proj.weight": "model-00039-of-00101.safetensors", + "model.layers.36.mlp.experts.101.down_proj.weight": "model-00039-of-00101.safetensors", + "model.layers.36.mlp.experts.102.gate_proj.weight": "model-00039-of-00101.safetensors", + "model.layers.36.mlp.experts.102.up_proj.weight": "model-00039-of-00101.safetensors", + "model.layers.36.mlp.experts.102.down_proj.weight": "model-00039-of-00101.safetensors", + "model.layers.36.mlp.experts.103.gate_proj.weight": "model-00039-of-00101.safetensors", + "model.layers.36.mlp.experts.103.up_proj.weight": "model-00039-of-00101.safetensors", + "model.layers.36.mlp.experts.103.down_proj.weight": "model-00039-of-00101.safetensors", + "model.layers.36.mlp.experts.104.gate_proj.weight": "model-00039-of-00101.safetensors", + "model.layers.36.mlp.experts.104.up_proj.weight": "model-00039-of-00101.safetensors", + "model.layers.36.mlp.experts.104.down_proj.weight": "model-00039-of-00101.safetensors", + "model.layers.36.mlp.experts.105.gate_proj.weight": "model-00039-of-00101.safetensors", + "model.layers.36.mlp.experts.105.up_proj.weight": "model-00039-of-00101.safetensors", + "model.layers.36.mlp.experts.105.down_proj.weight": "model-00039-of-00101.safetensors", + "model.layers.36.mlp.experts.106.gate_proj.weight": "model-00039-of-00101.safetensors", + "model.layers.36.mlp.experts.106.up_proj.weight": "model-00039-of-00101.safetensors", + "model.layers.36.mlp.experts.106.down_proj.weight": "model-00039-of-00101.safetensors", + "model.layers.36.mlp.experts.107.gate_proj.weight": "model-00039-of-00101.safetensors", + "model.layers.36.mlp.experts.107.up_proj.weight": "model-00039-of-00101.safetensors", + "model.layers.36.mlp.experts.107.down_proj.weight": "model-00039-of-00101.safetensors", + "model.layers.36.mlp.experts.108.gate_proj.weight": "model-00039-of-00101.safetensors", + "model.layers.36.mlp.experts.108.up_proj.weight": "model-00039-of-00101.safetensors", + "model.layers.36.mlp.experts.108.down_proj.weight": "model-00039-of-00101.safetensors", + "model.layers.36.mlp.experts.109.gate_proj.weight": "model-00039-of-00101.safetensors", + "model.layers.36.mlp.experts.109.up_proj.weight": "model-00039-of-00101.safetensors", + "model.layers.36.mlp.experts.109.down_proj.weight": "model-00039-of-00101.safetensors", + "model.layers.36.mlp.experts.110.gate_proj.weight": "model-00039-of-00101.safetensors", + "model.layers.36.mlp.experts.110.up_proj.weight": "model-00039-of-00101.safetensors", + "model.layers.36.mlp.experts.110.down_proj.weight": "model-00039-of-00101.safetensors", + "model.layers.36.mlp.experts.111.gate_proj.weight": "model-00039-of-00101.safetensors", + "model.layers.36.mlp.experts.111.up_proj.weight": "model-00039-of-00101.safetensors", + "model.layers.36.mlp.experts.111.down_proj.weight": "model-00039-of-00101.safetensors", + "model.layers.36.mlp.experts.112.gate_proj.weight": "model-00039-of-00101.safetensors", + "model.layers.36.mlp.experts.112.up_proj.weight": "model-00039-of-00101.safetensors", + "model.layers.36.mlp.experts.112.down_proj.weight": "model-00039-of-00101.safetensors", + "model.layers.36.mlp.experts.113.gate_proj.weight": "model-00039-of-00101.safetensors", + "model.layers.36.mlp.experts.113.up_proj.weight": "model-00039-of-00101.safetensors", + "model.layers.36.mlp.experts.113.down_proj.weight": "model-00039-of-00101.safetensors", + "model.layers.36.mlp.experts.114.gate_proj.weight": "model-00039-of-00101.safetensors", + "model.layers.36.mlp.experts.114.up_proj.weight": "model-00039-of-00101.safetensors", + "model.layers.36.mlp.experts.114.down_proj.weight": "model-00039-of-00101.safetensors", + "model.layers.36.mlp.experts.115.gate_proj.weight": "model-00039-of-00101.safetensors", + "model.layers.36.mlp.experts.115.up_proj.weight": "model-00039-of-00101.safetensors", + "model.layers.36.mlp.experts.115.down_proj.weight": "model-00039-of-00101.safetensors", + "model.layers.36.mlp.experts.116.gate_proj.weight": "model-00039-of-00101.safetensors", + "model.layers.36.mlp.experts.116.up_proj.weight": "model-00039-of-00101.safetensors", + "model.layers.36.mlp.experts.116.down_proj.weight": "model-00039-of-00101.safetensors", + "model.layers.36.mlp.experts.117.gate_proj.weight": "model-00039-of-00101.safetensors", + "model.layers.36.mlp.experts.117.up_proj.weight": "model-00039-of-00101.safetensors", + "model.layers.36.mlp.experts.117.down_proj.weight": "model-00039-of-00101.safetensors", + "model.layers.36.mlp.experts.118.gate_proj.weight": "model-00039-of-00101.safetensors", + "model.layers.36.mlp.experts.118.up_proj.weight": "model-00039-of-00101.safetensors", + "model.layers.36.mlp.experts.118.down_proj.weight": "model-00039-of-00101.safetensors", + "model.layers.36.mlp.experts.119.gate_proj.weight": "model-00039-of-00101.safetensors", + "model.layers.36.mlp.experts.119.up_proj.weight": "model-00039-of-00101.safetensors", + "model.layers.36.mlp.experts.119.down_proj.weight": "model-00039-of-00101.safetensors", + "model.layers.36.mlp.gate.weight": "model-00039-of-00101.safetensors", + "model.layers.36.mlp.gate.e_score_correction_bias": "model-00039-of-00101.safetensors", + "model.layers.36.mlp.shared_experts.gate_proj.weight": "model-00039-of-00101.safetensors", + "model.layers.36.mlp.shared_experts.up_proj.weight": "model-00039-of-00101.safetensors", + "model.layers.36.mlp.shared_experts.down_proj.weight": "model-00039-of-00101.safetensors", + "model.layers.36.input_layernorm.weight": "model-00039-of-00101.safetensors", + "model.layers.36.post_attention_layernorm.weight": "model-00039-of-00101.safetensors", + "model.layers.37.self_attn.q_proj.weight": "model-00039-of-00101.safetensors", + "model.layers.37.self_attn.q_proj.bias": "model-00039-of-00101.safetensors", + "model.layers.37.self_attn.k_proj.weight": "model-00039-of-00101.safetensors", + "model.layers.37.self_attn.k_proj.bias": "model-00039-of-00101.safetensors", + "model.layers.37.self_attn.v_proj.weight": "model-00039-of-00101.safetensors", + "model.layers.37.self_attn.v_proj.bias": "model-00039-of-00101.safetensors", + "model.layers.37.self_attn.o_proj.weight": "model-00039-of-00101.safetensors", + "model.layers.37.self_attn.q_norm.weight": "model-00039-of-00101.safetensors", + "model.layers.37.self_attn.k_norm.weight": "model-00039-of-00101.safetensors", + "model.layers.37.mlp.experts.0.gate_proj.weight": "model-00039-of-00101.safetensors", + "model.layers.37.mlp.experts.0.up_proj.weight": "model-00039-of-00101.safetensors", + "model.layers.37.mlp.experts.0.down_proj.weight": "model-00039-of-00101.safetensors", + "model.layers.37.mlp.experts.1.gate_proj.weight": "model-00039-of-00101.safetensors", + "model.layers.37.mlp.experts.1.up_proj.weight": "model-00039-of-00101.safetensors", + "model.layers.37.mlp.experts.1.down_proj.weight": "model-00039-of-00101.safetensors", + "model.layers.37.mlp.experts.2.gate_proj.weight": "model-00039-of-00101.safetensors", + "model.layers.37.mlp.experts.2.up_proj.weight": "model-00039-of-00101.safetensors", + "model.layers.37.mlp.experts.2.down_proj.weight": "model-00039-of-00101.safetensors", + "model.layers.37.mlp.experts.3.gate_proj.weight": "model-00039-of-00101.safetensors", + "model.layers.37.mlp.experts.3.up_proj.weight": "model-00039-of-00101.safetensors", + "model.layers.37.mlp.experts.3.down_proj.weight": "model-00039-of-00101.safetensors", + "model.layers.37.mlp.experts.4.gate_proj.weight": "model-00039-of-00101.safetensors", + "model.layers.37.mlp.experts.4.up_proj.weight": "model-00039-of-00101.safetensors", + "model.layers.37.mlp.experts.4.down_proj.weight": "model-00039-of-00101.safetensors", + "model.layers.37.mlp.experts.5.gate_proj.weight": "model-00039-of-00101.safetensors", + "model.layers.37.mlp.experts.5.up_proj.weight": "model-00039-of-00101.safetensors", + "model.layers.37.mlp.experts.5.down_proj.weight": "model-00039-of-00101.safetensors", + "model.layers.37.mlp.experts.6.gate_proj.weight": "model-00039-of-00101.safetensors", + "model.layers.37.mlp.experts.6.up_proj.weight": "model-00039-of-00101.safetensors", + "model.layers.37.mlp.experts.6.down_proj.weight": "model-00039-of-00101.safetensors", + "model.layers.37.mlp.experts.7.gate_proj.weight": "model-00039-of-00101.safetensors", + "model.layers.37.mlp.experts.7.up_proj.weight": "model-00039-of-00101.safetensors", + "model.layers.37.mlp.experts.7.down_proj.weight": "model-00039-of-00101.safetensors", + "model.layers.37.mlp.experts.8.gate_proj.weight": "model-00039-of-00101.safetensors", + "model.layers.37.mlp.experts.8.up_proj.weight": "model-00039-of-00101.safetensors", + "model.layers.37.mlp.experts.8.down_proj.weight": "model-00039-of-00101.safetensors", + "model.layers.37.mlp.experts.9.gate_proj.weight": "model-00039-of-00101.safetensors", + "model.layers.37.mlp.experts.9.up_proj.weight": "model-00039-of-00101.safetensors", + "model.layers.37.mlp.experts.9.down_proj.weight": "model-00039-of-00101.safetensors", + "model.layers.37.mlp.experts.10.gate_proj.weight": "model-00039-of-00101.safetensors", + "model.layers.37.mlp.experts.10.up_proj.weight": "model-00039-of-00101.safetensors", + "model.layers.37.mlp.experts.10.down_proj.weight": "model-00039-of-00101.safetensors", + "model.layers.37.mlp.experts.11.gate_proj.weight": "model-00039-of-00101.safetensors", + "model.layers.37.mlp.experts.11.up_proj.weight": "model-00039-of-00101.safetensors", + "model.layers.37.mlp.experts.11.down_proj.weight": "model-00039-of-00101.safetensors", + "model.layers.37.mlp.experts.12.gate_proj.weight": "model-00039-of-00101.safetensors", + "model.layers.37.mlp.experts.12.up_proj.weight": "model-00039-of-00101.safetensors", + "model.layers.37.mlp.experts.12.down_proj.weight": "model-00039-of-00101.safetensors", + "model.layers.37.mlp.experts.13.gate_proj.weight": "model-00039-of-00101.safetensors", + "model.layers.37.mlp.experts.13.up_proj.weight": "model-00039-of-00101.safetensors", + "model.layers.37.mlp.experts.13.down_proj.weight": "model-00039-of-00101.safetensors", + "model.layers.37.mlp.experts.14.gate_proj.weight": "model-00039-of-00101.safetensors", + "model.layers.37.mlp.experts.14.up_proj.weight": "model-00039-of-00101.safetensors", + "model.layers.37.mlp.experts.14.down_proj.weight": "model-00039-of-00101.safetensors", + "model.layers.37.mlp.experts.15.gate_proj.weight": "model-00039-of-00101.safetensors", + "model.layers.37.mlp.experts.15.up_proj.weight": "model-00039-of-00101.safetensors", + "model.layers.37.mlp.experts.15.down_proj.weight": "model-00039-of-00101.safetensors", + "model.layers.37.mlp.experts.16.gate_proj.weight": "model-00039-of-00101.safetensors", + "model.layers.37.mlp.experts.16.up_proj.weight": "model-00039-of-00101.safetensors", + "model.layers.37.mlp.experts.16.down_proj.weight": "model-00039-of-00101.safetensors", + "model.layers.37.mlp.experts.17.gate_proj.weight": "model-00039-of-00101.safetensors", + "model.layers.37.mlp.experts.17.up_proj.weight": "model-00039-of-00101.safetensors", + "model.layers.37.mlp.experts.17.down_proj.weight": "model-00039-of-00101.safetensors", + "model.layers.37.mlp.experts.18.gate_proj.weight": "model-00039-of-00101.safetensors", + "model.layers.37.mlp.experts.18.up_proj.weight": "model-00039-of-00101.safetensors", + "model.layers.37.mlp.experts.18.down_proj.weight": "model-00039-of-00101.safetensors", + "model.layers.37.mlp.experts.19.gate_proj.weight": "model-00039-of-00101.safetensors", + "model.layers.37.mlp.experts.19.up_proj.weight": "model-00039-of-00101.safetensors", + "model.layers.37.mlp.experts.19.down_proj.weight": "model-00039-of-00101.safetensors", + "model.layers.37.mlp.experts.20.gate_proj.weight": "model-00039-of-00101.safetensors", + "model.layers.37.mlp.experts.20.up_proj.weight": "model-00039-of-00101.safetensors", + "model.layers.37.mlp.experts.20.down_proj.weight": "model-00039-of-00101.safetensors", + "model.layers.37.mlp.experts.21.gate_proj.weight": "model-00039-of-00101.safetensors", + "model.layers.37.mlp.experts.21.up_proj.weight": "model-00039-of-00101.safetensors", + "model.layers.37.mlp.experts.21.down_proj.weight": "model-00039-of-00101.safetensors", + "model.layers.37.mlp.experts.22.gate_proj.weight": "model-00039-of-00101.safetensors", + "model.layers.37.mlp.experts.22.up_proj.weight": "model-00039-of-00101.safetensors", + "model.layers.37.mlp.experts.22.down_proj.weight": "model-00039-of-00101.safetensors", + "model.layers.37.mlp.experts.23.gate_proj.weight": "model-00039-of-00101.safetensors", + "model.layers.37.mlp.experts.23.up_proj.weight": "model-00039-of-00101.safetensors", + "model.layers.37.mlp.experts.23.down_proj.weight": "model-00039-of-00101.safetensors", + "model.layers.37.mlp.experts.24.gate_proj.weight": "model-00039-of-00101.safetensors", + "model.layers.37.mlp.experts.24.up_proj.weight": "model-00039-of-00101.safetensors", + "model.layers.37.mlp.experts.24.down_proj.weight": "model-00039-of-00101.safetensors", + "model.layers.37.mlp.experts.25.gate_proj.weight": "model-00039-of-00101.safetensors", + "model.layers.37.mlp.experts.25.up_proj.weight": "model-00039-of-00101.safetensors", + "model.layers.37.mlp.experts.25.down_proj.weight": "model-00039-of-00101.safetensors", + "model.layers.37.mlp.experts.26.gate_proj.weight": "model-00039-of-00101.safetensors", + "model.layers.37.mlp.experts.26.up_proj.weight": "model-00039-of-00101.safetensors", + "model.layers.37.mlp.experts.26.down_proj.weight": "model-00039-of-00101.safetensors", + "model.layers.37.mlp.experts.27.gate_proj.weight": "model-00039-of-00101.safetensors", + "model.layers.37.mlp.experts.27.up_proj.weight": "model-00039-of-00101.safetensors", + "model.layers.37.mlp.experts.27.down_proj.weight": "model-00039-of-00101.safetensors", + "model.layers.37.mlp.experts.28.gate_proj.weight": "model-00039-of-00101.safetensors", + "model.layers.37.mlp.experts.28.up_proj.weight": "model-00039-of-00101.safetensors", + "model.layers.37.mlp.experts.28.down_proj.weight": "model-00039-of-00101.safetensors", + "model.layers.37.mlp.experts.29.gate_proj.weight": "model-00039-of-00101.safetensors", + "model.layers.37.mlp.experts.29.up_proj.weight": "model-00039-of-00101.safetensors", + "model.layers.37.mlp.experts.29.down_proj.weight": "model-00039-of-00101.safetensors", + "model.layers.37.mlp.experts.30.gate_proj.weight": "model-00039-of-00101.safetensors", + "model.layers.37.mlp.experts.30.up_proj.weight": "model-00039-of-00101.safetensors", + "model.layers.37.mlp.experts.30.down_proj.weight": "model-00039-of-00101.safetensors", + "model.layers.37.mlp.experts.31.gate_proj.weight": "model-00039-of-00101.safetensors", + "model.layers.37.mlp.experts.31.up_proj.weight": "model-00039-of-00101.safetensors", + "model.layers.37.mlp.experts.31.down_proj.weight": "model-00039-of-00101.safetensors", + "model.layers.37.mlp.experts.32.gate_proj.weight": "model-00039-of-00101.safetensors", + "model.layers.37.mlp.experts.32.up_proj.weight": "model-00039-of-00101.safetensors", + "model.layers.37.mlp.experts.32.down_proj.weight": "model-00039-of-00101.safetensors", + "model.layers.37.mlp.experts.33.gate_proj.weight": "model-00039-of-00101.safetensors", + "model.layers.37.mlp.experts.33.up_proj.weight": "model-00039-of-00101.safetensors", + "model.layers.37.mlp.experts.33.down_proj.weight": "model-00039-of-00101.safetensors", + "model.layers.37.mlp.experts.34.gate_proj.weight": "model-00039-of-00101.safetensors", + "model.layers.37.mlp.experts.34.up_proj.weight": "model-00039-of-00101.safetensors", + "model.layers.37.mlp.experts.34.down_proj.weight": "model-00040-of-00101.safetensors", + "model.layers.37.mlp.experts.35.gate_proj.weight": "model-00040-of-00101.safetensors", + "model.layers.37.mlp.experts.35.up_proj.weight": "model-00040-of-00101.safetensors", + "model.layers.37.mlp.experts.35.down_proj.weight": "model-00040-of-00101.safetensors", + "model.layers.37.mlp.experts.36.gate_proj.weight": "model-00040-of-00101.safetensors", + "model.layers.37.mlp.experts.36.up_proj.weight": "model-00040-of-00101.safetensors", + "model.layers.37.mlp.experts.36.down_proj.weight": "model-00040-of-00101.safetensors", + "model.layers.37.mlp.experts.37.gate_proj.weight": "model-00040-of-00101.safetensors", + "model.layers.37.mlp.experts.37.up_proj.weight": "model-00040-of-00101.safetensors", + "model.layers.37.mlp.experts.37.down_proj.weight": "model-00040-of-00101.safetensors", + "model.layers.37.mlp.experts.38.gate_proj.weight": "model-00040-of-00101.safetensors", + "model.layers.37.mlp.experts.38.up_proj.weight": "model-00040-of-00101.safetensors", + "model.layers.37.mlp.experts.38.down_proj.weight": "model-00040-of-00101.safetensors", + "model.layers.37.mlp.experts.39.gate_proj.weight": "model-00040-of-00101.safetensors", + "model.layers.37.mlp.experts.39.up_proj.weight": "model-00040-of-00101.safetensors", + "model.layers.37.mlp.experts.39.down_proj.weight": "model-00040-of-00101.safetensors", + "model.layers.37.mlp.experts.40.gate_proj.weight": "model-00040-of-00101.safetensors", + "model.layers.37.mlp.experts.40.up_proj.weight": "model-00040-of-00101.safetensors", + "model.layers.37.mlp.experts.40.down_proj.weight": "model-00040-of-00101.safetensors", + "model.layers.37.mlp.experts.41.gate_proj.weight": "model-00040-of-00101.safetensors", + "model.layers.37.mlp.experts.41.up_proj.weight": "model-00040-of-00101.safetensors", + "model.layers.37.mlp.experts.41.down_proj.weight": "model-00040-of-00101.safetensors", + "model.layers.37.mlp.experts.42.gate_proj.weight": "model-00040-of-00101.safetensors", + "model.layers.37.mlp.experts.42.up_proj.weight": "model-00040-of-00101.safetensors", + "model.layers.37.mlp.experts.42.down_proj.weight": "model-00040-of-00101.safetensors", + "model.layers.37.mlp.experts.43.gate_proj.weight": "model-00040-of-00101.safetensors", + "model.layers.37.mlp.experts.43.up_proj.weight": "model-00040-of-00101.safetensors", + "model.layers.37.mlp.experts.43.down_proj.weight": "model-00040-of-00101.safetensors", + "model.layers.37.mlp.experts.44.gate_proj.weight": "model-00040-of-00101.safetensors", + "model.layers.37.mlp.experts.44.up_proj.weight": "model-00040-of-00101.safetensors", + "model.layers.37.mlp.experts.44.down_proj.weight": "model-00040-of-00101.safetensors", + "model.layers.37.mlp.experts.45.gate_proj.weight": "model-00040-of-00101.safetensors", + "model.layers.37.mlp.experts.45.up_proj.weight": "model-00040-of-00101.safetensors", + "model.layers.37.mlp.experts.45.down_proj.weight": "model-00040-of-00101.safetensors", + "model.layers.37.mlp.experts.46.gate_proj.weight": "model-00040-of-00101.safetensors", + "model.layers.37.mlp.experts.46.up_proj.weight": "model-00040-of-00101.safetensors", + "model.layers.37.mlp.experts.46.down_proj.weight": "model-00040-of-00101.safetensors", + "model.layers.37.mlp.experts.47.gate_proj.weight": "model-00040-of-00101.safetensors", + "model.layers.37.mlp.experts.47.up_proj.weight": "model-00040-of-00101.safetensors", + "model.layers.37.mlp.experts.47.down_proj.weight": "model-00040-of-00101.safetensors", + "model.layers.37.mlp.experts.48.gate_proj.weight": "model-00040-of-00101.safetensors", + "model.layers.37.mlp.experts.48.up_proj.weight": "model-00040-of-00101.safetensors", + "model.layers.37.mlp.experts.48.down_proj.weight": "model-00040-of-00101.safetensors", + "model.layers.37.mlp.experts.49.gate_proj.weight": "model-00040-of-00101.safetensors", + "model.layers.37.mlp.experts.49.up_proj.weight": "model-00040-of-00101.safetensors", + "model.layers.37.mlp.experts.49.down_proj.weight": "model-00040-of-00101.safetensors", + "model.layers.37.mlp.experts.50.gate_proj.weight": "model-00040-of-00101.safetensors", + "model.layers.37.mlp.experts.50.up_proj.weight": "model-00040-of-00101.safetensors", + "model.layers.37.mlp.experts.50.down_proj.weight": "model-00040-of-00101.safetensors", + "model.layers.37.mlp.experts.51.gate_proj.weight": "model-00040-of-00101.safetensors", + "model.layers.37.mlp.experts.51.up_proj.weight": "model-00040-of-00101.safetensors", + "model.layers.37.mlp.experts.51.down_proj.weight": "model-00040-of-00101.safetensors", + "model.layers.37.mlp.experts.52.gate_proj.weight": "model-00040-of-00101.safetensors", + "model.layers.37.mlp.experts.52.up_proj.weight": "model-00040-of-00101.safetensors", + "model.layers.37.mlp.experts.52.down_proj.weight": "model-00040-of-00101.safetensors", + "model.layers.37.mlp.experts.53.gate_proj.weight": "model-00040-of-00101.safetensors", + "model.layers.37.mlp.experts.53.up_proj.weight": "model-00040-of-00101.safetensors", + "model.layers.37.mlp.experts.53.down_proj.weight": "model-00040-of-00101.safetensors", + "model.layers.37.mlp.experts.54.gate_proj.weight": "model-00040-of-00101.safetensors", + "model.layers.37.mlp.experts.54.up_proj.weight": "model-00040-of-00101.safetensors", + "model.layers.37.mlp.experts.54.down_proj.weight": "model-00040-of-00101.safetensors", + "model.layers.37.mlp.experts.55.gate_proj.weight": "model-00040-of-00101.safetensors", + "model.layers.37.mlp.experts.55.up_proj.weight": "model-00040-of-00101.safetensors", + "model.layers.37.mlp.experts.55.down_proj.weight": "model-00040-of-00101.safetensors", + "model.layers.37.mlp.experts.56.gate_proj.weight": "model-00040-of-00101.safetensors", + "model.layers.37.mlp.experts.56.up_proj.weight": "model-00040-of-00101.safetensors", + "model.layers.37.mlp.experts.56.down_proj.weight": "model-00040-of-00101.safetensors", + "model.layers.37.mlp.experts.57.gate_proj.weight": "model-00040-of-00101.safetensors", + "model.layers.37.mlp.experts.57.up_proj.weight": "model-00040-of-00101.safetensors", + "model.layers.37.mlp.experts.57.down_proj.weight": "model-00040-of-00101.safetensors", + "model.layers.37.mlp.experts.58.gate_proj.weight": "model-00040-of-00101.safetensors", + "model.layers.37.mlp.experts.58.up_proj.weight": "model-00040-of-00101.safetensors", + "model.layers.37.mlp.experts.58.down_proj.weight": "model-00040-of-00101.safetensors", + "model.layers.37.mlp.experts.59.gate_proj.weight": "model-00040-of-00101.safetensors", + "model.layers.37.mlp.experts.59.up_proj.weight": "model-00040-of-00101.safetensors", + "model.layers.37.mlp.experts.59.down_proj.weight": "model-00040-of-00101.safetensors", + "model.layers.37.mlp.experts.60.gate_proj.weight": "model-00040-of-00101.safetensors", + "model.layers.37.mlp.experts.60.up_proj.weight": "model-00040-of-00101.safetensors", + "model.layers.37.mlp.experts.60.down_proj.weight": "model-00040-of-00101.safetensors", + "model.layers.37.mlp.experts.61.gate_proj.weight": "model-00040-of-00101.safetensors", + "model.layers.37.mlp.experts.61.up_proj.weight": "model-00040-of-00101.safetensors", + "model.layers.37.mlp.experts.61.down_proj.weight": "model-00040-of-00101.safetensors", + "model.layers.37.mlp.experts.62.gate_proj.weight": "model-00040-of-00101.safetensors", + "model.layers.37.mlp.experts.62.up_proj.weight": "model-00040-of-00101.safetensors", + "model.layers.37.mlp.experts.62.down_proj.weight": "model-00040-of-00101.safetensors", + "model.layers.37.mlp.experts.63.gate_proj.weight": "model-00040-of-00101.safetensors", + "model.layers.37.mlp.experts.63.up_proj.weight": "model-00040-of-00101.safetensors", + "model.layers.37.mlp.experts.63.down_proj.weight": "model-00040-of-00101.safetensors", + "model.layers.37.mlp.experts.64.gate_proj.weight": "model-00040-of-00101.safetensors", + "model.layers.37.mlp.experts.64.up_proj.weight": "model-00040-of-00101.safetensors", + "model.layers.37.mlp.experts.64.down_proj.weight": "model-00040-of-00101.safetensors", + "model.layers.37.mlp.experts.65.gate_proj.weight": "model-00040-of-00101.safetensors", + "model.layers.37.mlp.experts.65.up_proj.weight": "model-00040-of-00101.safetensors", + "model.layers.37.mlp.experts.65.down_proj.weight": "model-00040-of-00101.safetensors", + "model.layers.37.mlp.experts.66.gate_proj.weight": "model-00040-of-00101.safetensors", + "model.layers.37.mlp.experts.66.up_proj.weight": "model-00040-of-00101.safetensors", + "model.layers.37.mlp.experts.66.down_proj.weight": "model-00040-of-00101.safetensors", + "model.layers.37.mlp.experts.67.gate_proj.weight": "model-00040-of-00101.safetensors", + "model.layers.37.mlp.experts.67.up_proj.weight": "model-00040-of-00101.safetensors", + "model.layers.37.mlp.experts.67.down_proj.weight": "model-00040-of-00101.safetensors", + "model.layers.37.mlp.experts.68.gate_proj.weight": "model-00040-of-00101.safetensors", + "model.layers.37.mlp.experts.68.up_proj.weight": "model-00040-of-00101.safetensors", + "model.layers.37.mlp.experts.68.down_proj.weight": "model-00040-of-00101.safetensors", + "model.layers.37.mlp.experts.69.gate_proj.weight": "model-00040-of-00101.safetensors", + "model.layers.37.mlp.experts.69.up_proj.weight": "model-00040-of-00101.safetensors", + "model.layers.37.mlp.experts.69.down_proj.weight": "model-00040-of-00101.safetensors", + "model.layers.37.mlp.experts.70.gate_proj.weight": "model-00040-of-00101.safetensors", + "model.layers.37.mlp.experts.70.up_proj.weight": "model-00040-of-00101.safetensors", + "model.layers.37.mlp.experts.70.down_proj.weight": "model-00040-of-00101.safetensors", + "model.layers.37.mlp.experts.71.gate_proj.weight": "model-00040-of-00101.safetensors", + "model.layers.37.mlp.experts.71.up_proj.weight": "model-00040-of-00101.safetensors", + "model.layers.37.mlp.experts.71.down_proj.weight": "model-00040-of-00101.safetensors", + "model.layers.37.mlp.experts.72.gate_proj.weight": "model-00040-of-00101.safetensors", + "model.layers.37.mlp.experts.72.up_proj.weight": "model-00040-of-00101.safetensors", + "model.layers.37.mlp.experts.72.down_proj.weight": "model-00040-of-00101.safetensors", + "model.layers.37.mlp.experts.73.gate_proj.weight": "model-00040-of-00101.safetensors", + "model.layers.37.mlp.experts.73.up_proj.weight": "model-00040-of-00101.safetensors", + "model.layers.37.mlp.experts.73.down_proj.weight": "model-00040-of-00101.safetensors", + "model.layers.37.mlp.experts.74.gate_proj.weight": "model-00040-of-00101.safetensors", + "model.layers.37.mlp.experts.74.up_proj.weight": "model-00040-of-00101.safetensors", + "model.layers.37.mlp.experts.74.down_proj.weight": "model-00040-of-00101.safetensors", + "model.layers.37.mlp.experts.75.gate_proj.weight": "model-00040-of-00101.safetensors", + "model.layers.37.mlp.experts.75.up_proj.weight": "model-00040-of-00101.safetensors", + "model.layers.37.mlp.experts.75.down_proj.weight": "model-00040-of-00101.safetensors", + "model.layers.37.mlp.experts.76.gate_proj.weight": "model-00040-of-00101.safetensors", + "model.layers.37.mlp.experts.76.up_proj.weight": "model-00040-of-00101.safetensors", + "model.layers.37.mlp.experts.76.down_proj.weight": "model-00040-of-00101.safetensors", + "model.layers.37.mlp.experts.77.gate_proj.weight": "model-00040-of-00101.safetensors", + "model.layers.37.mlp.experts.77.up_proj.weight": "model-00040-of-00101.safetensors", + "model.layers.37.mlp.experts.77.down_proj.weight": "model-00040-of-00101.safetensors", + "model.layers.37.mlp.experts.78.gate_proj.weight": "model-00040-of-00101.safetensors", + "model.layers.37.mlp.experts.78.up_proj.weight": "model-00040-of-00101.safetensors", + "model.layers.37.mlp.experts.78.down_proj.weight": "model-00040-of-00101.safetensors", + "model.layers.37.mlp.experts.79.gate_proj.weight": "model-00040-of-00101.safetensors", + "model.layers.37.mlp.experts.79.up_proj.weight": "model-00040-of-00101.safetensors", + "model.layers.37.mlp.experts.79.down_proj.weight": "model-00040-of-00101.safetensors", + "model.layers.37.mlp.experts.80.gate_proj.weight": "model-00040-of-00101.safetensors", + "model.layers.37.mlp.experts.80.up_proj.weight": "model-00040-of-00101.safetensors", + "model.layers.37.mlp.experts.80.down_proj.weight": "model-00040-of-00101.safetensors", + "model.layers.37.mlp.experts.81.gate_proj.weight": "model-00040-of-00101.safetensors", + "model.layers.37.mlp.experts.81.up_proj.weight": "model-00040-of-00101.safetensors", + "model.layers.37.mlp.experts.81.down_proj.weight": "model-00040-of-00101.safetensors", + "model.layers.37.mlp.experts.82.gate_proj.weight": "model-00040-of-00101.safetensors", + "model.layers.37.mlp.experts.82.up_proj.weight": "model-00040-of-00101.safetensors", + "model.layers.37.mlp.experts.82.down_proj.weight": "model-00040-of-00101.safetensors", + "model.layers.37.mlp.experts.83.gate_proj.weight": "model-00040-of-00101.safetensors", + "model.layers.37.mlp.experts.83.up_proj.weight": "model-00040-of-00101.safetensors", + "model.layers.37.mlp.experts.83.down_proj.weight": "model-00040-of-00101.safetensors", + "model.layers.37.mlp.experts.84.gate_proj.weight": "model-00040-of-00101.safetensors", + "model.layers.37.mlp.experts.84.up_proj.weight": "model-00040-of-00101.safetensors", + "model.layers.37.mlp.experts.84.down_proj.weight": "model-00040-of-00101.safetensors", + "model.layers.37.mlp.experts.85.gate_proj.weight": "model-00040-of-00101.safetensors", + "model.layers.37.mlp.experts.85.up_proj.weight": "model-00040-of-00101.safetensors", + "model.layers.37.mlp.experts.85.down_proj.weight": "model-00040-of-00101.safetensors", + "model.layers.37.mlp.experts.86.gate_proj.weight": "model-00040-of-00101.safetensors", + "model.layers.37.mlp.experts.86.up_proj.weight": "model-00040-of-00101.safetensors", + "model.layers.37.mlp.experts.86.down_proj.weight": "model-00040-of-00101.safetensors", + "model.layers.37.mlp.experts.87.gate_proj.weight": "model-00040-of-00101.safetensors", + "model.layers.37.mlp.experts.87.up_proj.weight": "model-00040-of-00101.safetensors", + "model.layers.37.mlp.experts.87.down_proj.weight": "model-00040-of-00101.safetensors", + "model.layers.37.mlp.experts.88.gate_proj.weight": "model-00040-of-00101.safetensors", + "model.layers.37.mlp.experts.88.up_proj.weight": "model-00040-of-00101.safetensors", + "model.layers.37.mlp.experts.88.down_proj.weight": "model-00040-of-00101.safetensors", + "model.layers.37.mlp.experts.89.gate_proj.weight": "model-00040-of-00101.safetensors", + "model.layers.37.mlp.experts.89.up_proj.weight": "model-00040-of-00101.safetensors", + "model.layers.37.mlp.experts.89.down_proj.weight": "model-00040-of-00101.safetensors", + "model.layers.37.mlp.experts.90.gate_proj.weight": "model-00040-of-00101.safetensors", + "model.layers.37.mlp.experts.90.up_proj.weight": "model-00040-of-00101.safetensors", + "model.layers.37.mlp.experts.90.down_proj.weight": "model-00040-of-00101.safetensors", + "model.layers.37.mlp.experts.91.gate_proj.weight": "model-00040-of-00101.safetensors", + "model.layers.37.mlp.experts.91.up_proj.weight": "model-00040-of-00101.safetensors", + "model.layers.37.mlp.experts.91.down_proj.weight": "model-00040-of-00101.safetensors", + "model.layers.37.mlp.experts.92.gate_proj.weight": "model-00040-of-00101.safetensors", + "model.layers.37.mlp.experts.92.up_proj.weight": "model-00040-of-00101.safetensors", + "model.layers.37.mlp.experts.92.down_proj.weight": "model-00040-of-00101.safetensors", + "model.layers.37.mlp.experts.93.gate_proj.weight": "model-00040-of-00101.safetensors", + "model.layers.37.mlp.experts.93.up_proj.weight": "model-00040-of-00101.safetensors", + "model.layers.37.mlp.experts.93.down_proj.weight": "model-00040-of-00101.safetensors", + "model.layers.37.mlp.experts.94.gate_proj.weight": "model-00040-of-00101.safetensors", + "model.layers.37.mlp.experts.94.up_proj.weight": "model-00040-of-00101.safetensors", + "model.layers.37.mlp.experts.94.down_proj.weight": "model-00040-of-00101.safetensors", + "model.layers.37.mlp.experts.95.gate_proj.weight": "model-00040-of-00101.safetensors", + "model.layers.37.mlp.experts.95.up_proj.weight": "model-00040-of-00101.safetensors", + "model.layers.37.mlp.experts.95.down_proj.weight": "model-00040-of-00101.safetensors", + "model.layers.37.mlp.experts.96.gate_proj.weight": "model-00040-of-00101.safetensors", + "model.layers.37.mlp.experts.96.up_proj.weight": "model-00040-of-00101.safetensors", + "model.layers.37.mlp.experts.96.down_proj.weight": "model-00040-of-00101.safetensors", + "model.layers.37.mlp.experts.97.gate_proj.weight": "model-00040-of-00101.safetensors", + "model.layers.37.mlp.experts.97.up_proj.weight": "model-00040-of-00101.safetensors", + "model.layers.37.mlp.experts.97.down_proj.weight": "model-00040-of-00101.safetensors", + "model.layers.37.mlp.experts.98.gate_proj.weight": "model-00040-of-00101.safetensors", + "model.layers.37.mlp.experts.98.up_proj.weight": "model-00040-of-00101.safetensors", + "model.layers.37.mlp.experts.98.down_proj.weight": "model-00040-of-00101.safetensors", + "model.layers.37.mlp.experts.99.gate_proj.weight": "model-00040-of-00101.safetensors", + "model.layers.37.mlp.experts.99.up_proj.weight": "model-00040-of-00101.safetensors", + "model.layers.37.mlp.experts.99.down_proj.weight": "model-00040-of-00101.safetensors", + "model.layers.37.mlp.experts.100.gate_proj.weight": "model-00040-of-00101.safetensors", + "model.layers.37.mlp.experts.100.up_proj.weight": "model-00040-of-00101.safetensors", + "model.layers.37.mlp.experts.100.down_proj.weight": "model-00040-of-00101.safetensors", + "model.layers.37.mlp.experts.101.gate_proj.weight": "model-00040-of-00101.safetensors", + "model.layers.37.mlp.experts.101.up_proj.weight": "model-00040-of-00101.safetensors", + "model.layers.37.mlp.experts.101.down_proj.weight": "model-00040-of-00101.safetensors", + "model.layers.37.mlp.experts.102.gate_proj.weight": "model-00040-of-00101.safetensors", + "model.layers.37.mlp.experts.102.up_proj.weight": "model-00040-of-00101.safetensors", + "model.layers.37.mlp.experts.102.down_proj.weight": "model-00040-of-00101.safetensors", + "model.layers.37.mlp.experts.103.gate_proj.weight": "model-00040-of-00101.safetensors", + "model.layers.37.mlp.experts.103.up_proj.weight": "model-00040-of-00101.safetensors", + "model.layers.37.mlp.experts.103.down_proj.weight": "model-00040-of-00101.safetensors", + "model.layers.37.mlp.experts.104.gate_proj.weight": "model-00040-of-00101.safetensors", + "model.layers.37.mlp.experts.104.up_proj.weight": "model-00040-of-00101.safetensors", + "model.layers.37.mlp.experts.104.down_proj.weight": "model-00040-of-00101.safetensors", + "model.layers.37.mlp.experts.105.gate_proj.weight": "model-00040-of-00101.safetensors", + "model.layers.37.mlp.experts.105.up_proj.weight": "model-00040-of-00101.safetensors", + "model.layers.37.mlp.experts.105.down_proj.weight": "model-00040-of-00101.safetensors", + "model.layers.37.mlp.experts.106.gate_proj.weight": "model-00040-of-00101.safetensors", + "model.layers.37.mlp.experts.106.up_proj.weight": "model-00040-of-00101.safetensors", + "model.layers.37.mlp.experts.106.down_proj.weight": "model-00040-of-00101.safetensors", + "model.layers.37.mlp.experts.107.gate_proj.weight": "model-00040-of-00101.safetensors", + "model.layers.37.mlp.experts.107.up_proj.weight": "model-00040-of-00101.safetensors", + "model.layers.37.mlp.experts.107.down_proj.weight": "model-00040-of-00101.safetensors", + "model.layers.37.mlp.experts.108.gate_proj.weight": "model-00040-of-00101.safetensors", + "model.layers.37.mlp.experts.108.up_proj.weight": "model-00040-of-00101.safetensors", + "model.layers.37.mlp.experts.108.down_proj.weight": "model-00040-of-00101.safetensors", + "model.layers.37.mlp.experts.109.gate_proj.weight": "model-00040-of-00101.safetensors", + "model.layers.37.mlp.experts.109.up_proj.weight": "model-00040-of-00101.safetensors", + "model.layers.37.mlp.experts.109.down_proj.weight": "model-00040-of-00101.safetensors", + "model.layers.37.mlp.experts.110.gate_proj.weight": "model-00040-of-00101.safetensors", + "model.layers.37.mlp.experts.110.up_proj.weight": "model-00040-of-00101.safetensors", + "model.layers.37.mlp.experts.110.down_proj.weight": "model-00040-of-00101.safetensors", + "model.layers.37.mlp.experts.111.gate_proj.weight": "model-00040-of-00101.safetensors", + "model.layers.37.mlp.experts.111.up_proj.weight": "model-00040-of-00101.safetensors", + "model.layers.37.mlp.experts.111.down_proj.weight": "model-00040-of-00101.safetensors", + "model.layers.37.mlp.experts.112.gate_proj.weight": "model-00040-of-00101.safetensors", + "model.layers.37.mlp.experts.112.up_proj.weight": "model-00040-of-00101.safetensors", + "model.layers.37.mlp.experts.112.down_proj.weight": "model-00040-of-00101.safetensors", + "model.layers.37.mlp.experts.113.gate_proj.weight": "model-00040-of-00101.safetensors", + "model.layers.37.mlp.experts.113.up_proj.weight": "model-00040-of-00101.safetensors", + "model.layers.37.mlp.experts.113.down_proj.weight": "model-00040-of-00101.safetensors", + "model.layers.37.mlp.experts.114.gate_proj.weight": "model-00040-of-00101.safetensors", + "model.layers.37.mlp.experts.114.up_proj.weight": "model-00040-of-00101.safetensors", + "model.layers.37.mlp.experts.114.down_proj.weight": "model-00040-of-00101.safetensors", + "model.layers.37.mlp.experts.115.gate_proj.weight": "model-00040-of-00101.safetensors", + "model.layers.37.mlp.experts.115.up_proj.weight": "model-00040-of-00101.safetensors", + "model.layers.37.mlp.experts.115.down_proj.weight": "model-00040-of-00101.safetensors", + "model.layers.37.mlp.experts.116.gate_proj.weight": "model-00040-of-00101.safetensors", + "model.layers.37.mlp.experts.116.up_proj.weight": "model-00040-of-00101.safetensors", + "model.layers.37.mlp.experts.116.down_proj.weight": "model-00040-of-00101.safetensors", + "model.layers.37.mlp.experts.117.gate_proj.weight": "model-00040-of-00101.safetensors", + "model.layers.37.mlp.experts.117.up_proj.weight": "model-00040-of-00101.safetensors", + "model.layers.37.mlp.experts.117.down_proj.weight": "model-00040-of-00101.safetensors", + "model.layers.37.mlp.experts.118.gate_proj.weight": "model-00040-of-00101.safetensors", + "model.layers.37.mlp.experts.118.up_proj.weight": "model-00040-of-00101.safetensors", + "model.layers.37.mlp.experts.118.down_proj.weight": "model-00040-of-00101.safetensors", + "model.layers.37.mlp.experts.119.gate_proj.weight": "model-00040-of-00101.safetensors", + "model.layers.37.mlp.experts.119.up_proj.weight": "model-00040-of-00101.safetensors", + "model.layers.37.mlp.experts.119.down_proj.weight": "model-00040-of-00101.safetensors", + "model.layers.37.mlp.gate.weight": "model-00040-of-00101.safetensors", + "model.layers.37.mlp.gate.e_score_correction_bias": "model-00040-of-00101.safetensors", + "model.layers.37.mlp.shared_experts.gate_proj.weight": "model-00040-of-00101.safetensors", + "model.layers.37.mlp.shared_experts.up_proj.weight": "model-00040-of-00101.safetensors", + "model.layers.37.mlp.shared_experts.down_proj.weight": "model-00040-of-00101.safetensors", + "model.layers.37.input_layernorm.weight": "model-00040-of-00101.safetensors", + "model.layers.37.post_attention_layernorm.weight": "model-00040-of-00101.safetensors", + "model.layers.38.self_attn.q_proj.weight": "model-00040-of-00101.safetensors", + "model.layers.38.self_attn.q_proj.bias": "model-00040-of-00101.safetensors", + "model.layers.38.self_attn.k_proj.weight": "model-00040-of-00101.safetensors", + "model.layers.38.self_attn.k_proj.bias": "model-00040-of-00101.safetensors", + "model.layers.38.self_attn.v_proj.weight": "model-00040-of-00101.safetensors", + "model.layers.38.self_attn.v_proj.bias": "model-00040-of-00101.safetensors", + "model.layers.38.self_attn.o_proj.weight": "model-00040-of-00101.safetensors", + "model.layers.38.self_attn.q_norm.weight": "model-00040-of-00101.safetensors", + "model.layers.38.self_attn.k_norm.weight": "model-00040-of-00101.safetensors", + "model.layers.38.mlp.experts.0.gate_proj.weight": "model-00040-of-00101.safetensors", + "model.layers.38.mlp.experts.0.up_proj.weight": "model-00040-of-00101.safetensors", + "model.layers.38.mlp.experts.0.down_proj.weight": "model-00040-of-00101.safetensors", + "model.layers.38.mlp.experts.1.gate_proj.weight": "model-00040-of-00101.safetensors", + "model.layers.38.mlp.experts.1.up_proj.weight": "model-00040-of-00101.safetensors", + "model.layers.38.mlp.experts.1.down_proj.weight": "model-00040-of-00101.safetensors", + "model.layers.38.mlp.experts.2.gate_proj.weight": "model-00040-of-00101.safetensors", + "model.layers.38.mlp.experts.2.up_proj.weight": "model-00040-of-00101.safetensors", + "model.layers.38.mlp.experts.2.down_proj.weight": "model-00040-of-00101.safetensors", + "model.layers.38.mlp.experts.3.gate_proj.weight": "model-00040-of-00101.safetensors", + "model.layers.38.mlp.experts.3.up_proj.weight": "model-00040-of-00101.safetensors", + "model.layers.38.mlp.experts.3.down_proj.weight": "model-00040-of-00101.safetensors", + "model.layers.38.mlp.experts.4.gate_proj.weight": "model-00040-of-00101.safetensors", + "model.layers.38.mlp.experts.4.up_proj.weight": "model-00040-of-00101.safetensors", + "model.layers.38.mlp.experts.4.down_proj.weight": "model-00040-of-00101.safetensors", + "model.layers.38.mlp.experts.5.gate_proj.weight": "model-00040-of-00101.safetensors", + "model.layers.38.mlp.experts.5.up_proj.weight": "model-00040-of-00101.safetensors", + "model.layers.38.mlp.experts.5.down_proj.weight": "model-00040-of-00101.safetensors", + "model.layers.38.mlp.experts.6.gate_proj.weight": "model-00040-of-00101.safetensors", + "model.layers.38.mlp.experts.6.up_proj.weight": "model-00040-of-00101.safetensors", + "model.layers.38.mlp.experts.6.down_proj.weight": "model-00040-of-00101.safetensors", + "model.layers.38.mlp.experts.7.gate_proj.weight": "model-00040-of-00101.safetensors", + "model.layers.38.mlp.experts.7.up_proj.weight": "model-00040-of-00101.safetensors", + "model.layers.38.mlp.experts.7.down_proj.weight": "model-00040-of-00101.safetensors", + "model.layers.38.mlp.experts.8.gate_proj.weight": "model-00040-of-00101.safetensors", + "model.layers.38.mlp.experts.8.up_proj.weight": "model-00040-of-00101.safetensors", + "model.layers.38.mlp.experts.8.down_proj.weight": "model-00040-of-00101.safetensors", + "model.layers.38.mlp.experts.9.gate_proj.weight": "model-00040-of-00101.safetensors", + "model.layers.38.mlp.experts.9.up_proj.weight": "model-00040-of-00101.safetensors", + "model.layers.38.mlp.experts.9.down_proj.weight": "model-00040-of-00101.safetensors", + "model.layers.38.mlp.experts.10.gate_proj.weight": "model-00040-of-00101.safetensors", + "model.layers.38.mlp.experts.10.up_proj.weight": "model-00040-of-00101.safetensors", + "model.layers.38.mlp.experts.10.down_proj.weight": "model-00040-of-00101.safetensors", + "model.layers.38.mlp.experts.11.gate_proj.weight": "model-00040-of-00101.safetensors", + "model.layers.38.mlp.experts.11.up_proj.weight": "model-00040-of-00101.safetensors", + "model.layers.38.mlp.experts.11.down_proj.weight": "model-00040-of-00101.safetensors", + "model.layers.38.mlp.experts.12.gate_proj.weight": "model-00040-of-00101.safetensors", + "model.layers.38.mlp.experts.12.up_proj.weight": "model-00040-of-00101.safetensors", + "model.layers.38.mlp.experts.12.down_proj.weight": "model-00040-of-00101.safetensors", + "model.layers.38.mlp.experts.13.gate_proj.weight": "model-00040-of-00101.safetensors", + "model.layers.38.mlp.experts.13.up_proj.weight": "model-00040-of-00101.safetensors", + "model.layers.38.mlp.experts.13.down_proj.weight": "model-00040-of-00101.safetensors", + "model.layers.38.mlp.experts.14.gate_proj.weight": "model-00040-of-00101.safetensors", + "model.layers.38.mlp.experts.14.up_proj.weight": "model-00040-of-00101.safetensors", + "model.layers.38.mlp.experts.14.down_proj.weight": "model-00040-of-00101.safetensors", + "model.layers.38.mlp.experts.15.gate_proj.weight": "model-00040-of-00101.safetensors", + "model.layers.38.mlp.experts.15.up_proj.weight": "model-00040-of-00101.safetensors", + "model.layers.38.mlp.experts.15.down_proj.weight": "model-00040-of-00101.safetensors", + "model.layers.38.mlp.experts.16.gate_proj.weight": "model-00040-of-00101.safetensors", + "model.layers.38.mlp.experts.16.up_proj.weight": "model-00040-of-00101.safetensors", + "model.layers.38.mlp.experts.16.down_proj.weight": "model-00040-of-00101.safetensors", + "model.layers.38.mlp.experts.17.gate_proj.weight": "model-00040-of-00101.safetensors", + "model.layers.38.mlp.experts.17.up_proj.weight": "model-00040-of-00101.safetensors", + "model.layers.38.mlp.experts.17.down_proj.weight": "model-00040-of-00101.safetensors", + "model.layers.38.mlp.experts.18.gate_proj.weight": "model-00040-of-00101.safetensors", + "model.layers.38.mlp.experts.18.up_proj.weight": "model-00040-of-00101.safetensors", + "model.layers.38.mlp.experts.18.down_proj.weight": "model-00040-of-00101.safetensors", + "model.layers.38.mlp.experts.19.gate_proj.weight": "model-00040-of-00101.safetensors", + "model.layers.38.mlp.experts.19.up_proj.weight": "model-00040-of-00101.safetensors", + "model.layers.38.mlp.experts.19.down_proj.weight": "model-00040-of-00101.safetensors", + "model.layers.38.mlp.experts.20.gate_proj.weight": "model-00040-of-00101.safetensors", + "model.layers.38.mlp.experts.20.up_proj.weight": "model-00040-of-00101.safetensors", + "model.layers.38.mlp.experts.20.down_proj.weight": "model-00040-of-00101.safetensors", + "model.layers.38.mlp.experts.21.gate_proj.weight": "model-00040-of-00101.safetensors", + "model.layers.38.mlp.experts.21.up_proj.weight": "model-00041-of-00101.safetensors", + "model.layers.38.mlp.experts.21.down_proj.weight": "model-00041-of-00101.safetensors", + "model.layers.38.mlp.experts.22.gate_proj.weight": "model-00041-of-00101.safetensors", + "model.layers.38.mlp.experts.22.up_proj.weight": "model-00041-of-00101.safetensors", + "model.layers.38.mlp.experts.22.down_proj.weight": "model-00041-of-00101.safetensors", + "model.layers.38.mlp.experts.23.gate_proj.weight": "model-00041-of-00101.safetensors", + "model.layers.38.mlp.experts.23.up_proj.weight": "model-00041-of-00101.safetensors", + "model.layers.38.mlp.experts.23.down_proj.weight": "model-00041-of-00101.safetensors", + "model.layers.38.mlp.experts.24.gate_proj.weight": "model-00041-of-00101.safetensors", + "model.layers.38.mlp.experts.24.up_proj.weight": "model-00041-of-00101.safetensors", + "model.layers.38.mlp.experts.24.down_proj.weight": "model-00041-of-00101.safetensors", + "model.layers.38.mlp.experts.25.gate_proj.weight": "model-00041-of-00101.safetensors", + "model.layers.38.mlp.experts.25.up_proj.weight": "model-00041-of-00101.safetensors", + "model.layers.38.mlp.experts.25.down_proj.weight": "model-00041-of-00101.safetensors", + "model.layers.38.mlp.experts.26.gate_proj.weight": "model-00041-of-00101.safetensors", + "model.layers.38.mlp.experts.26.up_proj.weight": "model-00041-of-00101.safetensors", + "model.layers.38.mlp.experts.26.down_proj.weight": "model-00041-of-00101.safetensors", + "model.layers.38.mlp.experts.27.gate_proj.weight": "model-00041-of-00101.safetensors", + "model.layers.38.mlp.experts.27.up_proj.weight": "model-00041-of-00101.safetensors", + "model.layers.38.mlp.experts.27.down_proj.weight": "model-00041-of-00101.safetensors", + "model.layers.38.mlp.experts.28.gate_proj.weight": "model-00041-of-00101.safetensors", + "model.layers.38.mlp.experts.28.up_proj.weight": "model-00041-of-00101.safetensors", + "model.layers.38.mlp.experts.28.down_proj.weight": "model-00041-of-00101.safetensors", + "model.layers.38.mlp.experts.29.gate_proj.weight": "model-00041-of-00101.safetensors", + "model.layers.38.mlp.experts.29.up_proj.weight": "model-00041-of-00101.safetensors", + "model.layers.38.mlp.experts.29.down_proj.weight": "model-00041-of-00101.safetensors", + "model.layers.38.mlp.experts.30.gate_proj.weight": "model-00041-of-00101.safetensors", + "model.layers.38.mlp.experts.30.up_proj.weight": "model-00041-of-00101.safetensors", + "model.layers.38.mlp.experts.30.down_proj.weight": "model-00041-of-00101.safetensors", + "model.layers.38.mlp.experts.31.gate_proj.weight": "model-00041-of-00101.safetensors", + "model.layers.38.mlp.experts.31.up_proj.weight": "model-00041-of-00101.safetensors", + "model.layers.38.mlp.experts.31.down_proj.weight": "model-00041-of-00101.safetensors", + "model.layers.38.mlp.experts.32.gate_proj.weight": "model-00041-of-00101.safetensors", + "model.layers.38.mlp.experts.32.up_proj.weight": "model-00041-of-00101.safetensors", + "model.layers.38.mlp.experts.32.down_proj.weight": "model-00041-of-00101.safetensors", + "model.layers.38.mlp.experts.33.gate_proj.weight": "model-00041-of-00101.safetensors", + "model.layers.38.mlp.experts.33.up_proj.weight": "model-00041-of-00101.safetensors", + "model.layers.38.mlp.experts.33.down_proj.weight": "model-00041-of-00101.safetensors", + "model.layers.38.mlp.experts.34.gate_proj.weight": "model-00041-of-00101.safetensors", + "model.layers.38.mlp.experts.34.up_proj.weight": "model-00041-of-00101.safetensors", + "model.layers.38.mlp.experts.34.down_proj.weight": "model-00041-of-00101.safetensors", + "model.layers.38.mlp.experts.35.gate_proj.weight": "model-00041-of-00101.safetensors", + "model.layers.38.mlp.experts.35.up_proj.weight": "model-00041-of-00101.safetensors", + "model.layers.38.mlp.experts.35.down_proj.weight": "model-00041-of-00101.safetensors", + "model.layers.38.mlp.experts.36.gate_proj.weight": "model-00041-of-00101.safetensors", + "model.layers.38.mlp.experts.36.up_proj.weight": "model-00041-of-00101.safetensors", + "model.layers.38.mlp.experts.36.down_proj.weight": "model-00041-of-00101.safetensors", + "model.layers.38.mlp.experts.37.gate_proj.weight": "model-00041-of-00101.safetensors", + "model.layers.38.mlp.experts.37.up_proj.weight": "model-00041-of-00101.safetensors", + "model.layers.38.mlp.experts.37.down_proj.weight": "model-00041-of-00101.safetensors", + "model.layers.38.mlp.experts.38.gate_proj.weight": "model-00041-of-00101.safetensors", + "model.layers.38.mlp.experts.38.up_proj.weight": "model-00041-of-00101.safetensors", + "model.layers.38.mlp.experts.38.down_proj.weight": "model-00041-of-00101.safetensors", + "model.layers.38.mlp.experts.39.gate_proj.weight": "model-00041-of-00101.safetensors", + "model.layers.38.mlp.experts.39.up_proj.weight": "model-00041-of-00101.safetensors", + "model.layers.38.mlp.experts.39.down_proj.weight": "model-00041-of-00101.safetensors", + "model.layers.38.mlp.experts.40.gate_proj.weight": "model-00041-of-00101.safetensors", + "model.layers.38.mlp.experts.40.up_proj.weight": "model-00041-of-00101.safetensors", + "model.layers.38.mlp.experts.40.down_proj.weight": "model-00041-of-00101.safetensors", + "model.layers.38.mlp.experts.41.gate_proj.weight": "model-00041-of-00101.safetensors", + "model.layers.38.mlp.experts.41.up_proj.weight": "model-00041-of-00101.safetensors", + "model.layers.38.mlp.experts.41.down_proj.weight": "model-00041-of-00101.safetensors", + "model.layers.38.mlp.experts.42.gate_proj.weight": "model-00041-of-00101.safetensors", + "model.layers.38.mlp.experts.42.up_proj.weight": "model-00041-of-00101.safetensors", + "model.layers.38.mlp.experts.42.down_proj.weight": "model-00041-of-00101.safetensors", + "model.layers.38.mlp.experts.43.gate_proj.weight": "model-00041-of-00101.safetensors", + "model.layers.38.mlp.experts.43.up_proj.weight": "model-00041-of-00101.safetensors", + "model.layers.38.mlp.experts.43.down_proj.weight": "model-00041-of-00101.safetensors", + "model.layers.38.mlp.experts.44.gate_proj.weight": "model-00041-of-00101.safetensors", + "model.layers.38.mlp.experts.44.up_proj.weight": "model-00041-of-00101.safetensors", + "model.layers.38.mlp.experts.44.down_proj.weight": "model-00041-of-00101.safetensors", + "model.layers.38.mlp.experts.45.gate_proj.weight": "model-00041-of-00101.safetensors", + "model.layers.38.mlp.experts.45.up_proj.weight": "model-00041-of-00101.safetensors", + "model.layers.38.mlp.experts.45.down_proj.weight": "model-00041-of-00101.safetensors", + "model.layers.38.mlp.experts.46.gate_proj.weight": "model-00041-of-00101.safetensors", + "model.layers.38.mlp.experts.46.up_proj.weight": "model-00041-of-00101.safetensors", + "model.layers.38.mlp.experts.46.down_proj.weight": "model-00041-of-00101.safetensors", + "model.layers.38.mlp.experts.47.gate_proj.weight": "model-00041-of-00101.safetensors", + "model.layers.38.mlp.experts.47.up_proj.weight": "model-00041-of-00101.safetensors", + "model.layers.38.mlp.experts.47.down_proj.weight": "model-00041-of-00101.safetensors", + "model.layers.38.mlp.experts.48.gate_proj.weight": "model-00041-of-00101.safetensors", + "model.layers.38.mlp.experts.48.up_proj.weight": "model-00041-of-00101.safetensors", + "model.layers.38.mlp.experts.48.down_proj.weight": "model-00041-of-00101.safetensors", + "model.layers.38.mlp.experts.49.gate_proj.weight": "model-00041-of-00101.safetensors", + "model.layers.38.mlp.experts.49.up_proj.weight": "model-00041-of-00101.safetensors", + "model.layers.38.mlp.experts.49.down_proj.weight": "model-00041-of-00101.safetensors", + "model.layers.38.mlp.experts.50.gate_proj.weight": "model-00041-of-00101.safetensors", + "model.layers.38.mlp.experts.50.up_proj.weight": "model-00041-of-00101.safetensors", + "model.layers.38.mlp.experts.50.down_proj.weight": "model-00041-of-00101.safetensors", + "model.layers.38.mlp.experts.51.gate_proj.weight": "model-00041-of-00101.safetensors", + "model.layers.38.mlp.experts.51.up_proj.weight": "model-00041-of-00101.safetensors", + "model.layers.38.mlp.experts.51.down_proj.weight": "model-00041-of-00101.safetensors", + "model.layers.38.mlp.experts.52.gate_proj.weight": "model-00041-of-00101.safetensors", + "model.layers.38.mlp.experts.52.up_proj.weight": "model-00041-of-00101.safetensors", + "model.layers.38.mlp.experts.52.down_proj.weight": "model-00041-of-00101.safetensors", + "model.layers.38.mlp.experts.53.gate_proj.weight": "model-00041-of-00101.safetensors", + "model.layers.38.mlp.experts.53.up_proj.weight": "model-00041-of-00101.safetensors", + "model.layers.38.mlp.experts.53.down_proj.weight": "model-00041-of-00101.safetensors", + "model.layers.38.mlp.experts.54.gate_proj.weight": "model-00041-of-00101.safetensors", + "model.layers.38.mlp.experts.54.up_proj.weight": "model-00041-of-00101.safetensors", + "model.layers.38.mlp.experts.54.down_proj.weight": "model-00041-of-00101.safetensors", + "model.layers.38.mlp.experts.55.gate_proj.weight": "model-00041-of-00101.safetensors", + "model.layers.38.mlp.experts.55.up_proj.weight": "model-00041-of-00101.safetensors", + "model.layers.38.mlp.experts.55.down_proj.weight": "model-00041-of-00101.safetensors", + "model.layers.38.mlp.experts.56.gate_proj.weight": "model-00041-of-00101.safetensors", + "model.layers.38.mlp.experts.56.up_proj.weight": "model-00041-of-00101.safetensors", + "model.layers.38.mlp.experts.56.down_proj.weight": "model-00041-of-00101.safetensors", + "model.layers.38.mlp.experts.57.gate_proj.weight": "model-00041-of-00101.safetensors", + "model.layers.38.mlp.experts.57.up_proj.weight": "model-00041-of-00101.safetensors", + "model.layers.38.mlp.experts.57.down_proj.weight": "model-00041-of-00101.safetensors", + "model.layers.38.mlp.experts.58.gate_proj.weight": "model-00041-of-00101.safetensors", + "model.layers.38.mlp.experts.58.up_proj.weight": "model-00041-of-00101.safetensors", + "model.layers.38.mlp.experts.58.down_proj.weight": "model-00041-of-00101.safetensors", + "model.layers.38.mlp.experts.59.gate_proj.weight": "model-00041-of-00101.safetensors", + "model.layers.38.mlp.experts.59.up_proj.weight": "model-00041-of-00101.safetensors", + "model.layers.38.mlp.experts.59.down_proj.weight": "model-00041-of-00101.safetensors", + "model.layers.38.mlp.experts.60.gate_proj.weight": "model-00041-of-00101.safetensors", + "model.layers.38.mlp.experts.60.up_proj.weight": "model-00041-of-00101.safetensors", + "model.layers.38.mlp.experts.60.down_proj.weight": "model-00041-of-00101.safetensors", + "model.layers.38.mlp.experts.61.gate_proj.weight": "model-00041-of-00101.safetensors", + "model.layers.38.mlp.experts.61.up_proj.weight": "model-00041-of-00101.safetensors", + "model.layers.38.mlp.experts.61.down_proj.weight": "model-00041-of-00101.safetensors", + "model.layers.38.mlp.experts.62.gate_proj.weight": "model-00041-of-00101.safetensors", + "model.layers.38.mlp.experts.62.up_proj.weight": "model-00041-of-00101.safetensors", + "model.layers.38.mlp.experts.62.down_proj.weight": "model-00041-of-00101.safetensors", + "model.layers.38.mlp.experts.63.gate_proj.weight": "model-00041-of-00101.safetensors", + "model.layers.38.mlp.experts.63.up_proj.weight": "model-00041-of-00101.safetensors", + "model.layers.38.mlp.experts.63.down_proj.weight": "model-00041-of-00101.safetensors", + "model.layers.38.mlp.experts.64.gate_proj.weight": "model-00041-of-00101.safetensors", + "model.layers.38.mlp.experts.64.up_proj.weight": "model-00041-of-00101.safetensors", + "model.layers.38.mlp.experts.64.down_proj.weight": "model-00041-of-00101.safetensors", + "model.layers.38.mlp.experts.65.gate_proj.weight": "model-00041-of-00101.safetensors", + "model.layers.38.mlp.experts.65.up_proj.weight": "model-00041-of-00101.safetensors", + "model.layers.38.mlp.experts.65.down_proj.weight": "model-00041-of-00101.safetensors", + "model.layers.38.mlp.experts.66.gate_proj.weight": "model-00041-of-00101.safetensors", + "model.layers.38.mlp.experts.66.up_proj.weight": "model-00041-of-00101.safetensors", + "model.layers.38.mlp.experts.66.down_proj.weight": "model-00041-of-00101.safetensors", + "model.layers.38.mlp.experts.67.gate_proj.weight": "model-00041-of-00101.safetensors", + "model.layers.38.mlp.experts.67.up_proj.weight": "model-00041-of-00101.safetensors", + "model.layers.38.mlp.experts.67.down_proj.weight": "model-00041-of-00101.safetensors", + "model.layers.38.mlp.experts.68.gate_proj.weight": "model-00041-of-00101.safetensors", + "model.layers.38.mlp.experts.68.up_proj.weight": "model-00041-of-00101.safetensors", + "model.layers.38.mlp.experts.68.down_proj.weight": "model-00041-of-00101.safetensors", + "model.layers.38.mlp.experts.69.gate_proj.weight": "model-00041-of-00101.safetensors", + "model.layers.38.mlp.experts.69.up_proj.weight": "model-00041-of-00101.safetensors", + "model.layers.38.mlp.experts.69.down_proj.weight": "model-00041-of-00101.safetensors", + "model.layers.38.mlp.experts.70.gate_proj.weight": "model-00041-of-00101.safetensors", + "model.layers.38.mlp.experts.70.up_proj.weight": "model-00041-of-00101.safetensors", + "model.layers.38.mlp.experts.70.down_proj.weight": "model-00041-of-00101.safetensors", + "model.layers.38.mlp.experts.71.gate_proj.weight": "model-00041-of-00101.safetensors", + "model.layers.38.mlp.experts.71.up_proj.weight": "model-00041-of-00101.safetensors", + "model.layers.38.mlp.experts.71.down_proj.weight": "model-00041-of-00101.safetensors", + "model.layers.38.mlp.experts.72.gate_proj.weight": "model-00041-of-00101.safetensors", + "model.layers.38.mlp.experts.72.up_proj.weight": "model-00041-of-00101.safetensors", + "model.layers.38.mlp.experts.72.down_proj.weight": "model-00041-of-00101.safetensors", + "model.layers.38.mlp.experts.73.gate_proj.weight": "model-00041-of-00101.safetensors", + "model.layers.38.mlp.experts.73.up_proj.weight": "model-00041-of-00101.safetensors", + "model.layers.38.mlp.experts.73.down_proj.weight": "model-00041-of-00101.safetensors", + "model.layers.38.mlp.experts.74.gate_proj.weight": "model-00041-of-00101.safetensors", + "model.layers.38.mlp.experts.74.up_proj.weight": "model-00041-of-00101.safetensors", + "model.layers.38.mlp.experts.74.down_proj.weight": "model-00041-of-00101.safetensors", + "model.layers.38.mlp.experts.75.gate_proj.weight": "model-00041-of-00101.safetensors", + "model.layers.38.mlp.experts.75.up_proj.weight": "model-00041-of-00101.safetensors", + "model.layers.38.mlp.experts.75.down_proj.weight": "model-00041-of-00101.safetensors", + "model.layers.38.mlp.experts.76.gate_proj.weight": "model-00041-of-00101.safetensors", + "model.layers.38.mlp.experts.76.up_proj.weight": "model-00041-of-00101.safetensors", + "model.layers.38.mlp.experts.76.down_proj.weight": "model-00041-of-00101.safetensors", + "model.layers.38.mlp.experts.77.gate_proj.weight": "model-00041-of-00101.safetensors", + "model.layers.38.mlp.experts.77.up_proj.weight": "model-00041-of-00101.safetensors", + "model.layers.38.mlp.experts.77.down_proj.weight": "model-00041-of-00101.safetensors", + "model.layers.38.mlp.experts.78.gate_proj.weight": "model-00041-of-00101.safetensors", + "model.layers.38.mlp.experts.78.up_proj.weight": "model-00041-of-00101.safetensors", + "model.layers.38.mlp.experts.78.down_proj.weight": "model-00041-of-00101.safetensors", + "model.layers.38.mlp.experts.79.gate_proj.weight": "model-00041-of-00101.safetensors", + "model.layers.38.mlp.experts.79.up_proj.weight": "model-00041-of-00101.safetensors", + "model.layers.38.mlp.experts.79.down_proj.weight": "model-00041-of-00101.safetensors", + "model.layers.38.mlp.experts.80.gate_proj.weight": "model-00041-of-00101.safetensors", + "model.layers.38.mlp.experts.80.up_proj.weight": "model-00041-of-00101.safetensors", + "model.layers.38.mlp.experts.80.down_proj.weight": "model-00041-of-00101.safetensors", + "model.layers.38.mlp.experts.81.gate_proj.weight": "model-00041-of-00101.safetensors", + "model.layers.38.mlp.experts.81.up_proj.weight": "model-00041-of-00101.safetensors", + "model.layers.38.mlp.experts.81.down_proj.weight": "model-00041-of-00101.safetensors", + "model.layers.38.mlp.experts.82.gate_proj.weight": "model-00041-of-00101.safetensors", + "model.layers.38.mlp.experts.82.up_proj.weight": "model-00041-of-00101.safetensors", + "model.layers.38.mlp.experts.82.down_proj.weight": "model-00041-of-00101.safetensors", + "model.layers.38.mlp.experts.83.gate_proj.weight": "model-00041-of-00101.safetensors", + "model.layers.38.mlp.experts.83.up_proj.weight": "model-00041-of-00101.safetensors", + "model.layers.38.mlp.experts.83.down_proj.weight": "model-00041-of-00101.safetensors", + "model.layers.38.mlp.experts.84.gate_proj.weight": "model-00041-of-00101.safetensors", + "model.layers.38.mlp.experts.84.up_proj.weight": "model-00041-of-00101.safetensors", + "model.layers.38.mlp.experts.84.down_proj.weight": "model-00041-of-00101.safetensors", + "model.layers.38.mlp.experts.85.gate_proj.weight": "model-00041-of-00101.safetensors", + "model.layers.38.mlp.experts.85.up_proj.weight": "model-00041-of-00101.safetensors", + "model.layers.38.mlp.experts.85.down_proj.weight": "model-00041-of-00101.safetensors", + "model.layers.38.mlp.experts.86.gate_proj.weight": "model-00041-of-00101.safetensors", + "model.layers.38.mlp.experts.86.up_proj.weight": "model-00041-of-00101.safetensors", + "model.layers.38.mlp.experts.86.down_proj.weight": "model-00041-of-00101.safetensors", + "model.layers.38.mlp.experts.87.gate_proj.weight": "model-00041-of-00101.safetensors", + "model.layers.38.mlp.experts.87.up_proj.weight": "model-00041-of-00101.safetensors", + "model.layers.38.mlp.experts.87.down_proj.weight": "model-00041-of-00101.safetensors", + "model.layers.38.mlp.experts.88.gate_proj.weight": "model-00041-of-00101.safetensors", + "model.layers.38.mlp.experts.88.up_proj.weight": "model-00041-of-00101.safetensors", + "model.layers.38.mlp.experts.88.down_proj.weight": "model-00041-of-00101.safetensors", + "model.layers.38.mlp.experts.89.gate_proj.weight": "model-00041-of-00101.safetensors", + "model.layers.38.mlp.experts.89.up_proj.weight": "model-00041-of-00101.safetensors", + "model.layers.38.mlp.experts.89.down_proj.weight": "model-00041-of-00101.safetensors", + "model.layers.38.mlp.experts.90.gate_proj.weight": "model-00041-of-00101.safetensors", + "model.layers.38.mlp.experts.90.up_proj.weight": "model-00041-of-00101.safetensors", + "model.layers.38.mlp.experts.90.down_proj.weight": "model-00041-of-00101.safetensors", + "model.layers.38.mlp.experts.91.gate_proj.weight": "model-00041-of-00101.safetensors", + "model.layers.38.mlp.experts.91.up_proj.weight": "model-00041-of-00101.safetensors", + "model.layers.38.mlp.experts.91.down_proj.weight": "model-00041-of-00101.safetensors", + "model.layers.38.mlp.experts.92.gate_proj.weight": "model-00041-of-00101.safetensors", + "model.layers.38.mlp.experts.92.up_proj.weight": "model-00041-of-00101.safetensors", + "model.layers.38.mlp.experts.92.down_proj.weight": "model-00041-of-00101.safetensors", + "model.layers.38.mlp.experts.93.gate_proj.weight": "model-00041-of-00101.safetensors", + "model.layers.38.mlp.experts.93.up_proj.weight": "model-00041-of-00101.safetensors", + "model.layers.38.mlp.experts.93.down_proj.weight": "model-00041-of-00101.safetensors", + "model.layers.38.mlp.experts.94.gate_proj.weight": "model-00041-of-00101.safetensors", + "model.layers.38.mlp.experts.94.up_proj.weight": "model-00041-of-00101.safetensors", + "model.layers.38.mlp.experts.94.down_proj.weight": "model-00041-of-00101.safetensors", + "model.layers.38.mlp.experts.95.gate_proj.weight": "model-00041-of-00101.safetensors", + "model.layers.38.mlp.experts.95.up_proj.weight": "model-00041-of-00101.safetensors", + "model.layers.38.mlp.experts.95.down_proj.weight": "model-00041-of-00101.safetensors", + "model.layers.38.mlp.experts.96.gate_proj.weight": "model-00041-of-00101.safetensors", + "model.layers.38.mlp.experts.96.up_proj.weight": "model-00041-of-00101.safetensors", + "model.layers.38.mlp.experts.96.down_proj.weight": "model-00041-of-00101.safetensors", + "model.layers.38.mlp.experts.97.gate_proj.weight": "model-00041-of-00101.safetensors", + "model.layers.38.mlp.experts.97.up_proj.weight": "model-00041-of-00101.safetensors", + "model.layers.38.mlp.experts.97.down_proj.weight": "model-00041-of-00101.safetensors", + "model.layers.38.mlp.experts.98.gate_proj.weight": "model-00041-of-00101.safetensors", + "model.layers.38.mlp.experts.98.up_proj.weight": "model-00041-of-00101.safetensors", + "model.layers.38.mlp.experts.98.down_proj.weight": "model-00041-of-00101.safetensors", + "model.layers.38.mlp.experts.99.gate_proj.weight": "model-00041-of-00101.safetensors", + "model.layers.38.mlp.experts.99.up_proj.weight": "model-00041-of-00101.safetensors", + "model.layers.38.mlp.experts.99.down_proj.weight": "model-00041-of-00101.safetensors", + "model.layers.38.mlp.experts.100.gate_proj.weight": "model-00041-of-00101.safetensors", + "model.layers.38.mlp.experts.100.up_proj.weight": "model-00041-of-00101.safetensors", + "model.layers.38.mlp.experts.100.down_proj.weight": "model-00041-of-00101.safetensors", + "model.layers.38.mlp.experts.101.gate_proj.weight": "model-00041-of-00101.safetensors", + "model.layers.38.mlp.experts.101.up_proj.weight": "model-00041-of-00101.safetensors", + "model.layers.38.mlp.experts.101.down_proj.weight": "model-00041-of-00101.safetensors", + "model.layers.38.mlp.experts.102.gate_proj.weight": "model-00041-of-00101.safetensors", + "model.layers.38.mlp.experts.102.up_proj.weight": "model-00041-of-00101.safetensors", + "model.layers.38.mlp.experts.102.down_proj.weight": "model-00041-of-00101.safetensors", + "model.layers.38.mlp.experts.103.gate_proj.weight": "model-00041-of-00101.safetensors", + "model.layers.38.mlp.experts.103.up_proj.weight": "model-00041-of-00101.safetensors", + "model.layers.38.mlp.experts.103.down_proj.weight": "model-00041-of-00101.safetensors", + "model.layers.38.mlp.experts.104.gate_proj.weight": "model-00041-of-00101.safetensors", + "model.layers.38.mlp.experts.104.up_proj.weight": "model-00041-of-00101.safetensors", + "model.layers.38.mlp.experts.104.down_proj.weight": "model-00041-of-00101.safetensors", + "model.layers.38.mlp.experts.105.gate_proj.weight": "model-00041-of-00101.safetensors", + "model.layers.38.mlp.experts.105.up_proj.weight": "model-00041-of-00101.safetensors", + "model.layers.38.mlp.experts.105.down_proj.weight": "model-00041-of-00101.safetensors", + "model.layers.38.mlp.experts.106.gate_proj.weight": "model-00041-of-00101.safetensors", + "model.layers.38.mlp.experts.106.up_proj.weight": "model-00041-of-00101.safetensors", + "model.layers.38.mlp.experts.106.down_proj.weight": "model-00041-of-00101.safetensors", + "model.layers.38.mlp.experts.107.gate_proj.weight": "model-00041-of-00101.safetensors", + "model.layers.38.mlp.experts.107.up_proj.weight": "model-00041-of-00101.safetensors", + "model.layers.38.mlp.experts.107.down_proj.weight": "model-00041-of-00101.safetensors", + "model.layers.38.mlp.experts.108.gate_proj.weight": "model-00041-of-00101.safetensors", + "model.layers.38.mlp.experts.108.up_proj.weight": "model-00041-of-00101.safetensors", + "model.layers.38.mlp.experts.108.down_proj.weight": "model-00041-of-00101.safetensors", + "model.layers.38.mlp.experts.109.gate_proj.weight": "model-00041-of-00101.safetensors", + "model.layers.38.mlp.experts.109.up_proj.weight": "model-00041-of-00101.safetensors", + "model.layers.38.mlp.experts.109.down_proj.weight": "model-00041-of-00101.safetensors", + "model.layers.38.mlp.experts.110.gate_proj.weight": "model-00041-of-00101.safetensors", + "model.layers.38.mlp.experts.110.up_proj.weight": "model-00041-of-00101.safetensors", + "model.layers.38.mlp.experts.110.down_proj.weight": "model-00041-of-00101.safetensors", + "model.layers.38.mlp.experts.111.gate_proj.weight": "model-00041-of-00101.safetensors", + "model.layers.38.mlp.experts.111.up_proj.weight": "model-00041-of-00101.safetensors", + "model.layers.38.mlp.experts.111.down_proj.weight": "model-00041-of-00101.safetensors", + "model.layers.38.mlp.experts.112.gate_proj.weight": "model-00041-of-00101.safetensors", + "model.layers.38.mlp.experts.112.up_proj.weight": "model-00041-of-00101.safetensors", + "model.layers.38.mlp.experts.112.down_proj.weight": "model-00041-of-00101.safetensors", + "model.layers.38.mlp.experts.113.gate_proj.weight": "model-00041-of-00101.safetensors", + "model.layers.38.mlp.experts.113.up_proj.weight": "model-00041-of-00101.safetensors", + "model.layers.38.mlp.experts.113.down_proj.weight": "model-00041-of-00101.safetensors", + "model.layers.38.mlp.experts.114.gate_proj.weight": "model-00041-of-00101.safetensors", + "model.layers.38.mlp.experts.114.up_proj.weight": "model-00041-of-00101.safetensors", + "model.layers.38.mlp.experts.114.down_proj.weight": "model-00041-of-00101.safetensors", + "model.layers.38.mlp.experts.115.gate_proj.weight": "model-00041-of-00101.safetensors", + "model.layers.38.mlp.experts.115.up_proj.weight": "model-00041-of-00101.safetensors", + "model.layers.38.mlp.experts.115.down_proj.weight": "model-00041-of-00101.safetensors", + "model.layers.38.mlp.experts.116.gate_proj.weight": "model-00041-of-00101.safetensors", + "model.layers.38.mlp.experts.116.up_proj.weight": "model-00041-of-00101.safetensors", + "model.layers.38.mlp.experts.116.down_proj.weight": "model-00041-of-00101.safetensors", + "model.layers.38.mlp.experts.117.gate_proj.weight": "model-00041-of-00101.safetensors", + "model.layers.38.mlp.experts.117.up_proj.weight": "model-00041-of-00101.safetensors", + "model.layers.38.mlp.experts.117.down_proj.weight": "model-00041-of-00101.safetensors", + "model.layers.38.mlp.experts.118.gate_proj.weight": "model-00041-of-00101.safetensors", + "model.layers.38.mlp.experts.118.up_proj.weight": "model-00041-of-00101.safetensors", + "model.layers.38.mlp.experts.118.down_proj.weight": "model-00041-of-00101.safetensors", + "model.layers.38.mlp.experts.119.gate_proj.weight": "model-00041-of-00101.safetensors", + "model.layers.38.mlp.experts.119.up_proj.weight": "model-00041-of-00101.safetensors", + "model.layers.38.mlp.experts.119.down_proj.weight": "model-00041-of-00101.safetensors", + "model.layers.38.mlp.gate.weight": "model-00041-of-00101.safetensors", + "model.layers.38.mlp.gate.e_score_correction_bias": "model-00041-of-00101.safetensors", + "model.layers.38.mlp.shared_experts.gate_proj.weight": "model-00041-of-00101.safetensors", + "model.layers.38.mlp.shared_experts.up_proj.weight": "model-00041-of-00101.safetensors", + "model.layers.38.mlp.shared_experts.down_proj.weight": "model-00041-of-00101.safetensors", + "model.layers.38.input_layernorm.weight": "model-00041-of-00101.safetensors", + "model.layers.38.post_attention_layernorm.weight": "model-00041-of-00101.safetensors", + "model.layers.39.self_attn.q_proj.weight": "model-00041-of-00101.safetensors", + "model.layers.39.self_attn.q_proj.bias": "model-00041-of-00101.safetensors", + "model.layers.39.self_attn.k_proj.weight": "model-00041-of-00101.safetensors", + "model.layers.39.self_attn.k_proj.bias": "model-00041-of-00101.safetensors", + "model.layers.39.self_attn.v_proj.weight": "model-00041-of-00101.safetensors", + "model.layers.39.self_attn.v_proj.bias": "model-00041-of-00101.safetensors", + "model.layers.39.self_attn.o_proj.weight": "model-00041-of-00101.safetensors", + "model.layers.39.self_attn.q_norm.weight": "model-00041-of-00101.safetensors", + "model.layers.39.self_attn.k_norm.weight": "model-00041-of-00101.safetensors", + "model.layers.39.mlp.experts.0.gate_proj.weight": "model-00041-of-00101.safetensors", + "model.layers.39.mlp.experts.0.up_proj.weight": "model-00041-of-00101.safetensors", + "model.layers.39.mlp.experts.0.down_proj.weight": "model-00041-of-00101.safetensors", + "model.layers.39.mlp.experts.1.gate_proj.weight": "model-00041-of-00101.safetensors", + "model.layers.39.mlp.experts.1.up_proj.weight": "model-00041-of-00101.safetensors", + "model.layers.39.mlp.experts.1.down_proj.weight": "model-00041-of-00101.safetensors", + "model.layers.39.mlp.experts.2.gate_proj.weight": "model-00041-of-00101.safetensors", + "model.layers.39.mlp.experts.2.up_proj.weight": "model-00041-of-00101.safetensors", + "model.layers.39.mlp.experts.2.down_proj.weight": "model-00041-of-00101.safetensors", + "model.layers.39.mlp.experts.3.gate_proj.weight": "model-00041-of-00101.safetensors", + "model.layers.39.mlp.experts.3.up_proj.weight": "model-00041-of-00101.safetensors", + "model.layers.39.mlp.experts.3.down_proj.weight": "model-00041-of-00101.safetensors", + "model.layers.39.mlp.experts.4.gate_proj.weight": "model-00041-of-00101.safetensors", + "model.layers.39.mlp.experts.4.up_proj.weight": "model-00041-of-00101.safetensors", + "model.layers.39.mlp.experts.4.down_proj.weight": "model-00041-of-00101.safetensors", + "model.layers.39.mlp.experts.5.gate_proj.weight": "model-00041-of-00101.safetensors", + "model.layers.39.mlp.experts.5.up_proj.weight": "model-00041-of-00101.safetensors", + "model.layers.39.mlp.experts.5.down_proj.weight": "model-00041-of-00101.safetensors", + "model.layers.39.mlp.experts.6.gate_proj.weight": "model-00041-of-00101.safetensors", + "model.layers.39.mlp.experts.6.up_proj.weight": "model-00041-of-00101.safetensors", + "model.layers.39.mlp.experts.6.down_proj.weight": "model-00041-of-00101.safetensors", + "model.layers.39.mlp.experts.7.gate_proj.weight": "model-00041-of-00101.safetensors", + "model.layers.39.mlp.experts.7.up_proj.weight": "model-00041-of-00101.safetensors", + "model.layers.39.mlp.experts.7.down_proj.weight": "model-00041-of-00101.safetensors", + "model.layers.39.mlp.experts.8.gate_proj.weight": "model-00042-of-00101.safetensors", + "model.layers.39.mlp.experts.8.up_proj.weight": "model-00042-of-00101.safetensors", + "model.layers.39.mlp.experts.8.down_proj.weight": "model-00042-of-00101.safetensors", + "model.layers.39.mlp.experts.9.gate_proj.weight": "model-00042-of-00101.safetensors", + "model.layers.39.mlp.experts.9.up_proj.weight": "model-00042-of-00101.safetensors", + "model.layers.39.mlp.experts.9.down_proj.weight": "model-00042-of-00101.safetensors", + "model.layers.39.mlp.experts.10.gate_proj.weight": "model-00042-of-00101.safetensors", + "model.layers.39.mlp.experts.10.up_proj.weight": "model-00042-of-00101.safetensors", + "model.layers.39.mlp.experts.10.down_proj.weight": "model-00042-of-00101.safetensors", + "model.layers.39.mlp.experts.11.gate_proj.weight": "model-00042-of-00101.safetensors", + "model.layers.39.mlp.experts.11.up_proj.weight": "model-00042-of-00101.safetensors", + "model.layers.39.mlp.experts.11.down_proj.weight": "model-00042-of-00101.safetensors", + "model.layers.39.mlp.experts.12.gate_proj.weight": "model-00042-of-00101.safetensors", + "model.layers.39.mlp.experts.12.up_proj.weight": "model-00042-of-00101.safetensors", + "model.layers.39.mlp.experts.12.down_proj.weight": "model-00042-of-00101.safetensors", + "model.layers.39.mlp.experts.13.gate_proj.weight": "model-00042-of-00101.safetensors", + "model.layers.39.mlp.experts.13.up_proj.weight": "model-00042-of-00101.safetensors", + "model.layers.39.mlp.experts.13.down_proj.weight": "model-00042-of-00101.safetensors", + "model.layers.39.mlp.experts.14.gate_proj.weight": "model-00042-of-00101.safetensors", + "model.layers.39.mlp.experts.14.up_proj.weight": "model-00042-of-00101.safetensors", + "model.layers.39.mlp.experts.14.down_proj.weight": "model-00042-of-00101.safetensors", + "model.layers.39.mlp.experts.15.gate_proj.weight": "model-00042-of-00101.safetensors", + "model.layers.39.mlp.experts.15.up_proj.weight": "model-00042-of-00101.safetensors", + "model.layers.39.mlp.experts.15.down_proj.weight": "model-00042-of-00101.safetensors", + "model.layers.39.mlp.experts.16.gate_proj.weight": "model-00042-of-00101.safetensors", + "model.layers.39.mlp.experts.16.up_proj.weight": "model-00042-of-00101.safetensors", + "model.layers.39.mlp.experts.16.down_proj.weight": "model-00042-of-00101.safetensors", + "model.layers.39.mlp.experts.17.gate_proj.weight": "model-00042-of-00101.safetensors", + "model.layers.39.mlp.experts.17.up_proj.weight": "model-00042-of-00101.safetensors", + "model.layers.39.mlp.experts.17.down_proj.weight": "model-00042-of-00101.safetensors", + "model.layers.39.mlp.experts.18.gate_proj.weight": "model-00042-of-00101.safetensors", + "model.layers.39.mlp.experts.18.up_proj.weight": "model-00042-of-00101.safetensors", + "model.layers.39.mlp.experts.18.down_proj.weight": "model-00042-of-00101.safetensors", + "model.layers.39.mlp.experts.19.gate_proj.weight": "model-00042-of-00101.safetensors", + "model.layers.39.mlp.experts.19.up_proj.weight": "model-00042-of-00101.safetensors", + "model.layers.39.mlp.experts.19.down_proj.weight": "model-00042-of-00101.safetensors", + "model.layers.39.mlp.experts.20.gate_proj.weight": "model-00042-of-00101.safetensors", + "model.layers.39.mlp.experts.20.up_proj.weight": "model-00042-of-00101.safetensors", + "model.layers.39.mlp.experts.20.down_proj.weight": "model-00042-of-00101.safetensors", + "model.layers.39.mlp.experts.21.gate_proj.weight": "model-00042-of-00101.safetensors", + "model.layers.39.mlp.experts.21.up_proj.weight": "model-00042-of-00101.safetensors", + "model.layers.39.mlp.experts.21.down_proj.weight": "model-00042-of-00101.safetensors", + "model.layers.39.mlp.experts.22.gate_proj.weight": "model-00042-of-00101.safetensors", + "model.layers.39.mlp.experts.22.up_proj.weight": "model-00042-of-00101.safetensors", + "model.layers.39.mlp.experts.22.down_proj.weight": "model-00042-of-00101.safetensors", + "model.layers.39.mlp.experts.23.gate_proj.weight": "model-00042-of-00101.safetensors", + "model.layers.39.mlp.experts.23.up_proj.weight": "model-00042-of-00101.safetensors", + "model.layers.39.mlp.experts.23.down_proj.weight": "model-00042-of-00101.safetensors", + "model.layers.39.mlp.experts.24.gate_proj.weight": "model-00042-of-00101.safetensors", + "model.layers.39.mlp.experts.24.up_proj.weight": "model-00042-of-00101.safetensors", + "model.layers.39.mlp.experts.24.down_proj.weight": "model-00042-of-00101.safetensors", + "model.layers.39.mlp.experts.25.gate_proj.weight": "model-00042-of-00101.safetensors", + "model.layers.39.mlp.experts.25.up_proj.weight": "model-00042-of-00101.safetensors", + "model.layers.39.mlp.experts.25.down_proj.weight": "model-00042-of-00101.safetensors", + "model.layers.39.mlp.experts.26.gate_proj.weight": "model-00042-of-00101.safetensors", + "model.layers.39.mlp.experts.26.up_proj.weight": "model-00042-of-00101.safetensors", + "model.layers.39.mlp.experts.26.down_proj.weight": "model-00042-of-00101.safetensors", + "model.layers.39.mlp.experts.27.gate_proj.weight": "model-00042-of-00101.safetensors", + "model.layers.39.mlp.experts.27.up_proj.weight": "model-00042-of-00101.safetensors", + "model.layers.39.mlp.experts.27.down_proj.weight": "model-00042-of-00101.safetensors", + "model.layers.39.mlp.experts.28.gate_proj.weight": "model-00042-of-00101.safetensors", + "model.layers.39.mlp.experts.28.up_proj.weight": "model-00042-of-00101.safetensors", + "model.layers.39.mlp.experts.28.down_proj.weight": "model-00042-of-00101.safetensors", + "model.layers.39.mlp.experts.29.gate_proj.weight": "model-00042-of-00101.safetensors", + "model.layers.39.mlp.experts.29.up_proj.weight": "model-00042-of-00101.safetensors", + "model.layers.39.mlp.experts.29.down_proj.weight": "model-00042-of-00101.safetensors", + "model.layers.39.mlp.experts.30.gate_proj.weight": "model-00042-of-00101.safetensors", + "model.layers.39.mlp.experts.30.up_proj.weight": "model-00042-of-00101.safetensors", + "model.layers.39.mlp.experts.30.down_proj.weight": "model-00042-of-00101.safetensors", + "model.layers.39.mlp.experts.31.gate_proj.weight": "model-00042-of-00101.safetensors", + "model.layers.39.mlp.experts.31.up_proj.weight": "model-00042-of-00101.safetensors", + "model.layers.39.mlp.experts.31.down_proj.weight": "model-00042-of-00101.safetensors", + "model.layers.39.mlp.experts.32.gate_proj.weight": "model-00042-of-00101.safetensors", + "model.layers.39.mlp.experts.32.up_proj.weight": "model-00042-of-00101.safetensors", + "model.layers.39.mlp.experts.32.down_proj.weight": "model-00042-of-00101.safetensors", + "model.layers.39.mlp.experts.33.gate_proj.weight": "model-00042-of-00101.safetensors", + "model.layers.39.mlp.experts.33.up_proj.weight": "model-00042-of-00101.safetensors", + "model.layers.39.mlp.experts.33.down_proj.weight": "model-00042-of-00101.safetensors", + "model.layers.39.mlp.experts.34.gate_proj.weight": "model-00042-of-00101.safetensors", + "model.layers.39.mlp.experts.34.up_proj.weight": "model-00042-of-00101.safetensors", + "model.layers.39.mlp.experts.34.down_proj.weight": "model-00042-of-00101.safetensors", + "model.layers.39.mlp.experts.35.gate_proj.weight": "model-00042-of-00101.safetensors", + "model.layers.39.mlp.experts.35.up_proj.weight": "model-00042-of-00101.safetensors", + "model.layers.39.mlp.experts.35.down_proj.weight": "model-00042-of-00101.safetensors", + "model.layers.39.mlp.experts.36.gate_proj.weight": "model-00042-of-00101.safetensors", + "model.layers.39.mlp.experts.36.up_proj.weight": "model-00042-of-00101.safetensors", + "model.layers.39.mlp.experts.36.down_proj.weight": "model-00042-of-00101.safetensors", + "model.layers.39.mlp.experts.37.gate_proj.weight": "model-00042-of-00101.safetensors", + "model.layers.39.mlp.experts.37.up_proj.weight": "model-00042-of-00101.safetensors", + "model.layers.39.mlp.experts.37.down_proj.weight": "model-00042-of-00101.safetensors", + "model.layers.39.mlp.experts.38.gate_proj.weight": "model-00042-of-00101.safetensors", + "model.layers.39.mlp.experts.38.up_proj.weight": "model-00042-of-00101.safetensors", + "model.layers.39.mlp.experts.38.down_proj.weight": "model-00042-of-00101.safetensors", + "model.layers.39.mlp.experts.39.gate_proj.weight": "model-00042-of-00101.safetensors", + "model.layers.39.mlp.experts.39.up_proj.weight": "model-00042-of-00101.safetensors", + "model.layers.39.mlp.experts.39.down_proj.weight": "model-00042-of-00101.safetensors", + "model.layers.39.mlp.experts.40.gate_proj.weight": "model-00042-of-00101.safetensors", + "model.layers.39.mlp.experts.40.up_proj.weight": "model-00042-of-00101.safetensors", + "model.layers.39.mlp.experts.40.down_proj.weight": "model-00042-of-00101.safetensors", + "model.layers.39.mlp.experts.41.gate_proj.weight": "model-00042-of-00101.safetensors", + "model.layers.39.mlp.experts.41.up_proj.weight": "model-00042-of-00101.safetensors", + "model.layers.39.mlp.experts.41.down_proj.weight": "model-00042-of-00101.safetensors", + "model.layers.39.mlp.experts.42.gate_proj.weight": "model-00042-of-00101.safetensors", + "model.layers.39.mlp.experts.42.up_proj.weight": "model-00042-of-00101.safetensors", + "model.layers.39.mlp.experts.42.down_proj.weight": "model-00042-of-00101.safetensors", + "model.layers.39.mlp.experts.43.gate_proj.weight": "model-00042-of-00101.safetensors", + "model.layers.39.mlp.experts.43.up_proj.weight": "model-00042-of-00101.safetensors", + "model.layers.39.mlp.experts.43.down_proj.weight": "model-00042-of-00101.safetensors", + "model.layers.39.mlp.experts.44.gate_proj.weight": "model-00042-of-00101.safetensors", + "model.layers.39.mlp.experts.44.up_proj.weight": "model-00042-of-00101.safetensors", + "model.layers.39.mlp.experts.44.down_proj.weight": "model-00042-of-00101.safetensors", + "model.layers.39.mlp.experts.45.gate_proj.weight": "model-00042-of-00101.safetensors", + "model.layers.39.mlp.experts.45.up_proj.weight": "model-00042-of-00101.safetensors", + "model.layers.39.mlp.experts.45.down_proj.weight": "model-00042-of-00101.safetensors", + "model.layers.39.mlp.experts.46.gate_proj.weight": "model-00042-of-00101.safetensors", + "model.layers.39.mlp.experts.46.up_proj.weight": "model-00042-of-00101.safetensors", + "model.layers.39.mlp.experts.46.down_proj.weight": "model-00042-of-00101.safetensors", + "model.layers.39.mlp.experts.47.gate_proj.weight": "model-00042-of-00101.safetensors", + "model.layers.39.mlp.experts.47.up_proj.weight": "model-00042-of-00101.safetensors", + "model.layers.39.mlp.experts.47.down_proj.weight": "model-00042-of-00101.safetensors", + "model.layers.39.mlp.experts.48.gate_proj.weight": "model-00042-of-00101.safetensors", + "model.layers.39.mlp.experts.48.up_proj.weight": "model-00042-of-00101.safetensors", + "model.layers.39.mlp.experts.48.down_proj.weight": "model-00042-of-00101.safetensors", + "model.layers.39.mlp.experts.49.gate_proj.weight": "model-00042-of-00101.safetensors", + "model.layers.39.mlp.experts.49.up_proj.weight": "model-00042-of-00101.safetensors", + "model.layers.39.mlp.experts.49.down_proj.weight": "model-00042-of-00101.safetensors", + "model.layers.39.mlp.experts.50.gate_proj.weight": "model-00042-of-00101.safetensors", + "model.layers.39.mlp.experts.50.up_proj.weight": "model-00042-of-00101.safetensors", + "model.layers.39.mlp.experts.50.down_proj.weight": "model-00042-of-00101.safetensors", + "model.layers.39.mlp.experts.51.gate_proj.weight": "model-00042-of-00101.safetensors", + "model.layers.39.mlp.experts.51.up_proj.weight": "model-00042-of-00101.safetensors", + "model.layers.39.mlp.experts.51.down_proj.weight": "model-00042-of-00101.safetensors", + "model.layers.39.mlp.experts.52.gate_proj.weight": "model-00042-of-00101.safetensors", + "model.layers.39.mlp.experts.52.up_proj.weight": "model-00042-of-00101.safetensors", + "model.layers.39.mlp.experts.52.down_proj.weight": "model-00042-of-00101.safetensors", + "model.layers.39.mlp.experts.53.gate_proj.weight": "model-00042-of-00101.safetensors", + "model.layers.39.mlp.experts.53.up_proj.weight": "model-00042-of-00101.safetensors", + "model.layers.39.mlp.experts.53.down_proj.weight": "model-00042-of-00101.safetensors", + "model.layers.39.mlp.experts.54.gate_proj.weight": "model-00042-of-00101.safetensors", + "model.layers.39.mlp.experts.54.up_proj.weight": "model-00042-of-00101.safetensors", + "model.layers.39.mlp.experts.54.down_proj.weight": "model-00042-of-00101.safetensors", + "model.layers.39.mlp.experts.55.gate_proj.weight": "model-00042-of-00101.safetensors", + "model.layers.39.mlp.experts.55.up_proj.weight": "model-00042-of-00101.safetensors", + "model.layers.39.mlp.experts.55.down_proj.weight": "model-00042-of-00101.safetensors", + "model.layers.39.mlp.experts.56.gate_proj.weight": "model-00042-of-00101.safetensors", + "model.layers.39.mlp.experts.56.up_proj.weight": "model-00042-of-00101.safetensors", + "model.layers.39.mlp.experts.56.down_proj.weight": "model-00042-of-00101.safetensors", + "model.layers.39.mlp.experts.57.gate_proj.weight": "model-00042-of-00101.safetensors", + "model.layers.39.mlp.experts.57.up_proj.weight": "model-00042-of-00101.safetensors", + "model.layers.39.mlp.experts.57.down_proj.weight": "model-00042-of-00101.safetensors", + "model.layers.39.mlp.experts.58.gate_proj.weight": "model-00042-of-00101.safetensors", + "model.layers.39.mlp.experts.58.up_proj.weight": "model-00042-of-00101.safetensors", + "model.layers.39.mlp.experts.58.down_proj.weight": "model-00042-of-00101.safetensors", + "model.layers.39.mlp.experts.59.gate_proj.weight": "model-00042-of-00101.safetensors", + "model.layers.39.mlp.experts.59.up_proj.weight": "model-00042-of-00101.safetensors", + "model.layers.39.mlp.experts.59.down_proj.weight": "model-00042-of-00101.safetensors", + "model.layers.39.mlp.experts.60.gate_proj.weight": "model-00042-of-00101.safetensors", + "model.layers.39.mlp.experts.60.up_proj.weight": "model-00042-of-00101.safetensors", + "model.layers.39.mlp.experts.60.down_proj.weight": "model-00042-of-00101.safetensors", + "model.layers.39.mlp.experts.61.gate_proj.weight": "model-00042-of-00101.safetensors", + "model.layers.39.mlp.experts.61.up_proj.weight": "model-00042-of-00101.safetensors", + "model.layers.39.mlp.experts.61.down_proj.weight": "model-00042-of-00101.safetensors", + "model.layers.39.mlp.experts.62.gate_proj.weight": "model-00042-of-00101.safetensors", + "model.layers.39.mlp.experts.62.up_proj.weight": "model-00042-of-00101.safetensors", + "model.layers.39.mlp.experts.62.down_proj.weight": "model-00042-of-00101.safetensors", + "model.layers.39.mlp.experts.63.gate_proj.weight": "model-00042-of-00101.safetensors", + "model.layers.39.mlp.experts.63.up_proj.weight": "model-00042-of-00101.safetensors", + "model.layers.39.mlp.experts.63.down_proj.weight": "model-00042-of-00101.safetensors", + "model.layers.39.mlp.experts.64.gate_proj.weight": "model-00042-of-00101.safetensors", + "model.layers.39.mlp.experts.64.up_proj.weight": "model-00042-of-00101.safetensors", + "model.layers.39.mlp.experts.64.down_proj.weight": "model-00042-of-00101.safetensors", + "model.layers.39.mlp.experts.65.gate_proj.weight": "model-00042-of-00101.safetensors", + "model.layers.39.mlp.experts.65.up_proj.weight": "model-00042-of-00101.safetensors", + "model.layers.39.mlp.experts.65.down_proj.weight": "model-00042-of-00101.safetensors", + "model.layers.39.mlp.experts.66.gate_proj.weight": "model-00042-of-00101.safetensors", + "model.layers.39.mlp.experts.66.up_proj.weight": "model-00042-of-00101.safetensors", + "model.layers.39.mlp.experts.66.down_proj.weight": "model-00042-of-00101.safetensors", + "model.layers.39.mlp.experts.67.gate_proj.weight": "model-00042-of-00101.safetensors", + "model.layers.39.mlp.experts.67.up_proj.weight": "model-00042-of-00101.safetensors", + "model.layers.39.mlp.experts.67.down_proj.weight": "model-00042-of-00101.safetensors", + "model.layers.39.mlp.experts.68.gate_proj.weight": "model-00042-of-00101.safetensors", + "model.layers.39.mlp.experts.68.up_proj.weight": "model-00042-of-00101.safetensors", + "model.layers.39.mlp.experts.68.down_proj.weight": "model-00042-of-00101.safetensors", + "model.layers.39.mlp.experts.69.gate_proj.weight": "model-00042-of-00101.safetensors", + "model.layers.39.mlp.experts.69.up_proj.weight": "model-00042-of-00101.safetensors", + "model.layers.39.mlp.experts.69.down_proj.weight": "model-00042-of-00101.safetensors", + "model.layers.39.mlp.experts.70.gate_proj.weight": "model-00042-of-00101.safetensors", + "model.layers.39.mlp.experts.70.up_proj.weight": "model-00042-of-00101.safetensors", + "model.layers.39.mlp.experts.70.down_proj.weight": "model-00042-of-00101.safetensors", + "model.layers.39.mlp.experts.71.gate_proj.weight": "model-00042-of-00101.safetensors", + "model.layers.39.mlp.experts.71.up_proj.weight": "model-00042-of-00101.safetensors", + "model.layers.39.mlp.experts.71.down_proj.weight": "model-00042-of-00101.safetensors", + "model.layers.39.mlp.experts.72.gate_proj.weight": "model-00042-of-00101.safetensors", + "model.layers.39.mlp.experts.72.up_proj.weight": "model-00042-of-00101.safetensors", + "model.layers.39.mlp.experts.72.down_proj.weight": "model-00042-of-00101.safetensors", + "model.layers.39.mlp.experts.73.gate_proj.weight": "model-00042-of-00101.safetensors", + "model.layers.39.mlp.experts.73.up_proj.weight": "model-00042-of-00101.safetensors", + "model.layers.39.mlp.experts.73.down_proj.weight": "model-00042-of-00101.safetensors", + "model.layers.39.mlp.experts.74.gate_proj.weight": "model-00042-of-00101.safetensors", + "model.layers.39.mlp.experts.74.up_proj.weight": "model-00042-of-00101.safetensors", + "model.layers.39.mlp.experts.74.down_proj.weight": "model-00042-of-00101.safetensors", + "model.layers.39.mlp.experts.75.gate_proj.weight": "model-00042-of-00101.safetensors", + "model.layers.39.mlp.experts.75.up_proj.weight": "model-00042-of-00101.safetensors", + "model.layers.39.mlp.experts.75.down_proj.weight": "model-00042-of-00101.safetensors", + "model.layers.39.mlp.experts.76.gate_proj.weight": "model-00042-of-00101.safetensors", + "model.layers.39.mlp.experts.76.up_proj.weight": "model-00042-of-00101.safetensors", + "model.layers.39.mlp.experts.76.down_proj.weight": "model-00042-of-00101.safetensors", + "model.layers.39.mlp.experts.77.gate_proj.weight": "model-00042-of-00101.safetensors", + "model.layers.39.mlp.experts.77.up_proj.weight": "model-00042-of-00101.safetensors", + "model.layers.39.mlp.experts.77.down_proj.weight": "model-00042-of-00101.safetensors", + "model.layers.39.mlp.experts.78.gate_proj.weight": "model-00042-of-00101.safetensors", + "model.layers.39.mlp.experts.78.up_proj.weight": "model-00042-of-00101.safetensors", + "model.layers.39.mlp.experts.78.down_proj.weight": "model-00042-of-00101.safetensors", + "model.layers.39.mlp.experts.79.gate_proj.weight": "model-00042-of-00101.safetensors", + "model.layers.39.mlp.experts.79.up_proj.weight": "model-00042-of-00101.safetensors", + "model.layers.39.mlp.experts.79.down_proj.weight": "model-00042-of-00101.safetensors", + "model.layers.39.mlp.experts.80.gate_proj.weight": "model-00042-of-00101.safetensors", + "model.layers.39.mlp.experts.80.up_proj.weight": "model-00042-of-00101.safetensors", + "model.layers.39.mlp.experts.80.down_proj.weight": "model-00042-of-00101.safetensors", + "model.layers.39.mlp.experts.81.gate_proj.weight": "model-00042-of-00101.safetensors", + "model.layers.39.mlp.experts.81.up_proj.weight": "model-00042-of-00101.safetensors", + "model.layers.39.mlp.experts.81.down_proj.weight": "model-00042-of-00101.safetensors", + "model.layers.39.mlp.experts.82.gate_proj.weight": "model-00042-of-00101.safetensors", + "model.layers.39.mlp.experts.82.up_proj.weight": "model-00042-of-00101.safetensors", + "model.layers.39.mlp.experts.82.down_proj.weight": "model-00042-of-00101.safetensors", + "model.layers.39.mlp.experts.83.gate_proj.weight": "model-00042-of-00101.safetensors", + "model.layers.39.mlp.experts.83.up_proj.weight": "model-00042-of-00101.safetensors", + "model.layers.39.mlp.experts.83.down_proj.weight": "model-00042-of-00101.safetensors", + "model.layers.39.mlp.experts.84.gate_proj.weight": "model-00042-of-00101.safetensors", + "model.layers.39.mlp.experts.84.up_proj.weight": "model-00042-of-00101.safetensors", + "model.layers.39.mlp.experts.84.down_proj.weight": "model-00042-of-00101.safetensors", + "model.layers.39.mlp.experts.85.gate_proj.weight": "model-00042-of-00101.safetensors", + "model.layers.39.mlp.experts.85.up_proj.weight": "model-00042-of-00101.safetensors", + "model.layers.39.mlp.experts.85.down_proj.weight": "model-00042-of-00101.safetensors", + "model.layers.39.mlp.experts.86.gate_proj.weight": "model-00042-of-00101.safetensors", + "model.layers.39.mlp.experts.86.up_proj.weight": "model-00042-of-00101.safetensors", + "model.layers.39.mlp.experts.86.down_proj.weight": "model-00042-of-00101.safetensors", + "model.layers.39.mlp.experts.87.gate_proj.weight": "model-00042-of-00101.safetensors", + "model.layers.39.mlp.experts.87.up_proj.weight": "model-00042-of-00101.safetensors", + "model.layers.39.mlp.experts.87.down_proj.weight": "model-00042-of-00101.safetensors", + "model.layers.39.mlp.experts.88.gate_proj.weight": "model-00042-of-00101.safetensors", + "model.layers.39.mlp.experts.88.up_proj.weight": "model-00042-of-00101.safetensors", + "model.layers.39.mlp.experts.88.down_proj.weight": "model-00042-of-00101.safetensors", + "model.layers.39.mlp.experts.89.gate_proj.weight": "model-00042-of-00101.safetensors", + "model.layers.39.mlp.experts.89.up_proj.weight": "model-00042-of-00101.safetensors", + "model.layers.39.mlp.experts.89.down_proj.weight": "model-00042-of-00101.safetensors", + "model.layers.39.mlp.experts.90.gate_proj.weight": "model-00042-of-00101.safetensors", + "model.layers.39.mlp.experts.90.up_proj.weight": "model-00042-of-00101.safetensors", + "model.layers.39.mlp.experts.90.down_proj.weight": "model-00042-of-00101.safetensors", + "model.layers.39.mlp.experts.91.gate_proj.weight": "model-00042-of-00101.safetensors", + "model.layers.39.mlp.experts.91.up_proj.weight": "model-00042-of-00101.safetensors", + "model.layers.39.mlp.experts.91.down_proj.weight": "model-00042-of-00101.safetensors", + "model.layers.39.mlp.experts.92.gate_proj.weight": "model-00042-of-00101.safetensors", + "model.layers.39.mlp.experts.92.up_proj.weight": "model-00042-of-00101.safetensors", + "model.layers.39.mlp.experts.92.down_proj.weight": "model-00042-of-00101.safetensors", + "model.layers.39.mlp.experts.93.gate_proj.weight": "model-00042-of-00101.safetensors", + "model.layers.39.mlp.experts.93.up_proj.weight": "model-00042-of-00101.safetensors", + "model.layers.39.mlp.experts.93.down_proj.weight": "model-00042-of-00101.safetensors", + "model.layers.39.mlp.experts.94.gate_proj.weight": "model-00042-of-00101.safetensors", + "model.layers.39.mlp.experts.94.up_proj.weight": "model-00042-of-00101.safetensors", + "model.layers.39.mlp.experts.94.down_proj.weight": "model-00042-of-00101.safetensors", + "model.layers.39.mlp.experts.95.gate_proj.weight": "model-00042-of-00101.safetensors", + "model.layers.39.mlp.experts.95.up_proj.weight": "model-00042-of-00101.safetensors", + "model.layers.39.mlp.experts.95.down_proj.weight": "model-00042-of-00101.safetensors", + "model.layers.39.mlp.experts.96.gate_proj.weight": "model-00042-of-00101.safetensors", + "model.layers.39.mlp.experts.96.up_proj.weight": "model-00042-of-00101.safetensors", + "model.layers.39.mlp.experts.96.down_proj.weight": "model-00042-of-00101.safetensors", + "model.layers.39.mlp.experts.97.gate_proj.weight": "model-00042-of-00101.safetensors", + "model.layers.39.mlp.experts.97.up_proj.weight": "model-00042-of-00101.safetensors", + "model.layers.39.mlp.experts.97.down_proj.weight": "model-00042-of-00101.safetensors", + "model.layers.39.mlp.experts.98.gate_proj.weight": "model-00042-of-00101.safetensors", + "model.layers.39.mlp.experts.98.up_proj.weight": "model-00042-of-00101.safetensors", + "model.layers.39.mlp.experts.98.down_proj.weight": "model-00042-of-00101.safetensors", + "model.layers.39.mlp.experts.99.gate_proj.weight": "model-00042-of-00101.safetensors", + "model.layers.39.mlp.experts.99.up_proj.weight": "model-00042-of-00101.safetensors", + "model.layers.39.mlp.experts.99.down_proj.weight": "model-00042-of-00101.safetensors", + "model.layers.39.mlp.experts.100.gate_proj.weight": "model-00042-of-00101.safetensors", + "model.layers.39.mlp.experts.100.up_proj.weight": "model-00042-of-00101.safetensors", + "model.layers.39.mlp.experts.100.down_proj.weight": "model-00042-of-00101.safetensors", + "model.layers.39.mlp.experts.101.gate_proj.weight": "model-00042-of-00101.safetensors", + "model.layers.39.mlp.experts.101.up_proj.weight": "model-00042-of-00101.safetensors", + "model.layers.39.mlp.experts.101.down_proj.weight": "model-00042-of-00101.safetensors", + "model.layers.39.mlp.experts.102.gate_proj.weight": "model-00042-of-00101.safetensors", + "model.layers.39.mlp.experts.102.up_proj.weight": "model-00042-of-00101.safetensors", + "model.layers.39.mlp.experts.102.down_proj.weight": "model-00042-of-00101.safetensors", + "model.layers.39.mlp.experts.103.gate_proj.weight": "model-00042-of-00101.safetensors", + "model.layers.39.mlp.experts.103.up_proj.weight": "model-00042-of-00101.safetensors", + "model.layers.39.mlp.experts.103.down_proj.weight": "model-00042-of-00101.safetensors", + "model.layers.39.mlp.experts.104.gate_proj.weight": "model-00042-of-00101.safetensors", + "model.layers.39.mlp.experts.104.up_proj.weight": "model-00042-of-00101.safetensors", + "model.layers.39.mlp.experts.104.down_proj.weight": "model-00042-of-00101.safetensors", + "model.layers.39.mlp.experts.105.gate_proj.weight": "model-00042-of-00101.safetensors", + "model.layers.39.mlp.experts.105.up_proj.weight": "model-00042-of-00101.safetensors", + "model.layers.39.mlp.experts.105.down_proj.weight": "model-00042-of-00101.safetensors", + "model.layers.39.mlp.experts.106.gate_proj.weight": "model-00042-of-00101.safetensors", + "model.layers.39.mlp.experts.106.up_proj.weight": "model-00042-of-00101.safetensors", + "model.layers.39.mlp.experts.106.down_proj.weight": "model-00042-of-00101.safetensors", + "model.layers.39.mlp.experts.107.gate_proj.weight": "model-00042-of-00101.safetensors", + "model.layers.39.mlp.experts.107.up_proj.weight": "model-00042-of-00101.safetensors", + "model.layers.39.mlp.experts.107.down_proj.weight": "model-00042-of-00101.safetensors", + "model.layers.39.mlp.experts.108.gate_proj.weight": "model-00042-of-00101.safetensors", + "model.layers.39.mlp.experts.108.up_proj.weight": "model-00042-of-00101.safetensors", + "model.layers.39.mlp.experts.108.down_proj.weight": "model-00042-of-00101.safetensors", + "model.layers.39.mlp.experts.109.gate_proj.weight": "model-00042-of-00101.safetensors", + "model.layers.39.mlp.experts.109.up_proj.weight": "model-00042-of-00101.safetensors", + "model.layers.39.mlp.experts.109.down_proj.weight": "model-00042-of-00101.safetensors", + "model.layers.39.mlp.experts.110.gate_proj.weight": "model-00042-of-00101.safetensors", + "model.layers.39.mlp.experts.110.up_proj.weight": "model-00042-of-00101.safetensors", + "model.layers.39.mlp.experts.110.down_proj.weight": "model-00042-of-00101.safetensors", + "model.layers.39.mlp.experts.111.gate_proj.weight": "model-00042-of-00101.safetensors", + "model.layers.39.mlp.experts.111.up_proj.weight": "model-00042-of-00101.safetensors", + "model.layers.39.mlp.experts.111.down_proj.weight": "model-00042-of-00101.safetensors", + "model.layers.39.mlp.experts.112.gate_proj.weight": "model-00042-of-00101.safetensors", + "model.layers.39.mlp.experts.112.up_proj.weight": "model-00042-of-00101.safetensors", + "model.layers.39.mlp.experts.112.down_proj.weight": "model-00042-of-00101.safetensors", + "model.layers.39.mlp.experts.113.gate_proj.weight": "model-00042-of-00101.safetensors", + "model.layers.39.mlp.experts.113.up_proj.weight": "model-00042-of-00101.safetensors", + "model.layers.39.mlp.experts.113.down_proj.weight": "model-00042-of-00101.safetensors", + "model.layers.39.mlp.experts.114.gate_proj.weight": "model-00042-of-00101.safetensors", + "model.layers.39.mlp.experts.114.up_proj.weight": "model-00042-of-00101.safetensors", + "model.layers.39.mlp.experts.114.down_proj.weight": "model-00042-of-00101.safetensors", + "model.layers.39.mlp.experts.115.gate_proj.weight": "model-00042-of-00101.safetensors", + "model.layers.39.mlp.experts.115.up_proj.weight": "model-00042-of-00101.safetensors", + "model.layers.39.mlp.experts.115.down_proj.weight": "model-00042-of-00101.safetensors", + "model.layers.39.mlp.experts.116.gate_proj.weight": "model-00042-of-00101.safetensors", + "model.layers.39.mlp.experts.116.up_proj.weight": "model-00042-of-00101.safetensors", + "model.layers.39.mlp.experts.116.down_proj.weight": "model-00042-of-00101.safetensors", + "model.layers.39.mlp.experts.117.gate_proj.weight": "model-00042-of-00101.safetensors", + "model.layers.39.mlp.experts.117.up_proj.weight": "model-00042-of-00101.safetensors", + "model.layers.39.mlp.experts.117.down_proj.weight": "model-00042-of-00101.safetensors", + "model.layers.39.mlp.experts.118.gate_proj.weight": "model-00042-of-00101.safetensors", + "model.layers.39.mlp.experts.118.up_proj.weight": "model-00042-of-00101.safetensors", + "model.layers.39.mlp.experts.118.down_proj.weight": "model-00042-of-00101.safetensors", + "model.layers.39.mlp.experts.119.gate_proj.weight": "model-00042-of-00101.safetensors", + "model.layers.39.mlp.experts.119.up_proj.weight": "model-00042-of-00101.safetensors", + "model.layers.39.mlp.experts.119.down_proj.weight": "model-00042-of-00101.safetensors", + "model.layers.39.mlp.gate.weight": "model-00042-of-00101.safetensors", + "model.layers.39.mlp.gate.e_score_correction_bias": "model-00042-of-00101.safetensors", + "model.layers.39.mlp.shared_experts.gate_proj.weight": "model-00042-of-00101.safetensors", + "model.layers.39.mlp.shared_experts.up_proj.weight": "model-00042-of-00101.safetensors", + "model.layers.39.mlp.shared_experts.down_proj.weight": "model-00042-of-00101.safetensors", + "model.layers.39.input_layernorm.weight": "model-00042-of-00101.safetensors", + "model.layers.39.post_attention_layernorm.weight": "model-00042-of-00101.safetensors", + "model.layers.40.self_attn.q_proj.weight": "model-00043-of-00101.safetensors", + "model.layers.40.self_attn.q_proj.bias": "model-00043-of-00101.safetensors", + "model.layers.40.self_attn.k_proj.weight": "model-00043-of-00101.safetensors", + "model.layers.40.self_attn.k_proj.bias": "model-00043-of-00101.safetensors", + "model.layers.40.self_attn.v_proj.weight": "model-00043-of-00101.safetensors", + "model.layers.40.self_attn.v_proj.bias": "model-00043-of-00101.safetensors", + "model.layers.40.self_attn.o_proj.weight": "model-00043-of-00101.safetensors", + "model.layers.40.self_attn.q_norm.weight": "model-00043-of-00101.safetensors", + "model.layers.40.self_attn.k_norm.weight": "model-00043-of-00101.safetensors", + "model.layers.40.mlp.experts.0.gate_proj.weight": "model-00043-of-00101.safetensors", + "model.layers.40.mlp.experts.0.up_proj.weight": "model-00043-of-00101.safetensors", + "model.layers.40.mlp.experts.0.down_proj.weight": "model-00043-of-00101.safetensors", + "model.layers.40.mlp.experts.1.gate_proj.weight": "model-00043-of-00101.safetensors", + "model.layers.40.mlp.experts.1.up_proj.weight": "model-00043-of-00101.safetensors", + "model.layers.40.mlp.experts.1.down_proj.weight": "model-00043-of-00101.safetensors", + "model.layers.40.mlp.experts.2.gate_proj.weight": "model-00043-of-00101.safetensors", + "model.layers.40.mlp.experts.2.up_proj.weight": "model-00043-of-00101.safetensors", + "model.layers.40.mlp.experts.2.down_proj.weight": "model-00043-of-00101.safetensors", + "model.layers.40.mlp.experts.3.gate_proj.weight": "model-00043-of-00101.safetensors", + "model.layers.40.mlp.experts.3.up_proj.weight": "model-00043-of-00101.safetensors", + "model.layers.40.mlp.experts.3.down_proj.weight": "model-00043-of-00101.safetensors", + "model.layers.40.mlp.experts.4.gate_proj.weight": "model-00043-of-00101.safetensors", + "model.layers.40.mlp.experts.4.up_proj.weight": "model-00043-of-00101.safetensors", + "model.layers.40.mlp.experts.4.down_proj.weight": "model-00043-of-00101.safetensors", + "model.layers.40.mlp.experts.5.gate_proj.weight": "model-00043-of-00101.safetensors", + "model.layers.40.mlp.experts.5.up_proj.weight": "model-00043-of-00101.safetensors", + "model.layers.40.mlp.experts.5.down_proj.weight": "model-00043-of-00101.safetensors", + "model.layers.40.mlp.experts.6.gate_proj.weight": "model-00043-of-00101.safetensors", + "model.layers.40.mlp.experts.6.up_proj.weight": "model-00043-of-00101.safetensors", + "model.layers.40.mlp.experts.6.down_proj.weight": "model-00043-of-00101.safetensors", + "model.layers.40.mlp.experts.7.gate_proj.weight": "model-00043-of-00101.safetensors", + "model.layers.40.mlp.experts.7.up_proj.weight": "model-00043-of-00101.safetensors", + "model.layers.40.mlp.experts.7.down_proj.weight": "model-00043-of-00101.safetensors", + "model.layers.40.mlp.experts.8.gate_proj.weight": "model-00043-of-00101.safetensors", + "model.layers.40.mlp.experts.8.up_proj.weight": "model-00043-of-00101.safetensors", + "model.layers.40.mlp.experts.8.down_proj.weight": "model-00043-of-00101.safetensors", + "model.layers.40.mlp.experts.9.gate_proj.weight": "model-00043-of-00101.safetensors", + "model.layers.40.mlp.experts.9.up_proj.weight": "model-00043-of-00101.safetensors", + "model.layers.40.mlp.experts.9.down_proj.weight": "model-00043-of-00101.safetensors", + "model.layers.40.mlp.experts.10.gate_proj.weight": "model-00043-of-00101.safetensors", + "model.layers.40.mlp.experts.10.up_proj.weight": "model-00043-of-00101.safetensors", + "model.layers.40.mlp.experts.10.down_proj.weight": "model-00043-of-00101.safetensors", + "model.layers.40.mlp.experts.11.gate_proj.weight": "model-00043-of-00101.safetensors", + "model.layers.40.mlp.experts.11.up_proj.weight": "model-00043-of-00101.safetensors", + "model.layers.40.mlp.experts.11.down_proj.weight": "model-00043-of-00101.safetensors", + "model.layers.40.mlp.experts.12.gate_proj.weight": "model-00043-of-00101.safetensors", + "model.layers.40.mlp.experts.12.up_proj.weight": "model-00043-of-00101.safetensors", + "model.layers.40.mlp.experts.12.down_proj.weight": "model-00043-of-00101.safetensors", + "model.layers.40.mlp.experts.13.gate_proj.weight": "model-00043-of-00101.safetensors", + "model.layers.40.mlp.experts.13.up_proj.weight": "model-00043-of-00101.safetensors", + "model.layers.40.mlp.experts.13.down_proj.weight": "model-00043-of-00101.safetensors", + "model.layers.40.mlp.experts.14.gate_proj.weight": "model-00043-of-00101.safetensors", + "model.layers.40.mlp.experts.14.up_proj.weight": "model-00043-of-00101.safetensors", + "model.layers.40.mlp.experts.14.down_proj.weight": "model-00043-of-00101.safetensors", + "model.layers.40.mlp.experts.15.gate_proj.weight": "model-00043-of-00101.safetensors", + "model.layers.40.mlp.experts.15.up_proj.weight": "model-00043-of-00101.safetensors", + "model.layers.40.mlp.experts.15.down_proj.weight": "model-00043-of-00101.safetensors", + "model.layers.40.mlp.experts.16.gate_proj.weight": "model-00043-of-00101.safetensors", + "model.layers.40.mlp.experts.16.up_proj.weight": "model-00043-of-00101.safetensors", + "model.layers.40.mlp.experts.16.down_proj.weight": "model-00043-of-00101.safetensors", + "model.layers.40.mlp.experts.17.gate_proj.weight": "model-00043-of-00101.safetensors", + "model.layers.40.mlp.experts.17.up_proj.weight": "model-00043-of-00101.safetensors", + "model.layers.40.mlp.experts.17.down_proj.weight": "model-00043-of-00101.safetensors", + "model.layers.40.mlp.experts.18.gate_proj.weight": "model-00043-of-00101.safetensors", + "model.layers.40.mlp.experts.18.up_proj.weight": "model-00043-of-00101.safetensors", + "model.layers.40.mlp.experts.18.down_proj.weight": "model-00043-of-00101.safetensors", + "model.layers.40.mlp.experts.19.gate_proj.weight": "model-00043-of-00101.safetensors", + "model.layers.40.mlp.experts.19.up_proj.weight": "model-00043-of-00101.safetensors", + "model.layers.40.mlp.experts.19.down_proj.weight": "model-00043-of-00101.safetensors", + "model.layers.40.mlp.experts.20.gate_proj.weight": "model-00043-of-00101.safetensors", + "model.layers.40.mlp.experts.20.up_proj.weight": "model-00043-of-00101.safetensors", + "model.layers.40.mlp.experts.20.down_proj.weight": "model-00043-of-00101.safetensors", + "model.layers.40.mlp.experts.21.gate_proj.weight": "model-00043-of-00101.safetensors", + "model.layers.40.mlp.experts.21.up_proj.weight": "model-00043-of-00101.safetensors", + "model.layers.40.mlp.experts.21.down_proj.weight": "model-00043-of-00101.safetensors", + "model.layers.40.mlp.experts.22.gate_proj.weight": "model-00043-of-00101.safetensors", + "model.layers.40.mlp.experts.22.up_proj.weight": "model-00043-of-00101.safetensors", + "model.layers.40.mlp.experts.22.down_proj.weight": "model-00043-of-00101.safetensors", + "model.layers.40.mlp.experts.23.gate_proj.weight": "model-00043-of-00101.safetensors", + "model.layers.40.mlp.experts.23.up_proj.weight": "model-00043-of-00101.safetensors", + "model.layers.40.mlp.experts.23.down_proj.weight": "model-00043-of-00101.safetensors", + "model.layers.40.mlp.experts.24.gate_proj.weight": "model-00043-of-00101.safetensors", + "model.layers.40.mlp.experts.24.up_proj.weight": "model-00043-of-00101.safetensors", + "model.layers.40.mlp.experts.24.down_proj.weight": "model-00043-of-00101.safetensors", + "model.layers.40.mlp.experts.25.gate_proj.weight": "model-00043-of-00101.safetensors", + "model.layers.40.mlp.experts.25.up_proj.weight": "model-00043-of-00101.safetensors", + "model.layers.40.mlp.experts.25.down_proj.weight": "model-00043-of-00101.safetensors", + "model.layers.40.mlp.experts.26.gate_proj.weight": "model-00043-of-00101.safetensors", + "model.layers.40.mlp.experts.26.up_proj.weight": "model-00043-of-00101.safetensors", + "model.layers.40.mlp.experts.26.down_proj.weight": "model-00043-of-00101.safetensors", + "model.layers.40.mlp.experts.27.gate_proj.weight": "model-00043-of-00101.safetensors", + "model.layers.40.mlp.experts.27.up_proj.weight": "model-00043-of-00101.safetensors", + "model.layers.40.mlp.experts.27.down_proj.weight": "model-00043-of-00101.safetensors", + "model.layers.40.mlp.experts.28.gate_proj.weight": "model-00043-of-00101.safetensors", + "model.layers.40.mlp.experts.28.up_proj.weight": "model-00043-of-00101.safetensors", + "model.layers.40.mlp.experts.28.down_proj.weight": "model-00043-of-00101.safetensors", + "model.layers.40.mlp.experts.29.gate_proj.weight": "model-00043-of-00101.safetensors", + "model.layers.40.mlp.experts.29.up_proj.weight": "model-00043-of-00101.safetensors", + "model.layers.40.mlp.experts.29.down_proj.weight": "model-00043-of-00101.safetensors", + "model.layers.40.mlp.experts.30.gate_proj.weight": "model-00043-of-00101.safetensors", + "model.layers.40.mlp.experts.30.up_proj.weight": "model-00043-of-00101.safetensors", + "model.layers.40.mlp.experts.30.down_proj.weight": "model-00043-of-00101.safetensors", + "model.layers.40.mlp.experts.31.gate_proj.weight": "model-00043-of-00101.safetensors", + "model.layers.40.mlp.experts.31.up_proj.weight": "model-00043-of-00101.safetensors", + "model.layers.40.mlp.experts.31.down_proj.weight": "model-00043-of-00101.safetensors", + "model.layers.40.mlp.experts.32.gate_proj.weight": "model-00043-of-00101.safetensors", + "model.layers.40.mlp.experts.32.up_proj.weight": "model-00043-of-00101.safetensors", + "model.layers.40.mlp.experts.32.down_proj.weight": "model-00043-of-00101.safetensors", + "model.layers.40.mlp.experts.33.gate_proj.weight": "model-00043-of-00101.safetensors", + "model.layers.40.mlp.experts.33.up_proj.weight": "model-00043-of-00101.safetensors", + "model.layers.40.mlp.experts.33.down_proj.weight": "model-00043-of-00101.safetensors", + "model.layers.40.mlp.experts.34.gate_proj.weight": "model-00043-of-00101.safetensors", + "model.layers.40.mlp.experts.34.up_proj.weight": "model-00043-of-00101.safetensors", + "model.layers.40.mlp.experts.34.down_proj.weight": "model-00043-of-00101.safetensors", + "model.layers.40.mlp.experts.35.gate_proj.weight": "model-00043-of-00101.safetensors", + "model.layers.40.mlp.experts.35.up_proj.weight": "model-00043-of-00101.safetensors", + "model.layers.40.mlp.experts.35.down_proj.weight": "model-00043-of-00101.safetensors", + "model.layers.40.mlp.experts.36.gate_proj.weight": "model-00043-of-00101.safetensors", + "model.layers.40.mlp.experts.36.up_proj.weight": "model-00043-of-00101.safetensors", + "model.layers.40.mlp.experts.36.down_proj.weight": "model-00043-of-00101.safetensors", + "model.layers.40.mlp.experts.37.gate_proj.weight": "model-00043-of-00101.safetensors", + "model.layers.40.mlp.experts.37.up_proj.weight": "model-00043-of-00101.safetensors", + "model.layers.40.mlp.experts.37.down_proj.weight": "model-00043-of-00101.safetensors", + "model.layers.40.mlp.experts.38.gate_proj.weight": "model-00043-of-00101.safetensors", + "model.layers.40.mlp.experts.38.up_proj.weight": "model-00043-of-00101.safetensors", + "model.layers.40.mlp.experts.38.down_proj.weight": "model-00043-of-00101.safetensors", + "model.layers.40.mlp.experts.39.gate_proj.weight": "model-00043-of-00101.safetensors", + "model.layers.40.mlp.experts.39.up_proj.weight": "model-00043-of-00101.safetensors", + "model.layers.40.mlp.experts.39.down_proj.weight": "model-00043-of-00101.safetensors", + "model.layers.40.mlp.experts.40.gate_proj.weight": "model-00043-of-00101.safetensors", + "model.layers.40.mlp.experts.40.up_proj.weight": "model-00043-of-00101.safetensors", + "model.layers.40.mlp.experts.40.down_proj.weight": "model-00043-of-00101.safetensors", + "model.layers.40.mlp.experts.41.gate_proj.weight": "model-00043-of-00101.safetensors", + "model.layers.40.mlp.experts.41.up_proj.weight": "model-00043-of-00101.safetensors", + "model.layers.40.mlp.experts.41.down_proj.weight": "model-00043-of-00101.safetensors", + "model.layers.40.mlp.experts.42.gate_proj.weight": "model-00043-of-00101.safetensors", + "model.layers.40.mlp.experts.42.up_proj.weight": "model-00043-of-00101.safetensors", + "model.layers.40.mlp.experts.42.down_proj.weight": "model-00043-of-00101.safetensors", + "model.layers.40.mlp.experts.43.gate_proj.weight": "model-00043-of-00101.safetensors", + "model.layers.40.mlp.experts.43.up_proj.weight": "model-00043-of-00101.safetensors", + "model.layers.40.mlp.experts.43.down_proj.weight": "model-00043-of-00101.safetensors", + "model.layers.40.mlp.experts.44.gate_proj.weight": "model-00043-of-00101.safetensors", + "model.layers.40.mlp.experts.44.up_proj.weight": "model-00043-of-00101.safetensors", + "model.layers.40.mlp.experts.44.down_proj.weight": "model-00043-of-00101.safetensors", + "model.layers.40.mlp.experts.45.gate_proj.weight": "model-00043-of-00101.safetensors", + "model.layers.40.mlp.experts.45.up_proj.weight": "model-00043-of-00101.safetensors", + "model.layers.40.mlp.experts.45.down_proj.weight": "model-00043-of-00101.safetensors", + "model.layers.40.mlp.experts.46.gate_proj.weight": "model-00043-of-00101.safetensors", + "model.layers.40.mlp.experts.46.up_proj.weight": "model-00043-of-00101.safetensors", + "model.layers.40.mlp.experts.46.down_proj.weight": "model-00043-of-00101.safetensors", + "model.layers.40.mlp.experts.47.gate_proj.weight": "model-00043-of-00101.safetensors", + "model.layers.40.mlp.experts.47.up_proj.weight": "model-00043-of-00101.safetensors", + "model.layers.40.mlp.experts.47.down_proj.weight": "model-00043-of-00101.safetensors", + "model.layers.40.mlp.experts.48.gate_proj.weight": "model-00043-of-00101.safetensors", + "model.layers.40.mlp.experts.48.up_proj.weight": "model-00043-of-00101.safetensors", + "model.layers.40.mlp.experts.48.down_proj.weight": "model-00043-of-00101.safetensors", + "model.layers.40.mlp.experts.49.gate_proj.weight": "model-00043-of-00101.safetensors", + "model.layers.40.mlp.experts.49.up_proj.weight": "model-00043-of-00101.safetensors", + "model.layers.40.mlp.experts.49.down_proj.weight": "model-00043-of-00101.safetensors", + "model.layers.40.mlp.experts.50.gate_proj.weight": "model-00043-of-00101.safetensors", + "model.layers.40.mlp.experts.50.up_proj.weight": "model-00043-of-00101.safetensors", + "model.layers.40.mlp.experts.50.down_proj.weight": "model-00043-of-00101.safetensors", + "model.layers.40.mlp.experts.51.gate_proj.weight": "model-00043-of-00101.safetensors", + "model.layers.40.mlp.experts.51.up_proj.weight": "model-00043-of-00101.safetensors", + "model.layers.40.mlp.experts.51.down_proj.weight": "model-00043-of-00101.safetensors", + "model.layers.40.mlp.experts.52.gate_proj.weight": "model-00043-of-00101.safetensors", + "model.layers.40.mlp.experts.52.up_proj.weight": "model-00043-of-00101.safetensors", + "model.layers.40.mlp.experts.52.down_proj.weight": "model-00043-of-00101.safetensors", + "model.layers.40.mlp.experts.53.gate_proj.weight": "model-00043-of-00101.safetensors", + "model.layers.40.mlp.experts.53.up_proj.weight": "model-00043-of-00101.safetensors", + "model.layers.40.mlp.experts.53.down_proj.weight": "model-00043-of-00101.safetensors", + "model.layers.40.mlp.experts.54.gate_proj.weight": "model-00043-of-00101.safetensors", + "model.layers.40.mlp.experts.54.up_proj.weight": "model-00043-of-00101.safetensors", + "model.layers.40.mlp.experts.54.down_proj.weight": "model-00043-of-00101.safetensors", + "model.layers.40.mlp.experts.55.gate_proj.weight": "model-00043-of-00101.safetensors", + "model.layers.40.mlp.experts.55.up_proj.weight": "model-00043-of-00101.safetensors", + "model.layers.40.mlp.experts.55.down_proj.weight": "model-00043-of-00101.safetensors", + "model.layers.40.mlp.experts.56.gate_proj.weight": "model-00043-of-00101.safetensors", + "model.layers.40.mlp.experts.56.up_proj.weight": "model-00043-of-00101.safetensors", + "model.layers.40.mlp.experts.56.down_proj.weight": "model-00043-of-00101.safetensors", + "model.layers.40.mlp.experts.57.gate_proj.weight": "model-00043-of-00101.safetensors", + "model.layers.40.mlp.experts.57.up_proj.weight": "model-00043-of-00101.safetensors", + "model.layers.40.mlp.experts.57.down_proj.weight": "model-00043-of-00101.safetensors", + "model.layers.40.mlp.experts.58.gate_proj.weight": "model-00043-of-00101.safetensors", + "model.layers.40.mlp.experts.58.up_proj.weight": "model-00043-of-00101.safetensors", + "model.layers.40.mlp.experts.58.down_proj.weight": "model-00043-of-00101.safetensors", + "model.layers.40.mlp.experts.59.gate_proj.weight": "model-00043-of-00101.safetensors", + "model.layers.40.mlp.experts.59.up_proj.weight": "model-00043-of-00101.safetensors", + "model.layers.40.mlp.experts.59.down_proj.weight": "model-00043-of-00101.safetensors", + "model.layers.40.mlp.experts.60.gate_proj.weight": "model-00043-of-00101.safetensors", + "model.layers.40.mlp.experts.60.up_proj.weight": "model-00043-of-00101.safetensors", + "model.layers.40.mlp.experts.60.down_proj.weight": "model-00043-of-00101.safetensors", + "model.layers.40.mlp.experts.61.gate_proj.weight": "model-00043-of-00101.safetensors", + "model.layers.40.mlp.experts.61.up_proj.weight": "model-00043-of-00101.safetensors", + "model.layers.40.mlp.experts.61.down_proj.weight": "model-00043-of-00101.safetensors", + "model.layers.40.mlp.experts.62.gate_proj.weight": "model-00043-of-00101.safetensors", + "model.layers.40.mlp.experts.62.up_proj.weight": "model-00043-of-00101.safetensors", + "model.layers.40.mlp.experts.62.down_proj.weight": "model-00043-of-00101.safetensors", + "model.layers.40.mlp.experts.63.gate_proj.weight": "model-00043-of-00101.safetensors", + "model.layers.40.mlp.experts.63.up_proj.weight": "model-00043-of-00101.safetensors", + "model.layers.40.mlp.experts.63.down_proj.weight": "model-00043-of-00101.safetensors", + "model.layers.40.mlp.experts.64.gate_proj.weight": "model-00043-of-00101.safetensors", + "model.layers.40.mlp.experts.64.up_proj.weight": "model-00043-of-00101.safetensors", + "model.layers.40.mlp.experts.64.down_proj.weight": "model-00043-of-00101.safetensors", + "model.layers.40.mlp.experts.65.gate_proj.weight": "model-00043-of-00101.safetensors", + "model.layers.40.mlp.experts.65.up_proj.weight": "model-00043-of-00101.safetensors", + "model.layers.40.mlp.experts.65.down_proj.weight": "model-00043-of-00101.safetensors", + "model.layers.40.mlp.experts.66.gate_proj.weight": "model-00043-of-00101.safetensors", + "model.layers.40.mlp.experts.66.up_proj.weight": "model-00043-of-00101.safetensors", + "model.layers.40.mlp.experts.66.down_proj.weight": "model-00043-of-00101.safetensors", + "model.layers.40.mlp.experts.67.gate_proj.weight": "model-00043-of-00101.safetensors", + "model.layers.40.mlp.experts.67.up_proj.weight": "model-00043-of-00101.safetensors", + "model.layers.40.mlp.experts.67.down_proj.weight": "model-00043-of-00101.safetensors", + "model.layers.40.mlp.experts.68.gate_proj.weight": "model-00043-of-00101.safetensors", + "model.layers.40.mlp.experts.68.up_proj.weight": "model-00043-of-00101.safetensors", + "model.layers.40.mlp.experts.68.down_proj.weight": "model-00043-of-00101.safetensors", + "model.layers.40.mlp.experts.69.gate_proj.weight": "model-00043-of-00101.safetensors", + "model.layers.40.mlp.experts.69.up_proj.weight": "model-00043-of-00101.safetensors", + "model.layers.40.mlp.experts.69.down_proj.weight": "model-00043-of-00101.safetensors", + "model.layers.40.mlp.experts.70.gate_proj.weight": "model-00043-of-00101.safetensors", + "model.layers.40.mlp.experts.70.up_proj.weight": "model-00043-of-00101.safetensors", + "model.layers.40.mlp.experts.70.down_proj.weight": "model-00043-of-00101.safetensors", + "model.layers.40.mlp.experts.71.gate_proj.weight": "model-00043-of-00101.safetensors", + "model.layers.40.mlp.experts.71.up_proj.weight": "model-00043-of-00101.safetensors", + "model.layers.40.mlp.experts.71.down_proj.weight": "model-00043-of-00101.safetensors", + "model.layers.40.mlp.experts.72.gate_proj.weight": "model-00043-of-00101.safetensors", + "model.layers.40.mlp.experts.72.up_proj.weight": "model-00043-of-00101.safetensors", + "model.layers.40.mlp.experts.72.down_proj.weight": "model-00043-of-00101.safetensors", + "model.layers.40.mlp.experts.73.gate_proj.weight": "model-00043-of-00101.safetensors", + "model.layers.40.mlp.experts.73.up_proj.weight": "model-00043-of-00101.safetensors", + "model.layers.40.mlp.experts.73.down_proj.weight": "model-00043-of-00101.safetensors", + "model.layers.40.mlp.experts.74.gate_proj.weight": "model-00043-of-00101.safetensors", + "model.layers.40.mlp.experts.74.up_proj.weight": "model-00043-of-00101.safetensors", + "model.layers.40.mlp.experts.74.down_proj.weight": "model-00043-of-00101.safetensors", + "model.layers.40.mlp.experts.75.gate_proj.weight": "model-00043-of-00101.safetensors", + "model.layers.40.mlp.experts.75.up_proj.weight": "model-00043-of-00101.safetensors", + "model.layers.40.mlp.experts.75.down_proj.weight": "model-00043-of-00101.safetensors", + "model.layers.40.mlp.experts.76.gate_proj.weight": "model-00043-of-00101.safetensors", + "model.layers.40.mlp.experts.76.up_proj.weight": "model-00043-of-00101.safetensors", + "model.layers.40.mlp.experts.76.down_proj.weight": "model-00043-of-00101.safetensors", + "model.layers.40.mlp.experts.77.gate_proj.weight": "model-00043-of-00101.safetensors", + "model.layers.40.mlp.experts.77.up_proj.weight": "model-00043-of-00101.safetensors", + "model.layers.40.mlp.experts.77.down_proj.weight": "model-00043-of-00101.safetensors", + "model.layers.40.mlp.experts.78.gate_proj.weight": "model-00043-of-00101.safetensors", + "model.layers.40.mlp.experts.78.up_proj.weight": "model-00043-of-00101.safetensors", + "model.layers.40.mlp.experts.78.down_proj.weight": "model-00043-of-00101.safetensors", + "model.layers.40.mlp.experts.79.gate_proj.weight": "model-00043-of-00101.safetensors", + "model.layers.40.mlp.experts.79.up_proj.weight": "model-00043-of-00101.safetensors", + "model.layers.40.mlp.experts.79.down_proj.weight": "model-00043-of-00101.safetensors", + "model.layers.40.mlp.experts.80.gate_proj.weight": "model-00043-of-00101.safetensors", + "model.layers.40.mlp.experts.80.up_proj.weight": "model-00043-of-00101.safetensors", + "model.layers.40.mlp.experts.80.down_proj.weight": "model-00043-of-00101.safetensors", + "model.layers.40.mlp.experts.81.gate_proj.weight": "model-00043-of-00101.safetensors", + "model.layers.40.mlp.experts.81.up_proj.weight": "model-00043-of-00101.safetensors", + "model.layers.40.mlp.experts.81.down_proj.weight": "model-00043-of-00101.safetensors", + "model.layers.40.mlp.experts.82.gate_proj.weight": "model-00043-of-00101.safetensors", + "model.layers.40.mlp.experts.82.up_proj.weight": "model-00043-of-00101.safetensors", + "model.layers.40.mlp.experts.82.down_proj.weight": "model-00043-of-00101.safetensors", + "model.layers.40.mlp.experts.83.gate_proj.weight": "model-00043-of-00101.safetensors", + "model.layers.40.mlp.experts.83.up_proj.weight": "model-00043-of-00101.safetensors", + "model.layers.40.mlp.experts.83.down_proj.weight": "model-00043-of-00101.safetensors", + "model.layers.40.mlp.experts.84.gate_proj.weight": "model-00043-of-00101.safetensors", + "model.layers.40.mlp.experts.84.up_proj.weight": "model-00043-of-00101.safetensors", + "model.layers.40.mlp.experts.84.down_proj.weight": "model-00043-of-00101.safetensors", + "model.layers.40.mlp.experts.85.gate_proj.weight": "model-00043-of-00101.safetensors", + "model.layers.40.mlp.experts.85.up_proj.weight": "model-00043-of-00101.safetensors", + "model.layers.40.mlp.experts.85.down_proj.weight": "model-00043-of-00101.safetensors", + "model.layers.40.mlp.experts.86.gate_proj.weight": "model-00043-of-00101.safetensors", + "model.layers.40.mlp.experts.86.up_proj.weight": "model-00043-of-00101.safetensors", + "model.layers.40.mlp.experts.86.down_proj.weight": "model-00043-of-00101.safetensors", + "model.layers.40.mlp.experts.87.gate_proj.weight": "model-00043-of-00101.safetensors", + "model.layers.40.mlp.experts.87.up_proj.weight": "model-00043-of-00101.safetensors", + "model.layers.40.mlp.experts.87.down_proj.weight": "model-00043-of-00101.safetensors", + "model.layers.40.mlp.experts.88.gate_proj.weight": "model-00043-of-00101.safetensors", + "model.layers.40.mlp.experts.88.up_proj.weight": "model-00043-of-00101.safetensors", + "model.layers.40.mlp.experts.88.down_proj.weight": "model-00043-of-00101.safetensors", + "model.layers.40.mlp.experts.89.gate_proj.weight": "model-00043-of-00101.safetensors", + "model.layers.40.mlp.experts.89.up_proj.weight": "model-00043-of-00101.safetensors", + "model.layers.40.mlp.experts.89.down_proj.weight": "model-00043-of-00101.safetensors", + "model.layers.40.mlp.experts.90.gate_proj.weight": "model-00043-of-00101.safetensors", + "model.layers.40.mlp.experts.90.up_proj.weight": "model-00043-of-00101.safetensors", + "model.layers.40.mlp.experts.90.down_proj.weight": "model-00043-of-00101.safetensors", + "model.layers.40.mlp.experts.91.gate_proj.weight": "model-00043-of-00101.safetensors", + "model.layers.40.mlp.experts.91.up_proj.weight": "model-00043-of-00101.safetensors", + "model.layers.40.mlp.experts.91.down_proj.weight": "model-00043-of-00101.safetensors", + "model.layers.40.mlp.experts.92.gate_proj.weight": "model-00043-of-00101.safetensors", + "model.layers.40.mlp.experts.92.up_proj.weight": "model-00043-of-00101.safetensors", + "model.layers.40.mlp.experts.92.down_proj.weight": "model-00043-of-00101.safetensors", + "model.layers.40.mlp.experts.93.gate_proj.weight": "model-00043-of-00101.safetensors", + "model.layers.40.mlp.experts.93.up_proj.weight": "model-00043-of-00101.safetensors", + "model.layers.40.mlp.experts.93.down_proj.weight": "model-00043-of-00101.safetensors", + "model.layers.40.mlp.experts.94.gate_proj.weight": "model-00043-of-00101.safetensors", + "model.layers.40.mlp.experts.94.up_proj.weight": "model-00043-of-00101.safetensors", + "model.layers.40.mlp.experts.94.down_proj.weight": "model-00043-of-00101.safetensors", + "model.layers.40.mlp.experts.95.gate_proj.weight": "model-00043-of-00101.safetensors", + "model.layers.40.mlp.experts.95.up_proj.weight": "model-00043-of-00101.safetensors", + "model.layers.40.mlp.experts.95.down_proj.weight": "model-00043-of-00101.safetensors", + "model.layers.40.mlp.experts.96.gate_proj.weight": "model-00043-of-00101.safetensors", + "model.layers.40.mlp.experts.96.up_proj.weight": "model-00043-of-00101.safetensors", + "model.layers.40.mlp.experts.96.down_proj.weight": "model-00043-of-00101.safetensors", + "model.layers.40.mlp.experts.97.gate_proj.weight": "model-00043-of-00101.safetensors", + "model.layers.40.mlp.experts.97.up_proj.weight": "model-00043-of-00101.safetensors", + "model.layers.40.mlp.experts.97.down_proj.weight": "model-00043-of-00101.safetensors", + "model.layers.40.mlp.experts.98.gate_proj.weight": "model-00043-of-00101.safetensors", + "model.layers.40.mlp.experts.98.up_proj.weight": "model-00043-of-00101.safetensors", + "model.layers.40.mlp.experts.98.down_proj.weight": "model-00043-of-00101.safetensors", + "model.layers.40.mlp.experts.99.gate_proj.weight": "model-00043-of-00101.safetensors", + "model.layers.40.mlp.experts.99.up_proj.weight": "model-00043-of-00101.safetensors", + "model.layers.40.mlp.experts.99.down_proj.weight": "model-00043-of-00101.safetensors", + "model.layers.40.mlp.experts.100.gate_proj.weight": "model-00043-of-00101.safetensors", + "model.layers.40.mlp.experts.100.up_proj.weight": "model-00043-of-00101.safetensors", + "model.layers.40.mlp.experts.100.down_proj.weight": "model-00043-of-00101.safetensors", + "model.layers.40.mlp.experts.101.gate_proj.weight": "model-00043-of-00101.safetensors", + "model.layers.40.mlp.experts.101.up_proj.weight": "model-00043-of-00101.safetensors", + "model.layers.40.mlp.experts.101.down_proj.weight": "model-00043-of-00101.safetensors", + "model.layers.40.mlp.experts.102.gate_proj.weight": "model-00043-of-00101.safetensors", + "model.layers.40.mlp.experts.102.up_proj.weight": "model-00043-of-00101.safetensors", + "model.layers.40.mlp.experts.102.down_proj.weight": "model-00043-of-00101.safetensors", + "model.layers.40.mlp.experts.103.gate_proj.weight": "model-00043-of-00101.safetensors", + "model.layers.40.mlp.experts.103.up_proj.weight": "model-00043-of-00101.safetensors", + "model.layers.40.mlp.experts.103.down_proj.weight": "model-00043-of-00101.safetensors", + "model.layers.40.mlp.experts.104.gate_proj.weight": "model-00043-of-00101.safetensors", + "model.layers.40.mlp.experts.104.up_proj.weight": "model-00043-of-00101.safetensors", + "model.layers.40.mlp.experts.104.down_proj.weight": "model-00043-of-00101.safetensors", + "model.layers.40.mlp.experts.105.gate_proj.weight": "model-00043-of-00101.safetensors", + "model.layers.40.mlp.experts.105.up_proj.weight": "model-00043-of-00101.safetensors", + "model.layers.40.mlp.experts.105.down_proj.weight": "model-00043-of-00101.safetensors", + "model.layers.40.mlp.experts.106.gate_proj.weight": "model-00043-of-00101.safetensors", + "model.layers.40.mlp.experts.106.up_proj.weight": "model-00043-of-00101.safetensors", + "model.layers.40.mlp.experts.106.down_proj.weight": "model-00043-of-00101.safetensors", + "model.layers.40.mlp.experts.107.gate_proj.weight": "model-00043-of-00101.safetensors", + "model.layers.40.mlp.experts.107.up_proj.weight": "model-00043-of-00101.safetensors", + "model.layers.40.mlp.experts.107.down_proj.weight": "model-00044-of-00101.safetensors", + "model.layers.40.mlp.experts.108.gate_proj.weight": "model-00044-of-00101.safetensors", + "model.layers.40.mlp.experts.108.up_proj.weight": "model-00044-of-00101.safetensors", + "model.layers.40.mlp.experts.108.down_proj.weight": "model-00044-of-00101.safetensors", + "model.layers.40.mlp.experts.109.gate_proj.weight": "model-00044-of-00101.safetensors", + "model.layers.40.mlp.experts.109.up_proj.weight": "model-00044-of-00101.safetensors", + "model.layers.40.mlp.experts.109.down_proj.weight": "model-00044-of-00101.safetensors", + "model.layers.40.mlp.experts.110.gate_proj.weight": "model-00044-of-00101.safetensors", + "model.layers.40.mlp.experts.110.up_proj.weight": "model-00044-of-00101.safetensors", + "model.layers.40.mlp.experts.110.down_proj.weight": "model-00044-of-00101.safetensors", + "model.layers.40.mlp.experts.111.gate_proj.weight": "model-00044-of-00101.safetensors", + "model.layers.40.mlp.experts.111.up_proj.weight": "model-00044-of-00101.safetensors", + "model.layers.40.mlp.experts.111.down_proj.weight": "model-00044-of-00101.safetensors", + "model.layers.40.mlp.experts.112.gate_proj.weight": "model-00044-of-00101.safetensors", + "model.layers.40.mlp.experts.112.up_proj.weight": "model-00044-of-00101.safetensors", + "model.layers.40.mlp.experts.112.down_proj.weight": "model-00044-of-00101.safetensors", + "model.layers.40.mlp.experts.113.gate_proj.weight": "model-00044-of-00101.safetensors", + "model.layers.40.mlp.experts.113.up_proj.weight": "model-00044-of-00101.safetensors", + "model.layers.40.mlp.experts.113.down_proj.weight": "model-00044-of-00101.safetensors", + "model.layers.40.mlp.experts.114.gate_proj.weight": "model-00044-of-00101.safetensors", + "model.layers.40.mlp.experts.114.up_proj.weight": "model-00044-of-00101.safetensors", + "model.layers.40.mlp.experts.114.down_proj.weight": "model-00044-of-00101.safetensors", + "model.layers.40.mlp.experts.115.gate_proj.weight": "model-00044-of-00101.safetensors", + "model.layers.40.mlp.experts.115.up_proj.weight": "model-00044-of-00101.safetensors", + "model.layers.40.mlp.experts.115.down_proj.weight": "model-00044-of-00101.safetensors", + "model.layers.40.mlp.experts.116.gate_proj.weight": "model-00044-of-00101.safetensors", + "model.layers.40.mlp.experts.116.up_proj.weight": "model-00044-of-00101.safetensors", + "model.layers.40.mlp.experts.116.down_proj.weight": "model-00044-of-00101.safetensors", + "model.layers.40.mlp.experts.117.gate_proj.weight": "model-00044-of-00101.safetensors", + "model.layers.40.mlp.experts.117.up_proj.weight": "model-00044-of-00101.safetensors", + "model.layers.40.mlp.experts.117.down_proj.weight": "model-00044-of-00101.safetensors", + "model.layers.40.mlp.experts.118.gate_proj.weight": "model-00044-of-00101.safetensors", + "model.layers.40.mlp.experts.118.up_proj.weight": "model-00044-of-00101.safetensors", + "model.layers.40.mlp.experts.118.down_proj.weight": "model-00044-of-00101.safetensors", + "model.layers.40.mlp.experts.119.gate_proj.weight": "model-00044-of-00101.safetensors", + "model.layers.40.mlp.experts.119.up_proj.weight": "model-00044-of-00101.safetensors", + "model.layers.40.mlp.experts.119.down_proj.weight": "model-00044-of-00101.safetensors", + "model.layers.40.mlp.gate.weight": "model-00044-of-00101.safetensors", + "model.layers.40.mlp.gate.e_score_correction_bias": "model-00044-of-00101.safetensors", + "model.layers.40.mlp.shared_experts.gate_proj.weight": "model-00044-of-00101.safetensors", + "model.layers.40.mlp.shared_experts.up_proj.weight": "model-00044-of-00101.safetensors", + "model.layers.40.mlp.shared_experts.down_proj.weight": "model-00044-of-00101.safetensors", + "model.layers.40.input_layernorm.weight": "model-00044-of-00101.safetensors", + "model.layers.40.post_attention_layernorm.weight": "model-00044-of-00101.safetensors", + "model.layers.41.self_attn.q_proj.weight": "model-00044-of-00101.safetensors", + "model.layers.41.self_attn.q_proj.bias": "model-00044-of-00101.safetensors", + "model.layers.41.self_attn.k_proj.weight": "model-00044-of-00101.safetensors", + "model.layers.41.self_attn.k_proj.bias": "model-00044-of-00101.safetensors", + "model.layers.41.self_attn.v_proj.weight": "model-00044-of-00101.safetensors", + "model.layers.41.self_attn.v_proj.bias": "model-00044-of-00101.safetensors", + "model.layers.41.self_attn.o_proj.weight": "model-00044-of-00101.safetensors", + "model.layers.41.self_attn.q_norm.weight": "model-00044-of-00101.safetensors", + "model.layers.41.self_attn.k_norm.weight": "model-00044-of-00101.safetensors", + "model.layers.41.mlp.experts.0.gate_proj.weight": "model-00044-of-00101.safetensors", + "model.layers.41.mlp.experts.0.up_proj.weight": "model-00044-of-00101.safetensors", + "model.layers.41.mlp.experts.0.down_proj.weight": "model-00044-of-00101.safetensors", + "model.layers.41.mlp.experts.1.gate_proj.weight": "model-00044-of-00101.safetensors", + "model.layers.41.mlp.experts.1.up_proj.weight": "model-00044-of-00101.safetensors", + "model.layers.41.mlp.experts.1.down_proj.weight": "model-00044-of-00101.safetensors", + "model.layers.41.mlp.experts.2.gate_proj.weight": "model-00044-of-00101.safetensors", + "model.layers.41.mlp.experts.2.up_proj.weight": "model-00044-of-00101.safetensors", + "model.layers.41.mlp.experts.2.down_proj.weight": "model-00044-of-00101.safetensors", + "model.layers.41.mlp.experts.3.gate_proj.weight": "model-00044-of-00101.safetensors", + "model.layers.41.mlp.experts.3.up_proj.weight": "model-00044-of-00101.safetensors", + "model.layers.41.mlp.experts.3.down_proj.weight": "model-00044-of-00101.safetensors", + "model.layers.41.mlp.experts.4.gate_proj.weight": "model-00044-of-00101.safetensors", + "model.layers.41.mlp.experts.4.up_proj.weight": "model-00044-of-00101.safetensors", + "model.layers.41.mlp.experts.4.down_proj.weight": "model-00044-of-00101.safetensors", + "model.layers.41.mlp.experts.5.gate_proj.weight": "model-00044-of-00101.safetensors", + "model.layers.41.mlp.experts.5.up_proj.weight": "model-00044-of-00101.safetensors", + "model.layers.41.mlp.experts.5.down_proj.weight": "model-00044-of-00101.safetensors", + "model.layers.41.mlp.experts.6.gate_proj.weight": "model-00044-of-00101.safetensors", + "model.layers.41.mlp.experts.6.up_proj.weight": "model-00044-of-00101.safetensors", + "model.layers.41.mlp.experts.6.down_proj.weight": "model-00044-of-00101.safetensors", + "model.layers.41.mlp.experts.7.gate_proj.weight": "model-00044-of-00101.safetensors", + "model.layers.41.mlp.experts.7.up_proj.weight": "model-00044-of-00101.safetensors", + "model.layers.41.mlp.experts.7.down_proj.weight": "model-00044-of-00101.safetensors", + "model.layers.41.mlp.experts.8.gate_proj.weight": "model-00044-of-00101.safetensors", + "model.layers.41.mlp.experts.8.up_proj.weight": "model-00044-of-00101.safetensors", + "model.layers.41.mlp.experts.8.down_proj.weight": "model-00044-of-00101.safetensors", + "model.layers.41.mlp.experts.9.gate_proj.weight": "model-00044-of-00101.safetensors", + "model.layers.41.mlp.experts.9.up_proj.weight": "model-00044-of-00101.safetensors", + "model.layers.41.mlp.experts.9.down_proj.weight": "model-00044-of-00101.safetensors", + "model.layers.41.mlp.experts.10.gate_proj.weight": "model-00044-of-00101.safetensors", + "model.layers.41.mlp.experts.10.up_proj.weight": "model-00044-of-00101.safetensors", + "model.layers.41.mlp.experts.10.down_proj.weight": "model-00044-of-00101.safetensors", + "model.layers.41.mlp.experts.11.gate_proj.weight": "model-00044-of-00101.safetensors", + "model.layers.41.mlp.experts.11.up_proj.weight": "model-00044-of-00101.safetensors", + "model.layers.41.mlp.experts.11.down_proj.weight": "model-00044-of-00101.safetensors", + "model.layers.41.mlp.experts.12.gate_proj.weight": "model-00044-of-00101.safetensors", + "model.layers.41.mlp.experts.12.up_proj.weight": "model-00044-of-00101.safetensors", + "model.layers.41.mlp.experts.12.down_proj.weight": "model-00044-of-00101.safetensors", + "model.layers.41.mlp.experts.13.gate_proj.weight": "model-00044-of-00101.safetensors", + "model.layers.41.mlp.experts.13.up_proj.weight": "model-00044-of-00101.safetensors", + "model.layers.41.mlp.experts.13.down_proj.weight": "model-00044-of-00101.safetensors", + "model.layers.41.mlp.experts.14.gate_proj.weight": "model-00044-of-00101.safetensors", + "model.layers.41.mlp.experts.14.up_proj.weight": "model-00044-of-00101.safetensors", + "model.layers.41.mlp.experts.14.down_proj.weight": "model-00044-of-00101.safetensors", + "model.layers.41.mlp.experts.15.gate_proj.weight": "model-00044-of-00101.safetensors", + "model.layers.41.mlp.experts.15.up_proj.weight": "model-00044-of-00101.safetensors", + "model.layers.41.mlp.experts.15.down_proj.weight": "model-00044-of-00101.safetensors", + "model.layers.41.mlp.experts.16.gate_proj.weight": "model-00044-of-00101.safetensors", + "model.layers.41.mlp.experts.16.up_proj.weight": "model-00044-of-00101.safetensors", + "model.layers.41.mlp.experts.16.down_proj.weight": "model-00044-of-00101.safetensors", + "model.layers.41.mlp.experts.17.gate_proj.weight": "model-00044-of-00101.safetensors", + "model.layers.41.mlp.experts.17.up_proj.weight": "model-00044-of-00101.safetensors", + "model.layers.41.mlp.experts.17.down_proj.weight": "model-00044-of-00101.safetensors", + "model.layers.41.mlp.experts.18.gate_proj.weight": "model-00044-of-00101.safetensors", + "model.layers.41.mlp.experts.18.up_proj.weight": "model-00044-of-00101.safetensors", + "model.layers.41.mlp.experts.18.down_proj.weight": "model-00044-of-00101.safetensors", + "model.layers.41.mlp.experts.19.gate_proj.weight": "model-00044-of-00101.safetensors", + "model.layers.41.mlp.experts.19.up_proj.weight": "model-00044-of-00101.safetensors", + "model.layers.41.mlp.experts.19.down_proj.weight": "model-00044-of-00101.safetensors", + "model.layers.41.mlp.experts.20.gate_proj.weight": "model-00044-of-00101.safetensors", + "model.layers.41.mlp.experts.20.up_proj.weight": "model-00044-of-00101.safetensors", + "model.layers.41.mlp.experts.20.down_proj.weight": "model-00044-of-00101.safetensors", + "model.layers.41.mlp.experts.21.gate_proj.weight": "model-00044-of-00101.safetensors", + "model.layers.41.mlp.experts.21.up_proj.weight": "model-00044-of-00101.safetensors", + "model.layers.41.mlp.experts.21.down_proj.weight": "model-00044-of-00101.safetensors", + "model.layers.41.mlp.experts.22.gate_proj.weight": "model-00044-of-00101.safetensors", + "model.layers.41.mlp.experts.22.up_proj.weight": "model-00044-of-00101.safetensors", + "model.layers.41.mlp.experts.22.down_proj.weight": "model-00044-of-00101.safetensors", + "model.layers.41.mlp.experts.23.gate_proj.weight": "model-00044-of-00101.safetensors", + "model.layers.41.mlp.experts.23.up_proj.weight": "model-00044-of-00101.safetensors", + "model.layers.41.mlp.experts.23.down_proj.weight": "model-00044-of-00101.safetensors", + "model.layers.41.mlp.experts.24.gate_proj.weight": "model-00044-of-00101.safetensors", + "model.layers.41.mlp.experts.24.up_proj.weight": "model-00044-of-00101.safetensors", + "model.layers.41.mlp.experts.24.down_proj.weight": "model-00044-of-00101.safetensors", + "model.layers.41.mlp.experts.25.gate_proj.weight": "model-00044-of-00101.safetensors", + "model.layers.41.mlp.experts.25.up_proj.weight": "model-00044-of-00101.safetensors", + "model.layers.41.mlp.experts.25.down_proj.weight": "model-00044-of-00101.safetensors", + "model.layers.41.mlp.experts.26.gate_proj.weight": "model-00044-of-00101.safetensors", + "model.layers.41.mlp.experts.26.up_proj.weight": "model-00044-of-00101.safetensors", + "model.layers.41.mlp.experts.26.down_proj.weight": "model-00044-of-00101.safetensors", + "model.layers.41.mlp.experts.27.gate_proj.weight": "model-00044-of-00101.safetensors", + "model.layers.41.mlp.experts.27.up_proj.weight": "model-00044-of-00101.safetensors", + "model.layers.41.mlp.experts.27.down_proj.weight": "model-00044-of-00101.safetensors", + "model.layers.41.mlp.experts.28.gate_proj.weight": "model-00044-of-00101.safetensors", + "model.layers.41.mlp.experts.28.up_proj.weight": "model-00044-of-00101.safetensors", + "model.layers.41.mlp.experts.28.down_proj.weight": "model-00044-of-00101.safetensors", + "model.layers.41.mlp.experts.29.gate_proj.weight": "model-00044-of-00101.safetensors", + "model.layers.41.mlp.experts.29.up_proj.weight": "model-00044-of-00101.safetensors", + "model.layers.41.mlp.experts.29.down_proj.weight": "model-00044-of-00101.safetensors", + "model.layers.41.mlp.experts.30.gate_proj.weight": "model-00044-of-00101.safetensors", + "model.layers.41.mlp.experts.30.up_proj.weight": "model-00044-of-00101.safetensors", + "model.layers.41.mlp.experts.30.down_proj.weight": "model-00044-of-00101.safetensors", + "model.layers.41.mlp.experts.31.gate_proj.weight": "model-00044-of-00101.safetensors", + "model.layers.41.mlp.experts.31.up_proj.weight": "model-00044-of-00101.safetensors", + "model.layers.41.mlp.experts.31.down_proj.weight": "model-00044-of-00101.safetensors", + "model.layers.41.mlp.experts.32.gate_proj.weight": "model-00044-of-00101.safetensors", + "model.layers.41.mlp.experts.32.up_proj.weight": "model-00044-of-00101.safetensors", + "model.layers.41.mlp.experts.32.down_proj.weight": "model-00044-of-00101.safetensors", + "model.layers.41.mlp.experts.33.gate_proj.weight": "model-00044-of-00101.safetensors", + "model.layers.41.mlp.experts.33.up_proj.weight": "model-00044-of-00101.safetensors", + "model.layers.41.mlp.experts.33.down_proj.weight": "model-00044-of-00101.safetensors", + "model.layers.41.mlp.experts.34.gate_proj.weight": "model-00044-of-00101.safetensors", + "model.layers.41.mlp.experts.34.up_proj.weight": "model-00044-of-00101.safetensors", + "model.layers.41.mlp.experts.34.down_proj.weight": "model-00044-of-00101.safetensors", + "model.layers.41.mlp.experts.35.gate_proj.weight": "model-00044-of-00101.safetensors", + "model.layers.41.mlp.experts.35.up_proj.weight": "model-00044-of-00101.safetensors", + "model.layers.41.mlp.experts.35.down_proj.weight": "model-00044-of-00101.safetensors", + "model.layers.41.mlp.experts.36.gate_proj.weight": "model-00044-of-00101.safetensors", + "model.layers.41.mlp.experts.36.up_proj.weight": "model-00044-of-00101.safetensors", + "model.layers.41.mlp.experts.36.down_proj.weight": "model-00044-of-00101.safetensors", + "model.layers.41.mlp.experts.37.gate_proj.weight": "model-00044-of-00101.safetensors", + "model.layers.41.mlp.experts.37.up_proj.weight": "model-00044-of-00101.safetensors", + "model.layers.41.mlp.experts.37.down_proj.weight": "model-00044-of-00101.safetensors", + "model.layers.41.mlp.experts.38.gate_proj.weight": "model-00044-of-00101.safetensors", + "model.layers.41.mlp.experts.38.up_proj.weight": "model-00044-of-00101.safetensors", + "model.layers.41.mlp.experts.38.down_proj.weight": "model-00044-of-00101.safetensors", + "model.layers.41.mlp.experts.39.gate_proj.weight": "model-00044-of-00101.safetensors", + "model.layers.41.mlp.experts.39.up_proj.weight": "model-00044-of-00101.safetensors", + "model.layers.41.mlp.experts.39.down_proj.weight": "model-00044-of-00101.safetensors", + "model.layers.41.mlp.experts.40.gate_proj.weight": "model-00044-of-00101.safetensors", + "model.layers.41.mlp.experts.40.up_proj.weight": "model-00044-of-00101.safetensors", + "model.layers.41.mlp.experts.40.down_proj.weight": "model-00044-of-00101.safetensors", + "model.layers.41.mlp.experts.41.gate_proj.weight": "model-00044-of-00101.safetensors", + "model.layers.41.mlp.experts.41.up_proj.weight": "model-00044-of-00101.safetensors", + "model.layers.41.mlp.experts.41.down_proj.weight": "model-00044-of-00101.safetensors", + "model.layers.41.mlp.experts.42.gate_proj.weight": "model-00044-of-00101.safetensors", + "model.layers.41.mlp.experts.42.up_proj.weight": "model-00044-of-00101.safetensors", + "model.layers.41.mlp.experts.42.down_proj.weight": "model-00044-of-00101.safetensors", + "model.layers.41.mlp.experts.43.gate_proj.weight": "model-00044-of-00101.safetensors", + "model.layers.41.mlp.experts.43.up_proj.weight": "model-00044-of-00101.safetensors", + "model.layers.41.mlp.experts.43.down_proj.weight": "model-00044-of-00101.safetensors", + "model.layers.41.mlp.experts.44.gate_proj.weight": "model-00044-of-00101.safetensors", + "model.layers.41.mlp.experts.44.up_proj.weight": "model-00044-of-00101.safetensors", + "model.layers.41.mlp.experts.44.down_proj.weight": "model-00044-of-00101.safetensors", + "model.layers.41.mlp.experts.45.gate_proj.weight": "model-00044-of-00101.safetensors", + "model.layers.41.mlp.experts.45.up_proj.weight": "model-00044-of-00101.safetensors", + "model.layers.41.mlp.experts.45.down_proj.weight": "model-00044-of-00101.safetensors", + "model.layers.41.mlp.experts.46.gate_proj.weight": "model-00044-of-00101.safetensors", + "model.layers.41.mlp.experts.46.up_proj.weight": "model-00044-of-00101.safetensors", + "model.layers.41.mlp.experts.46.down_proj.weight": "model-00044-of-00101.safetensors", + "model.layers.41.mlp.experts.47.gate_proj.weight": "model-00044-of-00101.safetensors", + "model.layers.41.mlp.experts.47.up_proj.weight": "model-00044-of-00101.safetensors", + "model.layers.41.mlp.experts.47.down_proj.weight": "model-00044-of-00101.safetensors", + "model.layers.41.mlp.experts.48.gate_proj.weight": "model-00044-of-00101.safetensors", + "model.layers.41.mlp.experts.48.up_proj.weight": "model-00044-of-00101.safetensors", + "model.layers.41.mlp.experts.48.down_proj.weight": "model-00044-of-00101.safetensors", + "model.layers.41.mlp.experts.49.gate_proj.weight": "model-00044-of-00101.safetensors", + "model.layers.41.mlp.experts.49.up_proj.weight": "model-00044-of-00101.safetensors", + "model.layers.41.mlp.experts.49.down_proj.weight": "model-00044-of-00101.safetensors", + "model.layers.41.mlp.experts.50.gate_proj.weight": "model-00044-of-00101.safetensors", + "model.layers.41.mlp.experts.50.up_proj.weight": "model-00044-of-00101.safetensors", + "model.layers.41.mlp.experts.50.down_proj.weight": "model-00044-of-00101.safetensors", + "model.layers.41.mlp.experts.51.gate_proj.weight": "model-00044-of-00101.safetensors", + "model.layers.41.mlp.experts.51.up_proj.weight": "model-00044-of-00101.safetensors", + "model.layers.41.mlp.experts.51.down_proj.weight": "model-00044-of-00101.safetensors", + "model.layers.41.mlp.experts.52.gate_proj.weight": "model-00044-of-00101.safetensors", + "model.layers.41.mlp.experts.52.up_proj.weight": "model-00044-of-00101.safetensors", + "model.layers.41.mlp.experts.52.down_proj.weight": "model-00044-of-00101.safetensors", + "model.layers.41.mlp.experts.53.gate_proj.weight": "model-00044-of-00101.safetensors", + "model.layers.41.mlp.experts.53.up_proj.weight": "model-00044-of-00101.safetensors", + "model.layers.41.mlp.experts.53.down_proj.weight": "model-00044-of-00101.safetensors", + "model.layers.41.mlp.experts.54.gate_proj.weight": "model-00044-of-00101.safetensors", + "model.layers.41.mlp.experts.54.up_proj.weight": "model-00044-of-00101.safetensors", + "model.layers.41.mlp.experts.54.down_proj.weight": "model-00044-of-00101.safetensors", + "model.layers.41.mlp.experts.55.gate_proj.weight": "model-00044-of-00101.safetensors", + "model.layers.41.mlp.experts.55.up_proj.weight": "model-00044-of-00101.safetensors", + "model.layers.41.mlp.experts.55.down_proj.weight": "model-00044-of-00101.safetensors", + "model.layers.41.mlp.experts.56.gate_proj.weight": "model-00044-of-00101.safetensors", + "model.layers.41.mlp.experts.56.up_proj.weight": "model-00044-of-00101.safetensors", + "model.layers.41.mlp.experts.56.down_proj.weight": "model-00044-of-00101.safetensors", + "model.layers.41.mlp.experts.57.gate_proj.weight": "model-00044-of-00101.safetensors", + "model.layers.41.mlp.experts.57.up_proj.weight": "model-00044-of-00101.safetensors", + "model.layers.41.mlp.experts.57.down_proj.weight": "model-00044-of-00101.safetensors", + "model.layers.41.mlp.experts.58.gate_proj.weight": "model-00044-of-00101.safetensors", + "model.layers.41.mlp.experts.58.up_proj.weight": "model-00044-of-00101.safetensors", + "model.layers.41.mlp.experts.58.down_proj.weight": "model-00044-of-00101.safetensors", + "model.layers.41.mlp.experts.59.gate_proj.weight": "model-00044-of-00101.safetensors", + "model.layers.41.mlp.experts.59.up_proj.weight": "model-00044-of-00101.safetensors", + "model.layers.41.mlp.experts.59.down_proj.weight": "model-00044-of-00101.safetensors", + "model.layers.41.mlp.experts.60.gate_proj.weight": "model-00044-of-00101.safetensors", + "model.layers.41.mlp.experts.60.up_proj.weight": "model-00044-of-00101.safetensors", + "model.layers.41.mlp.experts.60.down_proj.weight": "model-00044-of-00101.safetensors", + "model.layers.41.mlp.experts.61.gate_proj.weight": "model-00044-of-00101.safetensors", + "model.layers.41.mlp.experts.61.up_proj.weight": "model-00044-of-00101.safetensors", + "model.layers.41.mlp.experts.61.down_proj.weight": "model-00044-of-00101.safetensors", + "model.layers.41.mlp.experts.62.gate_proj.weight": "model-00044-of-00101.safetensors", + "model.layers.41.mlp.experts.62.up_proj.weight": "model-00044-of-00101.safetensors", + "model.layers.41.mlp.experts.62.down_proj.weight": "model-00044-of-00101.safetensors", + "model.layers.41.mlp.experts.63.gate_proj.weight": "model-00044-of-00101.safetensors", + "model.layers.41.mlp.experts.63.up_proj.weight": "model-00044-of-00101.safetensors", + "model.layers.41.mlp.experts.63.down_proj.weight": "model-00044-of-00101.safetensors", + "model.layers.41.mlp.experts.64.gate_proj.weight": "model-00044-of-00101.safetensors", + "model.layers.41.mlp.experts.64.up_proj.weight": "model-00044-of-00101.safetensors", + "model.layers.41.mlp.experts.64.down_proj.weight": "model-00044-of-00101.safetensors", + "model.layers.41.mlp.experts.65.gate_proj.weight": "model-00044-of-00101.safetensors", + "model.layers.41.mlp.experts.65.up_proj.weight": "model-00044-of-00101.safetensors", + "model.layers.41.mlp.experts.65.down_proj.weight": "model-00044-of-00101.safetensors", + "model.layers.41.mlp.experts.66.gate_proj.weight": "model-00044-of-00101.safetensors", + "model.layers.41.mlp.experts.66.up_proj.weight": "model-00044-of-00101.safetensors", + "model.layers.41.mlp.experts.66.down_proj.weight": "model-00044-of-00101.safetensors", + "model.layers.41.mlp.experts.67.gate_proj.weight": "model-00044-of-00101.safetensors", + "model.layers.41.mlp.experts.67.up_proj.weight": "model-00044-of-00101.safetensors", + "model.layers.41.mlp.experts.67.down_proj.weight": "model-00044-of-00101.safetensors", + "model.layers.41.mlp.experts.68.gate_proj.weight": "model-00044-of-00101.safetensors", + "model.layers.41.mlp.experts.68.up_proj.weight": "model-00044-of-00101.safetensors", + "model.layers.41.mlp.experts.68.down_proj.weight": "model-00044-of-00101.safetensors", + "model.layers.41.mlp.experts.69.gate_proj.weight": "model-00044-of-00101.safetensors", + "model.layers.41.mlp.experts.69.up_proj.weight": "model-00044-of-00101.safetensors", + "model.layers.41.mlp.experts.69.down_proj.weight": "model-00044-of-00101.safetensors", + "model.layers.41.mlp.experts.70.gate_proj.weight": "model-00044-of-00101.safetensors", + "model.layers.41.mlp.experts.70.up_proj.weight": "model-00044-of-00101.safetensors", + "model.layers.41.mlp.experts.70.down_proj.weight": "model-00044-of-00101.safetensors", + "model.layers.41.mlp.experts.71.gate_proj.weight": "model-00044-of-00101.safetensors", + "model.layers.41.mlp.experts.71.up_proj.weight": "model-00044-of-00101.safetensors", + "model.layers.41.mlp.experts.71.down_proj.weight": "model-00044-of-00101.safetensors", + "model.layers.41.mlp.experts.72.gate_proj.weight": "model-00044-of-00101.safetensors", + "model.layers.41.mlp.experts.72.up_proj.weight": "model-00044-of-00101.safetensors", + "model.layers.41.mlp.experts.72.down_proj.weight": "model-00044-of-00101.safetensors", + "model.layers.41.mlp.experts.73.gate_proj.weight": "model-00044-of-00101.safetensors", + "model.layers.41.mlp.experts.73.up_proj.weight": "model-00044-of-00101.safetensors", + "model.layers.41.mlp.experts.73.down_proj.weight": "model-00044-of-00101.safetensors", + "model.layers.41.mlp.experts.74.gate_proj.weight": "model-00044-of-00101.safetensors", + "model.layers.41.mlp.experts.74.up_proj.weight": "model-00044-of-00101.safetensors", + "model.layers.41.mlp.experts.74.down_proj.weight": "model-00044-of-00101.safetensors", + "model.layers.41.mlp.experts.75.gate_proj.weight": "model-00044-of-00101.safetensors", + "model.layers.41.mlp.experts.75.up_proj.weight": "model-00044-of-00101.safetensors", + "model.layers.41.mlp.experts.75.down_proj.weight": "model-00044-of-00101.safetensors", + "model.layers.41.mlp.experts.76.gate_proj.weight": "model-00044-of-00101.safetensors", + "model.layers.41.mlp.experts.76.up_proj.weight": "model-00044-of-00101.safetensors", + "model.layers.41.mlp.experts.76.down_proj.weight": "model-00044-of-00101.safetensors", + "model.layers.41.mlp.experts.77.gate_proj.weight": "model-00044-of-00101.safetensors", + "model.layers.41.mlp.experts.77.up_proj.weight": "model-00044-of-00101.safetensors", + "model.layers.41.mlp.experts.77.down_proj.weight": "model-00044-of-00101.safetensors", + "model.layers.41.mlp.experts.78.gate_proj.weight": "model-00044-of-00101.safetensors", + "model.layers.41.mlp.experts.78.up_proj.weight": "model-00044-of-00101.safetensors", + "model.layers.41.mlp.experts.78.down_proj.weight": "model-00044-of-00101.safetensors", + "model.layers.41.mlp.experts.79.gate_proj.weight": "model-00044-of-00101.safetensors", + "model.layers.41.mlp.experts.79.up_proj.weight": "model-00044-of-00101.safetensors", + "model.layers.41.mlp.experts.79.down_proj.weight": "model-00044-of-00101.safetensors", + "model.layers.41.mlp.experts.80.gate_proj.weight": "model-00044-of-00101.safetensors", + "model.layers.41.mlp.experts.80.up_proj.weight": "model-00044-of-00101.safetensors", + "model.layers.41.mlp.experts.80.down_proj.weight": "model-00044-of-00101.safetensors", + "model.layers.41.mlp.experts.81.gate_proj.weight": "model-00044-of-00101.safetensors", + "model.layers.41.mlp.experts.81.up_proj.weight": "model-00044-of-00101.safetensors", + "model.layers.41.mlp.experts.81.down_proj.weight": "model-00044-of-00101.safetensors", + "model.layers.41.mlp.experts.82.gate_proj.weight": "model-00044-of-00101.safetensors", + "model.layers.41.mlp.experts.82.up_proj.weight": "model-00044-of-00101.safetensors", + "model.layers.41.mlp.experts.82.down_proj.weight": "model-00044-of-00101.safetensors", + "model.layers.41.mlp.experts.83.gate_proj.weight": "model-00044-of-00101.safetensors", + "model.layers.41.mlp.experts.83.up_proj.weight": "model-00044-of-00101.safetensors", + "model.layers.41.mlp.experts.83.down_proj.weight": "model-00044-of-00101.safetensors", + "model.layers.41.mlp.experts.84.gate_proj.weight": "model-00044-of-00101.safetensors", + "model.layers.41.mlp.experts.84.up_proj.weight": "model-00044-of-00101.safetensors", + "model.layers.41.mlp.experts.84.down_proj.weight": "model-00044-of-00101.safetensors", + "model.layers.41.mlp.experts.85.gate_proj.weight": "model-00044-of-00101.safetensors", + "model.layers.41.mlp.experts.85.up_proj.weight": "model-00044-of-00101.safetensors", + "model.layers.41.mlp.experts.85.down_proj.weight": "model-00044-of-00101.safetensors", + "model.layers.41.mlp.experts.86.gate_proj.weight": "model-00044-of-00101.safetensors", + "model.layers.41.mlp.experts.86.up_proj.weight": "model-00044-of-00101.safetensors", + "model.layers.41.mlp.experts.86.down_proj.weight": "model-00044-of-00101.safetensors", + "model.layers.41.mlp.experts.87.gate_proj.weight": "model-00044-of-00101.safetensors", + "model.layers.41.mlp.experts.87.up_proj.weight": "model-00044-of-00101.safetensors", + "model.layers.41.mlp.experts.87.down_proj.weight": "model-00044-of-00101.safetensors", + "model.layers.41.mlp.experts.88.gate_proj.weight": "model-00044-of-00101.safetensors", + "model.layers.41.mlp.experts.88.up_proj.weight": "model-00044-of-00101.safetensors", + "model.layers.41.mlp.experts.88.down_proj.weight": "model-00044-of-00101.safetensors", + "model.layers.41.mlp.experts.89.gate_proj.weight": "model-00044-of-00101.safetensors", + "model.layers.41.mlp.experts.89.up_proj.weight": "model-00044-of-00101.safetensors", + "model.layers.41.mlp.experts.89.down_proj.weight": "model-00044-of-00101.safetensors", + "model.layers.41.mlp.experts.90.gate_proj.weight": "model-00044-of-00101.safetensors", + "model.layers.41.mlp.experts.90.up_proj.weight": "model-00044-of-00101.safetensors", + "model.layers.41.mlp.experts.90.down_proj.weight": "model-00044-of-00101.safetensors", + "model.layers.41.mlp.experts.91.gate_proj.weight": "model-00044-of-00101.safetensors", + "model.layers.41.mlp.experts.91.up_proj.weight": "model-00044-of-00101.safetensors", + "model.layers.41.mlp.experts.91.down_proj.weight": "model-00044-of-00101.safetensors", + "model.layers.41.mlp.experts.92.gate_proj.weight": "model-00044-of-00101.safetensors", + "model.layers.41.mlp.experts.92.up_proj.weight": "model-00044-of-00101.safetensors", + "model.layers.41.mlp.experts.92.down_proj.weight": "model-00044-of-00101.safetensors", + "model.layers.41.mlp.experts.93.gate_proj.weight": "model-00044-of-00101.safetensors", + "model.layers.41.mlp.experts.93.up_proj.weight": "model-00044-of-00101.safetensors", + "model.layers.41.mlp.experts.93.down_proj.weight": "model-00044-of-00101.safetensors", + "model.layers.41.mlp.experts.94.gate_proj.weight": "model-00044-of-00101.safetensors", + "model.layers.41.mlp.experts.94.up_proj.weight": "model-00045-of-00101.safetensors", + "model.layers.41.mlp.experts.94.down_proj.weight": "model-00045-of-00101.safetensors", + "model.layers.41.mlp.experts.95.gate_proj.weight": "model-00045-of-00101.safetensors", + "model.layers.41.mlp.experts.95.up_proj.weight": "model-00045-of-00101.safetensors", + "model.layers.41.mlp.experts.95.down_proj.weight": "model-00045-of-00101.safetensors", + "model.layers.41.mlp.experts.96.gate_proj.weight": "model-00045-of-00101.safetensors", + "model.layers.41.mlp.experts.96.up_proj.weight": "model-00045-of-00101.safetensors", + "model.layers.41.mlp.experts.96.down_proj.weight": "model-00045-of-00101.safetensors", + "model.layers.41.mlp.experts.97.gate_proj.weight": "model-00045-of-00101.safetensors", + "model.layers.41.mlp.experts.97.up_proj.weight": "model-00045-of-00101.safetensors", + "model.layers.41.mlp.experts.97.down_proj.weight": "model-00045-of-00101.safetensors", + "model.layers.41.mlp.experts.98.gate_proj.weight": "model-00045-of-00101.safetensors", + "model.layers.41.mlp.experts.98.up_proj.weight": "model-00045-of-00101.safetensors", + "model.layers.41.mlp.experts.98.down_proj.weight": "model-00045-of-00101.safetensors", + "model.layers.41.mlp.experts.99.gate_proj.weight": "model-00045-of-00101.safetensors", + "model.layers.41.mlp.experts.99.up_proj.weight": "model-00045-of-00101.safetensors", + "model.layers.41.mlp.experts.99.down_proj.weight": "model-00045-of-00101.safetensors", + "model.layers.41.mlp.experts.100.gate_proj.weight": "model-00045-of-00101.safetensors", + "model.layers.41.mlp.experts.100.up_proj.weight": "model-00045-of-00101.safetensors", + "model.layers.41.mlp.experts.100.down_proj.weight": "model-00045-of-00101.safetensors", + "model.layers.41.mlp.experts.101.gate_proj.weight": "model-00045-of-00101.safetensors", + "model.layers.41.mlp.experts.101.up_proj.weight": "model-00045-of-00101.safetensors", + "model.layers.41.mlp.experts.101.down_proj.weight": "model-00045-of-00101.safetensors", + "model.layers.41.mlp.experts.102.gate_proj.weight": "model-00045-of-00101.safetensors", + "model.layers.41.mlp.experts.102.up_proj.weight": "model-00045-of-00101.safetensors", + "model.layers.41.mlp.experts.102.down_proj.weight": "model-00045-of-00101.safetensors", + "model.layers.41.mlp.experts.103.gate_proj.weight": "model-00045-of-00101.safetensors", + "model.layers.41.mlp.experts.103.up_proj.weight": "model-00045-of-00101.safetensors", + "model.layers.41.mlp.experts.103.down_proj.weight": "model-00045-of-00101.safetensors", + "model.layers.41.mlp.experts.104.gate_proj.weight": "model-00045-of-00101.safetensors", + "model.layers.41.mlp.experts.104.up_proj.weight": "model-00045-of-00101.safetensors", + "model.layers.41.mlp.experts.104.down_proj.weight": "model-00045-of-00101.safetensors", + "model.layers.41.mlp.experts.105.gate_proj.weight": "model-00045-of-00101.safetensors", + "model.layers.41.mlp.experts.105.up_proj.weight": "model-00045-of-00101.safetensors", + "model.layers.41.mlp.experts.105.down_proj.weight": "model-00045-of-00101.safetensors", + "model.layers.41.mlp.experts.106.gate_proj.weight": "model-00045-of-00101.safetensors", + "model.layers.41.mlp.experts.106.up_proj.weight": "model-00045-of-00101.safetensors", + "model.layers.41.mlp.experts.106.down_proj.weight": "model-00045-of-00101.safetensors", + "model.layers.41.mlp.experts.107.gate_proj.weight": "model-00045-of-00101.safetensors", + "model.layers.41.mlp.experts.107.up_proj.weight": "model-00045-of-00101.safetensors", + "model.layers.41.mlp.experts.107.down_proj.weight": "model-00045-of-00101.safetensors", + "model.layers.41.mlp.experts.108.gate_proj.weight": "model-00045-of-00101.safetensors", + "model.layers.41.mlp.experts.108.up_proj.weight": "model-00045-of-00101.safetensors", + "model.layers.41.mlp.experts.108.down_proj.weight": "model-00045-of-00101.safetensors", + "model.layers.41.mlp.experts.109.gate_proj.weight": "model-00045-of-00101.safetensors", + "model.layers.41.mlp.experts.109.up_proj.weight": "model-00045-of-00101.safetensors", + "model.layers.41.mlp.experts.109.down_proj.weight": "model-00045-of-00101.safetensors", + "model.layers.41.mlp.experts.110.gate_proj.weight": "model-00045-of-00101.safetensors", + "model.layers.41.mlp.experts.110.up_proj.weight": "model-00045-of-00101.safetensors", + "model.layers.41.mlp.experts.110.down_proj.weight": "model-00045-of-00101.safetensors", + "model.layers.41.mlp.experts.111.gate_proj.weight": "model-00045-of-00101.safetensors", + "model.layers.41.mlp.experts.111.up_proj.weight": "model-00045-of-00101.safetensors", + "model.layers.41.mlp.experts.111.down_proj.weight": "model-00045-of-00101.safetensors", + "model.layers.41.mlp.experts.112.gate_proj.weight": "model-00045-of-00101.safetensors", + "model.layers.41.mlp.experts.112.up_proj.weight": "model-00045-of-00101.safetensors", + "model.layers.41.mlp.experts.112.down_proj.weight": "model-00045-of-00101.safetensors", + "model.layers.41.mlp.experts.113.gate_proj.weight": "model-00045-of-00101.safetensors", + "model.layers.41.mlp.experts.113.up_proj.weight": "model-00045-of-00101.safetensors", + "model.layers.41.mlp.experts.113.down_proj.weight": "model-00045-of-00101.safetensors", + "model.layers.41.mlp.experts.114.gate_proj.weight": "model-00045-of-00101.safetensors", + "model.layers.41.mlp.experts.114.up_proj.weight": "model-00045-of-00101.safetensors", + "model.layers.41.mlp.experts.114.down_proj.weight": "model-00045-of-00101.safetensors", + "model.layers.41.mlp.experts.115.gate_proj.weight": "model-00045-of-00101.safetensors", + "model.layers.41.mlp.experts.115.up_proj.weight": "model-00045-of-00101.safetensors", + "model.layers.41.mlp.experts.115.down_proj.weight": "model-00045-of-00101.safetensors", + "model.layers.41.mlp.experts.116.gate_proj.weight": "model-00045-of-00101.safetensors", + "model.layers.41.mlp.experts.116.up_proj.weight": "model-00045-of-00101.safetensors", + "model.layers.41.mlp.experts.116.down_proj.weight": "model-00045-of-00101.safetensors", + "model.layers.41.mlp.experts.117.gate_proj.weight": "model-00045-of-00101.safetensors", + "model.layers.41.mlp.experts.117.up_proj.weight": "model-00045-of-00101.safetensors", + "model.layers.41.mlp.experts.117.down_proj.weight": "model-00045-of-00101.safetensors", + "model.layers.41.mlp.experts.118.gate_proj.weight": "model-00045-of-00101.safetensors", + "model.layers.41.mlp.experts.118.up_proj.weight": "model-00045-of-00101.safetensors", + "model.layers.41.mlp.experts.118.down_proj.weight": "model-00045-of-00101.safetensors", + "model.layers.41.mlp.experts.119.gate_proj.weight": "model-00045-of-00101.safetensors", + "model.layers.41.mlp.experts.119.up_proj.weight": "model-00045-of-00101.safetensors", + "model.layers.41.mlp.experts.119.down_proj.weight": "model-00045-of-00101.safetensors", + "model.layers.41.mlp.gate.weight": "model-00045-of-00101.safetensors", + "model.layers.41.mlp.gate.e_score_correction_bias": "model-00045-of-00101.safetensors", + "model.layers.41.mlp.shared_experts.gate_proj.weight": "model-00045-of-00101.safetensors", + "model.layers.41.mlp.shared_experts.up_proj.weight": "model-00045-of-00101.safetensors", + "model.layers.41.mlp.shared_experts.down_proj.weight": "model-00045-of-00101.safetensors", + "model.layers.41.input_layernorm.weight": "model-00045-of-00101.safetensors", + "model.layers.41.post_attention_layernorm.weight": "model-00045-of-00101.safetensors", + "model.layers.42.self_attn.q_proj.weight": "model-00045-of-00101.safetensors", + "model.layers.42.self_attn.q_proj.bias": "model-00045-of-00101.safetensors", + "model.layers.42.self_attn.k_proj.weight": "model-00045-of-00101.safetensors", + "model.layers.42.self_attn.k_proj.bias": "model-00045-of-00101.safetensors", + "model.layers.42.self_attn.v_proj.weight": "model-00045-of-00101.safetensors", + "model.layers.42.self_attn.v_proj.bias": "model-00045-of-00101.safetensors", + "model.layers.42.self_attn.o_proj.weight": "model-00045-of-00101.safetensors", + "model.layers.42.self_attn.q_norm.weight": "model-00045-of-00101.safetensors", + "model.layers.42.self_attn.k_norm.weight": "model-00045-of-00101.safetensors", + "model.layers.42.mlp.experts.0.gate_proj.weight": "model-00045-of-00101.safetensors", + "model.layers.42.mlp.experts.0.up_proj.weight": "model-00045-of-00101.safetensors", + "model.layers.42.mlp.experts.0.down_proj.weight": "model-00045-of-00101.safetensors", + "model.layers.42.mlp.experts.1.gate_proj.weight": "model-00045-of-00101.safetensors", + "model.layers.42.mlp.experts.1.up_proj.weight": "model-00045-of-00101.safetensors", + "model.layers.42.mlp.experts.1.down_proj.weight": "model-00045-of-00101.safetensors", + "model.layers.42.mlp.experts.2.gate_proj.weight": "model-00045-of-00101.safetensors", + "model.layers.42.mlp.experts.2.up_proj.weight": "model-00045-of-00101.safetensors", + "model.layers.42.mlp.experts.2.down_proj.weight": "model-00045-of-00101.safetensors", + "model.layers.42.mlp.experts.3.gate_proj.weight": "model-00045-of-00101.safetensors", + "model.layers.42.mlp.experts.3.up_proj.weight": "model-00045-of-00101.safetensors", + "model.layers.42.mlp.experts.3.down_proj.weight": "model-00045-of-00101.safetensors", + "model.layers.42.mlp.experts.4.gate_proj.weight": "model-00045-of-00101.safetensors", + "model.layers.42.mlp.experts.4.up_proj.weight": "model-00045-of-00101.safetensors", + "model.layers.42.mlp.experts.4.down_proj.weight": "model-00045-of-00101.safetensors", + "model.layers.42.mlp.experts.5.gate_proj.weight": "model-00045-of-00101.safetensors", + "model.layers.42.mlp.experts.5.up_proj.weight": "model-00045-of-00101.safetensors", + "model.layers.42.mlp.experts.5.down_proj.weight": "model-00045-of-00101.safetensors", + "model.layers.42.mlp.experts.6.gate_proj.weight": "model-00045-of-00101.safetensors", + "model.layers.42.mlp.experts.6.up_proj.weight": "model-00045-of-00101.safetensors", + "model.layers.42.mlp.experts.6.down_proj.weight": "model-00045-of-00101.safetensors", + "model.layers.42.mlp.experts.7.gate_proj.weight": "model-00045-of-00101.safetensors", + "model.layers.42.mlp.experts.7.up_proj.weight": "model-00045-of-00101.safetensors", + "model.layers.42.mlp.experts.7.down_proj.weight": "model-00045-of-00101.safetensors", + "model.layers.42.mlp.experts.8.gate_proj.weight": "model-00045-of-00101.safetensors", + "model.layers.42.mlp.experts.8.up_proj.weight": "model-00045-of-00101.safetensors", + "model.layers.42.mlp.experts.8.down_proj.weight": "model-00045-of-00101.safetensors", + "model.layers.42.mlp.experts.9.gate_proj.weight": "model-00045-of-00101.safetensors", + "model.layers.42.mlp.experts.9.up_proj.weight": "model-00045-of-00101.safetensors", + "model.layers.42.mlp.experts.9.down_proj.weight": "model-00045-of-00101.safetensors", + "model.layers.42.mlp.experts.10.gate_proj.weight": "model-00045-of-00101.safetensors", + "model.layers.42.mlp.experts.10.up_proj.weight": "model-00045-of-00101.safetensors", + "model.layers.42.mlp.experts.10.down_proj.weight": "model-00045-of-00101.safetensors", + "model.layers.42.mlp.experts.11.gate_proj.weight": "model-00045-of-00101.safetensors", + "model.layers.42.mlp.experts.11.up_proj.weight": "model-00045-of-00101.safetensors", + "model.layers.42.mlp.experts.11.down_proj.weight": "model-00045-of-00101.safetensors", + "model.layers.42.mlp.experts.12.gate_proj.weight": "model-00045-of-00101.safetensors", + "model.layers.42.mlp.experts.12.up_proj.weight": "model-00045-of-00101.safetensors", + "model.layers.42.mlp.experts.12.down_proj.weight": "model-00045-of-00101.safetensors", + "model.layers.42.mlp.experts.13.gate_proj.weight": "model-00045-of-00101.safetensors", + "model.layers.42.mlp.experts.13.up_proj.weight": "model-00045-of-00101.safetensors", + "model.layers.42.mlp.experts.13.down_proj.weight": "model-00045-of-00101.safetensors", + "model.layers.42.mlp.experts.14.gate_proj.weight": "model-00045-of-00101.safetensors", + "model.layers.42.mlp.experts.14.up_proj.weight": "model-00045-of-00101.safetensors", + "model.layers.42.mlp.experts.14.down_proj.weight": "model-00045-of-00101.safetensors", + "model.layers.42.mlp.experts.15.gate_proj.weight": "model-00045-of-00101.safetensors", + "model.layers.42.mlp.experts.15.up_proj.weight": "model-00045-of-00101.safetensors", + "model.layers.42.mlp.experts.15.down_proj.weight": "model-00045-of-00101.safetensors", + "model.layers.42.mlp.experts.16.gate_proj.weight": "model-00045-of-00101.safetensors", + "model.layers.42.mlp.experts.16.up_proj.weight": "model-00045-of-00101.safetensors", + "model.layers.42.mlp.experts.16.down_proj.weight": "model-00045-of-00101.safetensors", + "model.layers.42.mlp.experts.17.gate_proj.weight": "model-00045-of-00101.safetensors", + "model.layers.42.mlp.experts.17.up_proj.weight": "model-00045-of-00101.safetensors", + "model.layers.42.mlp.experts.17.down_proj.weight": "model-00045-of-00101.safetensors", + "model.layers.42.mlp.experts.18.gate_proj.weight": "model-00045-of-00101.safetensors", + "model.layers.42.mlp.experts.18.up_proj.weight": "model-00045-of-00101.safetensors", + "model.layers.42.mlp.experts.18.down_proj.weight": "model-00045-of-00101.safetensors", + "model.layers.42.mlp.experts.19.gate_proj.weight": "model-00045-of-00101.safetensors", + "model.layers.42.mlp.experts.19.up_proj.weight": "model-00045-of-00101.safetensors", + "model.layers.42.mlp.experts.19.down_proj.weight": "model-00045-of-00101.safetensors", + "model.layers.42.mlp.experts.20.gate_proj.weight": "model-00045-of-00101.safetensors", + "model.layers.42.mlp.experts.20.up_proj.weight": "model-00045-of-00101.safetensors", + "model.layers.42.mlp.experts.20.down_proj.weight": "model-00045-of-00101.safetensors", + "model.layers.42.mlp.experts.21.gate_proj.weight": "model-00045-of-00101.safetensors", + "model.layers.42.mlp.experts.21.up_proj.weight": "model-00045-of-00101.safetensors", + "model.layers.42.mlp.experts.21.down_proj.weight": "model-00045-of-00101.safetensors", + "model.layers.42.mlp.experts.22.gate_proj.weight": "model-00045-of-00101.safetensors", + "model.layers.42.mlp.experts.22.up_proj.weight": "model-00045-of-00101.safetensors", + "model.layers.42.mlp.experts.22.down_proj.weight": "model-00045-of-00101.safetensors", + "model.layers.42.mlp.experts.23.gate_proj.weight": "model-00045-of-00101.safetensors", + "model.layers.42.mlp.experts.23.up_proj.weight": "model-00045-of-00101.safetensors", + "model.layers.42.mlp.experts.23.down_proj.weight": "model-00045-of-00101.safetensors", + "model.layers.42.mlp.experts.24.gate_proj.weight": "model-00045-of-00101.safetensors", + "model.layers.42.mlp.experts.24.up_proj.weight": "model-00045-of-00101.safetensors", + "model.layers.42.mlp.experts.24.down_proj.weight": "model-00045-of-00101.safetensors", + "model.layers.42.mlp.experts.25.gate_proj.weight": "model-00045-of-00101.safetensors", + "model.layers.42.mlp.experts.25.up_proj.weight": "model-00045-of-00101.safetensors", + "model.layers.42.mlp.experts.25.down_proj.weight": "model-00045-of-00101.safetensors", + "model.layers.42.mlp.experts.26.gate_proj.weight": "model-00045-of-00101.safetensors", + "model.layers.42.mlp.experts.26.up_proj.weight": "model-00045-of-00101.safetensors", + "model.layers.42.mlp.experts.26.down_proj.weight": "model-00045-of-00101.safetensors", + "model.layers.42.mlp.experts.27.gate_proj.weight": "model-00045-of-00101.safetensors", + "model.layers.42.mlp.experts.27.up_proj.weight": "model-00045-of-00101.safetensors", + "model.layers.42.mlp.experts.27.down_proj.weight": "model-00045-of-00101.safetensors", + "model.layers.42.mlp.experts.28.gate_proj.weight": "model-00045-of-00101.safetensors", + "model.layers.42.mlp.experts.28.up_proj.weight": "model-00045-of-00101.safetensors", + "model.layers.42.mlp.experts.28.down_proj.weight": "model-00045-of-00101.safetensors", + "model.layers.42.mlp.experts.29.gate_proj.weight": "model-00045-of-00101.safetensors", + "model.layers.42.mlp.experts.29.up_proj.weight": "model-00045-of-00101.safetensors", + "model.layers.42.mlp.experts.29.down_proj.weight": "model-00045-of-00101.safetensors", + "model.layers.42.mlp.experts.30.gate_proj.weight": "model-00045-of-00101.safetensors", + "model.layers.42.mlp.experts.30.up_proj.weight": "model-00045-of-00101.safetensors", + "model.layers.42.mlp.experts.30.down_proj.weight": "model-00045-of-00101.safetensors", + "model.layers.42.mlp.experts.31.gate_proj.weight": "model-00045-of-00101.safetensors", + "model.layers.42.mlp.experts.31.up_proj.weight": "model-00045-of-00101.safetensors", + "model.layers.42.mlp.experts.31.down_proj.weight": "model-00045-of-00101.safetensors", + "model.layers.42.mlp.experts.32.gate_proj.weight": "model-00045-of-00101.safetensors", + "model.layers.42.mlp.experts.32.up_proj.weight": "model-00045-of-00101.safetensors", + "model.layers.42.mlp.experts.32.down_proj.weight": "model-00045-of-00101.safetensors", + "model.layers.42.mlp.experts.33.gate_proj.weight": "model-00045-of-00101.safetensors", + "model.layers.42.mlp.experts.33.up_proj.weight": "model-00045-of-00101.safetensors", + "model.layers.42.mlp.experts.33.down_proj.weight": "model-00045-of-00101.safetensors", + "model.layers.42.mlp.experts.34.gate_proj.weight": "model-00045-of-00101.safetensors", + "model.layers.42.mlp.experts.34.up_proj.weight": "model-00045-of-00101.safetensors", + "model.layers.42.mlp.experts.34.down_proj.weight": "model-00045-of-00101.safetensors", + "model.layers.42.mlp.experts.35.gate_proj.weight": "model-00045-of-00101.safetensors", + "model.layers.42.mlp.experts.35.up_proj.weight": "model-00045-of-00101.safetensors", + "model.layers.42.mlp.experts.35.down_proj.weight": "model-00045-of-00101.safetensors", + "model.layers.42.mlp.experts.36.gate_proj.weight": "model-00045-of-00101.safetensors", + "model.layers.42.mlp.experts.36.up_proj.weight": "model-00045-of-00101.safetensors", + "model.layers.42.mlp.experts.36.down_proj.weight": "model-00045-of-00101.safetensors", + "model.layers.42.mlp.experts.37.gate_proj.weight": "model-00045-of-00101.safetensors", + "model.layers.42.mlp.experts.37.up_proj.weight": "model-00045-of-00101.safetensors", + "model.layers.42.mlp.experts.37.down_proj.weight": "model-00045-of-00101.safetensors", + "model.layers.42.mlp.experts.38.gate_proj.weight": "model-00045-of-00101.safetensors", + "model.layers.42.mlp.experts.38.up_proj.weight": "model-00045-of-00101.safetensors", + "model.layers.42.mlp.experts.38.down_proj.weight": "model-00045-of-00101.safetensors", + "model.layers.42.mlp.experts.39.gate_proj.weight": "model-00045-of-00101.safetensors", + "model.layers.42.mlp.experts.39.up_proj.weight": "model-00045-of-00101.safetensors", + "model.layers.42.mlp.experts.39.down_proj.weight": "model-00045-of-00101.safetensors", + "model.layers.42.mlp.experts.40.gate_proj.weight": "model-00045-of-00101.safetensors", + "model.layers.42.mlp.experts.40.up_proj.weight": "model-00045-of-00101.safetensors", + "model.layers.42.mlp.experts.40.down_proj.weight": "model-00045-of-00101.safetensors", + "model.layers.42.mlp.experts.41.gate_proj.weight": "model-00045-of-00101.safetensors", + "model.layers.42.mlp.experts.41.up_proj.weight": "model-00045-of-00101.safetensors", + "model.layers.42.mlp.experts.41.down_proj.weight": "model-00045-of-00101.safetensors", + "model.layers.42.mlp.experts.42.gate_proj.weight": "model-00045-of-00101.safetensors", + "model.layers.42.mlp.experts.42.up_proj.weight": "model-00045-of-00101.safetensors", + "model.layers.42.mlp.experts.42.down_proj.weight": "model-00045-of-00101.safetensors", + "model.layers.42.mlp.experts.43.gate_proj.weight": "model-00045-of-00101.safetensors", + "model.layers.42.mlp.experts.43.up_proj.weight": "model-00045-of-00101.safetensors", + "model.layers.42.mlp.experts.43.down_proj.weight": "model-00045-of-00101.safetensors", + "model.layers.42.mlp.experts.44.gate_proj.weight": "model-00045-of-00101.safetensors", + "model.layers.42.mlp.experts.44.up_proj.weight": "model-00045-of-00101.safetensors", + "model.layers.42.mlp.experts.44.down_proj.weight": "model-00045-of-00101.safetensors", + "model.layers.42.mlp.experts.45.gate_proj.weight": "model-00045-of-00101.safetensors", + "model.layers.42.mlp.experts.45.up_proj.weight": "model-00045-of-00101.safetensors", + "model.layers.42.mlp.experts.45.down_proj.weight": "model-00045-of-00101.safetensors", + "model.layers.42.mlp.experts.46.gate_proj.weight": "model-00045-of-00101.safetensors", + "model.layers.42.mlp.experts.46.up_proj.weight": "model-00045-of-00101.safetensors", + "model.layers.42.mlp.experts.46.down_proj.weight": "model-00045-of-00101.safetensors", + "model.layers.42.mlp.experts.47.gate_proj.weight": "model-00045-of-00101.safetensors", + "model.layers.42.mlp.experts.47.up_proj.weight": "model-00045-of-00101.safetensors", + "model.layers.42.mlp.experts.47.down_proj.weight": "model-00045-of-00101.safetensors", + "model.layers.42.mlp.experts.48.gate_proj.weight": "model-00045-of-00101.safetensors", + "model.layers.42.mlp.experts.48.up_proj.weight": "model-00045-of-00101.safetensors", + "model.layers.42.mlp.experts.48.down_proj.weight": "model-00045-of-00101.safetensors", + "model.layers.42.mlp.experts.49.gate_proj.weight": "model-00045-of-00101.safetensors", + "model.layers.42.mlp.experts.49.up_proj.weight": "model-00045-of-00101.safetensors", + "model.layers.42.mlp.experts.49.down_proj.weight": "model-00045-of-00101.safetensors", + "model.layers.42.mlp.experts.50.gate_proj.weight": "model-00045-of-00101.safetensors", + "model.layers.42.mlp.experts.50.up_proj.weight": "model-00045-of-00101.safetensors", + "model.layers.42.mlp.experts.50.down_proj.weight": "model-00045-of-00101.safetensors", + "model.layers.42.mlp.experts.51.gate_proj.weight": "model-00045-of-00101.safetensors", + "model.layers.42.mlp.experts.51.up_proj.weight": "model-00045-of-00101.safetensors", + "model.layers.42.mlp.experts.51.down_proj.weight": "model-00045-of-00101.safetensors", + "model.layers.42.mlp.experts.52.gate_proj.weight": "model-00045-of-00101.safetensors", + "model.layers.42.mlp.experts.52.up_proj.weight": "model-00045-of-00101.safetensors", + "model.layers.42.mlp.experts.52.down_proj.weight": "model-00045-of-00101.safetensors", + "model.layers.42.mlp.experts.53.gate_proj.weight": "model-00045-of-00101.safetensors", + "model.layers.42.mlp.experts.53.up_proj.weight": "model-00045-of-00101.safetensors", + "model.layers.42.mlp.experts.53.down_proj.weight": "model-00045-of-00101.safetensors", + "model.layers.42.mlp.experts.54.gate_proj.weight": "model-00045-of-00101.safetensors", + "model.layers.42.mlp.experts.54.up_proj.weight": "model-00045-of-00101.safetensors", + "model.layers.42.mlp.experts.54.down_proj.weight": "model-00045-of-00101.safetensors", + "model.layers.42.mlp.experts.55.gate_proj.weight": "model-00045-of-00101.safetensors", + "model.layers.42.mlp.experts.55.up_proj.weight": "model-00045-of-00101.safetensors", + "model.layers.42.mlp.experts.55.down_proj.weight": "model-00045-of-00101.safetensors", + "model.layers.42.mlp.experts.56.gate_proj.weight": "model-00045-of-00101.safetensors", + "model.layers.42.mlp.experts.56.up_proj.weight": "model-00045-of-00101.safetensors", + "model.layers.42.mlp.experts.56.down_proj.weight": "model-00045-of-00101.safetensors", + "model.layers.42.mlp.experts.57.gate_proj.weight": "model-00045-of-00101.safetensors", + "model.layers.42.mlp.experts.57.up_proj.weight": "model-00045-of-00101.safetensors", + "model.layers.42.mlp.experts.57.down_proj.weight": "model-00045-of-00101.safetensors", + "model.layers.42.mlp.experts.58.gate_proj.weight": "model-00045-of-00101.safetensors", + "model.layers.42.mlp.experts.58.up_proj.weight": "model-00045-of-00101.safetensors", + "model.layers.42.mlp.experts.58.down_proj.weight": "model-00045-of-00101.safetensors", + "model.layers.42.mlp.experts.59.gate_proj.weight": "model-00045-of-00101.safetensors", + "model.layers.42.mlp.experts.59.up_proj.weight": "model-00045-of-00101.safetensors", + "model.layers.42.mlp.experts.59.down_proj.weight": "model-00045-of-00101.safetensors", + "model.layers.42.mlp.experts.60.gate_proj.weight": "model-00045-of-00101.safetensors", + "model.layers.42.mlp.experts.60.up_proj.weight": "model-00045-of-00101.safetensors", + "model.layers.42.mlp.experts.60.down_proj.weight": "model-00045-of-00101.safetensors", + "model.layers.42.mlp.experts.61.gate_proj.weight": "model-00045-of-00101.safetensors", + "model.layers.42.mlp.experts.61.up_proj.weight": "model-00045-of-00101.safetensors", + "model.layers.42.mlp.experts.61.down_proj.weight": "model-00045-of-00101.safetensors", + "model.layers.42.mlp.experts.62.gate_proj.weight": "model-00045-of-00101.safetensors", + "model.layers.42.mlp.experts.62.up_proj.weight": "model-00045-of-00101.safetensors", + "model.layers.42.mlp.experts.62.down_proj.weight": "model-00045-of-00101.safetensors", + "model.layers.42.mlp.experts.63.gate_proj.weight": "model-00045-of-00101.safetensors", + "model.layers.42.mlp.experts.63.up_proj.weight": "model-00045-of-00101.safetensors", + "model.layers.42.mlp.experts.63.down_proj.weight": "model-00045-of-00101.safetensors", + "model.layers.42.mlp.experts.64.gate_proj.weight": "model-00045-of-00101.safetensors", + "model.layers.42.mlp.experts.64.up_proj.weight": "model-00045-of-00101.safetensors", + "model.layers.42.mlp.experts.64.down_proj.weight": "model-00045-of-00101.safetensors", + "model.layers.42.mlp.experts.65.gate_proj.weight": "model-00045-of-00101.safetensors", + "model.layers.42.mlp.experts.65.up_proj.weight": "model-00045-of-00101.safetensors", + "model.layers.42.mlp.experts.65.down_proj.weight": "model-00045-of-00101.safetensors", + "model.layers.42.mlp.experts.66.gate_proj.weight": "model-00045-of-00101.safetensors", + "model.layers.42.mlp.experts.66.up_proj.weight": "model-00045-of-00101.safetensors", + "model.layers.42.mlp.experts.66.down_proj.weight": "model-00045-of-00101.safetensors", + "model.layers.42.mlp.experts.67.gate_proj.weight": "model-00045-of-00101.safetensors", + "model.layers.42.mlp.experts.67.up_proj.weight": "model-00045-of-00101.safetensors", + "model.layers.42.mlp.experts.67.down_proj.weight": "model-00045-of-00101.safetensors", + "model.layers.42.mlp.experts.68.gate_proj.weight": "model-00045-of-00101.safetensors", + "model.layers.42.mlp.experts.68.up_proj.weight": "model-00045-of-00101.safetensors", + "model.layers.42.mlp.experts.68.down_proj.weight": "model-00045-of-00101.safetensors", + "model.layers.42.mlp.experts.69.gate_proj.weight": "model-00045-of-00101.safetensors", + "model.layers.42.mlp.experts.69.up_proj.weight": "model-00045-of-00101.safetensors", + "model.layers.42.mlp.experts.69.down_proj.weight": "model-00045-of-00101.safetensors", + "model.layers.42.mlp.experts.70.gate_proj.weight": "model-00045-of-00101.safetensors", + "model.layers.42.mlp.experts.70.up_proj.weight": "model-00045-of-00101.safetensors", + "model.layers.42.mlp.experts.70.down_proj.weight": "model-00045-of-00101.safetensors", + "model.layers.42.mlp.experts.71.gate_proj.weight": "model-00045-of-00101.safetensors", + "model.layers.42.mlp.experts.71.up_proj.weight": "model-00045-of-00101.safetensors", + "model.layers.42.mlp.experts.71.down_proj.weight": "model-00045-of-00101.safetensors", + "model.layers.42.mlp.experts.72.gate_proj.weight": "model-00045-of-00101.safetensors", + "model.layers.42.mlp.experts.72.up_proj.weight": "model-00045-of-00101.safetensors", + "model.layers.42.mlp.experts.72.down_proj.weight": "model-00045-of-00101.safetensors", + "model.layers.42.mlp.experts.73.gate_proj.weight": "model-00045-of-00101.safetensors", + "model.layers.42.mlp.experts.73.up_proj.weight": "model-00045-of-00101.safetensors", + "model.layers.42.mlp.experts.73.down_proj.weight": "model-00045-of-00101.safetensors", + "model.layers.42.mlp.experts.74.gate_proj.weight": "model-00045-of-00101.safetensors", + "model.layers.42.mlp.experts.74.up_proj.weight": "model-00045-of-00101.safetensors", + "model.layers.42.mlp.experts.74.down_proj.weight": "model-00045-of-00101.safetensors", + "model.layers.42.mlp.experts.75.gate_proj.weight": "model-00045-of-00101.safetensors", + "model.layers.42.mlp.experts.75.up_proj.weight": "model-00045-of-00101.safetensors", + "model.layers.42.mlp.experts.75.down_proj.weight": "model-00045-of-00101.safetensors", + "model.layers.42.mlp.experts.76.gate_proj.weight": "model-00045-of-00101.safetensors", + "model.layers.42.mlp.experts.76.up_proj.weight": "model-00045-of-00101.safetensors", + "model.layers.42.mlp.experts.76.down_proj.weight": "model-00045-of-00101.safetensors", + "model.layers.42.mlp.experts.77.gate_proj.weight": "model-00045-of-00101.safetensors", + "model.layers.42.mlp.experts.77.up_proj.weight": "model-00045-of-00101.safetensors", + "model.layers.42.mlp.experts.77.down_proj.weight": "model-00045-of-00101.safetensors", + "model.layers.42.mlp.experts.78.gate_proj.weight": "model-00045-of-00101.safetensors", + "model.layers.42.mlp.experts.78.up_proj.weight": "model-00045-of-00101.safetensors", + "model.layers.42.mlp.experts.78.down_proj.weight": "model-00045-of-00101.safetensors", + "model.layers.42.mlp.experts.79.gate_proj.weight": "model-00045-of-00101.safetensors", + "model.layers.42.mlp.experts.79.up_proj.weight": "model-00045-of-00101.safetensors", + "model.layers.42.mlp.experts.79.down_proj.weight": "model-00045-of-00101.safetensors", + "model.layers.42.mlp.experts.80.gate_proj.weight": "model-00045-of-00101.safetensors", + "model.layers.42.mlp.experts.80.up_proj.weight": "model-00045-of-00101.safetensors", + "model.layers.42.mlp.experts.80.down_proj.weight": "model-00045-of-00101.safetensors", + "model.layers.42.mlp.experts.81.gate_proj.weight": "model-00046-of-00101.safetensors", + "model.layers.42.mlp.experts.81.up_proj.weight": "model-00046-of-00101.safetensors", + "model.layers.42.mlp.experts.81.down_proj.weight": "model-00046-of-00101.safetensors", + "model.layers.42.mlp.experts.82.gate_proj.weight": "model-00046-of-00101.safetensors", + "model.layers.42.mlp.experts.82.up_proj.weight": "model-00046-of-00101.safetensors", + "model.layers.42.mlp.experts.82.down_proj.weight": "model-00046-of-00101.safetensors", + "model.layers.42.mlp.experts.83.gate_proj.weight": "model-00046-of-00101.safetensors", + "model.layers.42.mlp.experts.83.up_proj.weight": "model-00046-of-00101.safetensors", + "model.layers.42.mlp.experts.83.down_proj.weight": "model-00046-of-00101.safetensors", + "model.layers.42.mlp.experts.84.gate_proj.weight": "model-00046-of-00101.safetensors", + "model.layers.42.mlp.experts.84.up_proj.weight": "model-00046-of-00101.safetensors", + "model.layers.42.mlp.experts.84.down_proj.weight": "model-00046-of-00101.safetensors", + "model.layers.42.mlp.experts.85.gate_proj.weight": "model-00046-of-00101.safetensors", + "model.layers.42.mlp.experts.85.up_proj.weight": "model-00046-of-00101.safetensors", + "model.layers.42.mlp.experts.85.down_proj.weight": "model-00046-of-00101.safetensors", + "model.layers.42.mlp.experts.86.gate_proj.weight": "model-00046-of-00101.safetensors", + "model.layers.42.mlp.experts.86.up_proj.weight": "model-00046-of-00101.safetensors", + "model.layers.42.mlp.experts.86.down_proj.weight": "model-00046-of-00101.safetensors", + "model.layers.42.mlp.experts.87.gate_proj.weight": "model-00046-of-00101.safetensors", + "model.layers.42.mlp.experts.87.up_proj.weight": "model-00046-of-00101.safetensors", + "model.layers.42.mlp.experts.87.down_proj.weight": "model-00046-of-00101.safetensors", + "model.layers.42.mlp.experts.88.gate_proj.weight": "model-00046-of-00101.safetensors", + "model.layers.42.mlp.experts.88.up_proj.weight": "model-00046-of-00101.safetensors", + "model.layers.42.mlp.experts.88.down_proj.weight": "model-00046-of-00101.safetensors", + "model.layers.42.mlp.experts.89.gate_proj.weight": "model-00046-of-00101.safetensors", + "model.layers.42.mlp.experts.89.up_proj.weight": "model-00046-of-00101.safetensors", + "model.layers.42.mlp.experts.89.down_proj.weight": "model-00046-of-00101.safetensors", + "model.layers.42.mlp.experts.90.gate_proj.weight": "model-00046-of-00101.safetensors", + "model.layers.42.mlp.experts.90.up_proj.weight": "model-00046-of-00101.safetensors", + "model.layers.42.mlp.experts.90.down_proj.weight": "model-00046-of-00101.safetensors", + "model.layers.42.mlp.experts.91.gate_proj.weight": "model-00046-of-00101.safetensors", + "model.layers.42.mlp.experts.91.up_proj.weight": "model-00046-of-00101.safetensors", + "model.layers.42.mlp.experts.91.down_proj.weight": "model-00046-of-00101.safetensors", + "model.layers.42.mlp.experts.92.gate_proj.weight": "model-00046-of-00101.safetensors", + "model.layers.42.mlp.experts.92.up_proj.weight": "model-00046-of-00101.safetensors", + "model.layers.42.mlp.experts.92.down_proj.weight": "model-00046-of-00101.safetensors", + "model.layers.42.mlp.experts.93.gate_proj.weight": "model-00046-of-00101.safetensors", + "model.layers.42.mlp.experts.93.up_proj.weight": "model-00046-of-00101.safetensors", + "model.layers.42.mlp.experts.93.down_proj.weight": "model-00046-of-00101.safetensors", + "model.layers.42.mlp.experts.94.gate_proj.weight": "model-00046-of-00101.safetensors", + "model.layers.42.mlp.experts.94.up_proj.weight": "model-00046-of-00101.safetensors", + "model.layers.42.mlp.experts.94.down_proj.weight": "model-00046-of-00101.safetensors", + "model.layers.42.mlp.experts.95.gate_proj.weight": "model-00046-of-00101.safetensors", + "model.layers.42.mlp.experts.95.up_proj.weight": "model-00046-of-00101.safetensors", + "model.layers.42.mlp.experts.95.down_proj.weight": "model-00046-of-00101.safetensors", + "model.layers.42.mlp.experts.96.gate_proj.weight": "model-00046-of-00101.safetensors", + "model.layers.42.mlp.experts.96.up_proj.weight": "model-00046-of-00101.safetensors", + "model.layers.42.mlp.experts.96.down_proj.weight": "model-00046-of-00101.safetensors", + "model.layers.42.mlp.experts.97.gate_proj.weight": "model-00046-of-00101.safetensors", + "model.layers.42.mlp.experts.97.up_proj.weight": "model-00046-of-00101.safetensors", + "model.layers.42.mlp.experts.97.down_proj.weight": "model-00046-of-00101.safetensors", + "model.layers.42.mlp.experts.98.gate_proj.weight": "model-00046-of-00101.safetensors", + "model.layers.42.mlp.experts.98.up_proj.weight": "model-00046-of-00101.safetensors", + "model.layers.42.mlp.experts.98.down_proj.weight": "model-00046-of-00101.safetensors", + "model.layers.42.mlp.experts.99.gate_proj.weight": "model-00046-of-00101.safetensors", + "model.layers.42.mlp.experts.99.up_proj.weight": "model-00046-of-00101.safetensors", + "model.layers.42.mlp.experts.99.down_proj.weight": "model-00046-of-00101.safetensors", + "model.layers.42.mlp.experts.100.gate_proj.weight": "model-00046-of-00101.safetensors", + "model.layers.42.mlp.experts.100.up_proj.weight": "model-00046-of-00101.safetensors", + "model.layers.42.mlp.experts.100.down_proj.weight": "model-00046-of-00101.safetensors", + "model.layers.42.mlp.experts.101.gate_proj.weight": "model-00046-of-00101.safetensors", + "model.layers.42.mlp.experts.101.up_proj.weight": "model-00046-of-00101.safetensors", + "model.layers.42.mlp.experts.101.down_proj.weight": "model-00046-of-00101.safetensors", + "model.layers.42.mlp.experts.102.gate_proj.weight": "model-00046-of-00101.safetensors", + "model.layers.42.mlp.experts.102.up_proj.weight": "model-00046-of-00101.safetensors", + "model.layers.42.mlp.experts.102.down_proj.weight": "model-00046-of-00101.safetensors", + "model.layers.42.mlp.experts.103.gate_proj.weight": "model-00046-of-00101.safetensors", + "model.layers.42.mlp.experts.103.up_proj.weight": "model-00046-of-00101.safetensors", + "model.layers.42.mlp.experts.103.down_proj.weight": "model-00046-of-00101.safetensors", + "model.layers.42.mlp.experts.104.gate_proj.weight": "model-00046-of-00101.safetensors", + "model.layers.42.mlp.experts.104.up_proj.weight": "model-00046-of-00101.safetensors", + "model.layers.42.mlp.experts.104.down_proj.weight": "model-00046-of-00101.safetensors", + "model.layers.42.mlp.experts.105.gate_proj.weight": "model-00046-of-00101.safetensors", + "model.layers.42.mlp.experts.105.up_proj.weight": "model-00046-of-00101.safetensors", + "model.layers.42.mlp.experts.105.down_proj.weight": "model-00046-of-00101.safetensors", + "model.layers.42.mlp.experts.106.gate_proj.weight": "model-00046-of-00101.safetensors", + "model.layers.42.mlp.experts.106.up_proj.weight": "model-00046-of-00101.safetensors", + "model.layers.42.mlp.experts.106.down_proj.weight": "model-00046-of-00101.safetensors", + "model.layers.42.mlp.experts.107.gate_proj.weight": "model-00046-of-00101.safetensors", + "model.layers.42.mlp.experts.107.up_proj.weight": "model-00046-of-00101.safetensors", + "model.layers.42.mlp.experts.107.down_proj.weight": "model-00046-of-00101.safetensors", + "model.layers.42.mlp.experts.108.gate_proj.weight": "model-00046-of-00101.safetensors", + "model.layers.42.mlp.experts.108.up_proj.weight": "model-00046-of-00101.safetensors", + "model.layers.42.mlp.experts.108.down_proj.weight": "model-00046-of-00101.safetensors", + "model.layers.42.mlp.experts.109.gate_proj.weight": "model-00046-of-00101.safetensors", + "model.layers.42.mlp.experts.109.up_proj.weight": "model-00046-of-00101.safetensors", + "model.layers.42.mlp.experts.109.down_proj.weight": "model-00046-of-00101.safetensors", + "model.layers.42.mlp.experts.110.gate_proj.weight": "model-00046-of-00101.safetensors", + "model.layers.42.mlp.experts.110.up_proj.weight": "model-00046-of-00101.safetensors", + "model.layers.42.mlp.experts.110.down_proj.weight": "model-00046-of-00101.safetensors", + "model.layers.42.mlp.experts.111.gate_proj.weight": "model-00046-of-00101.safetensors", + "model.layers.42.mlp.experts.111.up_proj.weight": "model-00046-of-00101.safetensors", + "model.layers.42.mlp.experts.111.down_proj.weight": "model-00046-of-00101.safetensors", + "model.layers.42.mlp.experts.112.gate_proj.weight": "model-00046-of-00101.safetensors", + "model.layers.42.mlp.experts.112.up_proj.weight": "model-00046-of-00101.safetensors", + "model.layers.42.mlp.experts.112.down_proj.weight": "model-00046-of-00101.safetensors", + "model.layers.42.mlp.experts.113.gate_proj.weight": "model-00046-of-00101.safetensors", + "model.layers.42.mlp.experts.113.up_proj.weight": "model-00046-of-00101.safetensors", + "model.layers.42.mlp.experts.113.down_proj.weight": "model-00046-of-00101.safetensors", + "model.layers.42.mlp.experts.114.gate_proj.weight": "model-00046-of-00101.safetensors", + "model.layers.42.mlp.experts.114.up_proj.weight": "model-00046-of-00101.safetensors", + "model.layers.42.mlp.experts.114.down_proj.weight": "model-00046-of-00101.safetensors", + "model.layers.42.mlp.experts.115.gate_proj.weight": "model-00046-of-00101.safetensors", + "model.layers.42.mlp.experts.115.up_proj.weight": "model-00046-of-00101.safetensors", + "model.layers.42.mlp.experts.115.down_proj.weight": "model-00046-of-00101.safetensors", + "model.layers.42.mlp.experts.116.gate_proj.weight": "model-00046-of-00101.safetensors", + "model.layers.42.mlp.experts.116.up_proj.weight": "model-00046-of-00101.safetensors", + "model.layers.42.mlp.experts.116.down_proj.weight": "model-00046-of-00101.safetensors", + "model.layers.42.mlp.experts.117.gate_proj.weight": "model-00046-of-00101.safetensors", + "model.layers.42.mlp.experts.117.up_proj.weight": "model-00046-of-00101.safetensors", + "model.layers.42.mlp.experts.117.down_proj.weight": "model-00046-of-00101.safetensors", + "model.layers.42.mlp.experts.118.gate_proj.weight": "model-00046-of-00101.safetensors", + "model.layers.42.mlp.experts.118.up_proj.weight": "model-00046-of-00101.safetensors", + "model.layers.42.mlp.experts.118.down_proj.weight": "model-00046-of-00101.safetensors", + "model.layers.42.mlp.experts.119.gate_proj.weight": "model-00046-of-00101.safetensors", + "model.layers.42.mlp.experts.119.up_proj.weight": "model-00046-of-00101.safetensors", + "model.layers.42.mlp.experts.119.down_proj.weight": "model-00046-of-00101.safetensors", + "model.layers.42.mlp.gate.weight": "model-00046-of-00101.safetensors", + "model.layers.42.mlp.gate.e_score_correction_bias": "model-00046-of-00101.safetensors", + "model.layers.42.mlp.shared_experts.gate_proj.weight": "model-00046-of-00101.safetensors", + "model.layers.42.mlp.shared_experts.up_proj.weight": "model-00046-of-00101.safetensors", + "model.layers.42.mlp.shared_experts.down_proj.weight": "model-00046-of-00101.safetensors", + "model.layers.42.input_layernorm.weight": "model-00046-of-00101.safetensors", + "model.layers.42.post_attention_layernorm.weight": "model-00046-of-00101.safetensors", + "model.layers.43.self_attn.q_proj.weight": "model-00046-of-00101.safetensors", + "model.layers.43.self_attn.q_proj.bias": "model-00046-of-00101.safetensors", + "model.layers.43.self_attn.k_proj.weight": "model-00046-of-00101.safetensors", + "model.layers.43.self_attn.k_proj.bias": "model-00046-of-00101.safetensors", + "model.layers.43.self_attn.v_proj.weight": "model-00046-of-00101.safetensors", + "model.layers.43.self_attn.v_proj.bias": "model-00046-of-00101.safetensors", + "model.layers.43.self_attn.o_proj.weight": "model-00046-of-00101.safetensors", + "model.layers.43.self_attn.q_norm.weight": "model-00046-of-00101.safetensors", + "model.layers.43.self_attn.k_norm.weight": "model-00046-of-00101.safetensors", + "model.layers.43.mlp.experts.0.gate_proj.weight": "model-00046-of-00101.safetensors", + "model.layers.43.mlp.experts.0.up_proj.weight": "model-00046-of-00101.safetensors", + "model.layers.43.mlp.experts.0.down_proj.weight": "model-00046-of-00101.safetensors", + "model.layers.43.mlp.experts.1.gate_proj.weight": "model-00046-of-00101.safetensors", + "model.layers.43.mlp.experts.1.up_proj.weight": "model-00046-of-00101.safetensors", + "model.layers.43.mlp.experts.1.down_proj.weight": "model-00046-of-00101.safetensors", + "model.layers.43.mlp.experts.2.gate_proj.weight": "model-00046-of-00101.safetensors", + "model.layers.43.mlp.experts.2.up_proj.weight": "model-00046-of-00101.safetensors", + "model.layers.43.mlp.experts.2.down_proj.weight": "model-00046-of-00101.safetensors", + "model.layers.43.mlp.experts.3.gate_proj.weight": "model-00046-of-00101.safetensors", + "model.layers.43.mlp.experts.3.up_proj.weight": "model-00046-of-00101.safetensors", + "model.layers.43.mlp.experts.3.down_proj.weight": "model-00046-of-00101.safetensors", + "model.layers.43.mlp.experts.4.gate_proj.weight": "model-00046-of-00101.safetensors", + "model.layers.43.mlp.experts.4.up_proj.weight": "model-00046-of-00101.safetensors", + "model.layers.43.mlp.experts.4.down_proj.weight": "model-00046-of-00101.safetensors", + "model.layers.43.mlp.experts.5.gate_proj.weight": "model-00046-of-00101.safetensors", + "model.layers.43.mlp.experts.5.up_proj.weight": "model-00046-of-00101.safetensors", + "model.layers.43.mlp.experts.5.down_proj.weight": "model-00046-of-00101.safetensors", + "model.layers.43.mlp.experts.6.gate_proj.weight": "model-00046-of-00101.safetensors", + "model.layers.43.mlp.experts.6.up_proj.weight": "model-00046-of-00101.safetensors", + "model.layers.43.mlp.experts.6.down_proj.weight": "model-00046-of-00101.safetensors", + "model.layers.43.mlp.experts.7.gate_proj.weight": "model-00046-of-00101.safetensors", + "model.layers.43.mlp.experts.7.up_proj.weight": "model-00046-of-00101.safetensors", + "model.layers.43.mlp.experts.7.down_proj.weight": "model-00046-of-00101.safetensors", + "model.layers.43.mlp.experts.8.gate_proj.weight": "model-00046-of-00101.safetensors", + "model.layers.43.mlp.experts.8.up_proj.weight": "model-00046-of-00101.safetensors", + "model.layers.43.mlp.experts.8.down_proj.weight": "model-00046-of-00101.safetensors", + "model.layers.43.mlp.experts.9.gate_proj.weight": "model-00046-of-00101.safetensors", + "model.layers.43.mlp.experts.9.up_proj.weight": "model-00046-of-00101.safetensors", + "model.layers.43.mlp.experts.9.down_proj.weight": "model-00046-of-00101.safetensors", + "model.layers.43.mlp.experts.10.gate_proj.weight": "model-00046-of-00101.safetensors", + "model.layers.43.mlp.experts.10.up_proj.weight": "model-00046-of-00101.safetensors", + "model.layers.43.mlp.experts.10.down_proj.weight": "model-00046-of-00101.safetensors", + "model.layers.43.mlp.experts.11.gate_proj.weight": "model-00046-of-00101.safetensors", + "model.layers.43.mlp.experts.11.up_proj.weight": "model-00046-of-00101.safetensors", + "model.layers.43.mlp.experts.11.down_proj.weight": "model-00046-of-00101.safetensors", + "model.layers.43.mlp.experts.12.gate_proj.weight": "model-00046-of-00101.safetensors", + "model.layers.43.mlp.experts.12.up_proj.weight": "model-00046-of-00101.safetensors", + "model.layers.43.mlp.experts.12.down_proj.weight": "model-00046-of-00101.safetensors", + "model.layers.43.mlp.experts.13.gate_proj.weight": "model-00046-of-00101.safetensors", + "model.layers.43.mlp.experts.13.up_proj.weight": "model-00046-of-00101.safetensors", + "model.layers.43.mlp.experts.13.down_proj.weight": "model-00046-of-00101.safetensors", + "model.layers.43.mlp.experts.14.gate_proj.weight": "model-00046-of-00101.safetensors", + "model.layers.43.mlp.experts.14.up_proj.weight": "model-00046-of-00101.safetensors", + "model.layers.43.mlp.experts.14.down_proj.weight": "model-00046-of-00101.safetensors", + "model.layers.43.mlp.experts.15.gate_proj.weight": "model-00046-of-00101.safetensors", + "model.layers.43.mlp.experts.15.up_proj.weight": "model-00046-of-00101.safetensors", + "model.layers.43.mlp.experts.15.down_proj.weight": "model-00046-of-00101.safetensors", + "model.layers.43.mlp.experts.16.gate_proj.weight": "model-00046-of-00101.safetensors", + "model.layers.43.mlp.experts.16.up_proj.weight": "model-00046-of-00101.safetensors", + "model.layers.43.mlp.experts.16.down_proj.weight": "model-00046-of-00101.safetensors", + "model.layers.43.mlp.experts.17.gate_proj.weight": "model-00046-of-00101.safetensors", + "model.layers.43.mlp.experts.17.up_proj.weight": "model-00046-of-00101.safetensors", + "model.layers.43.mlp.experts.17.down_proj.weight": "model-00046-of-00101.safetensors", + "model.layers.43.mlp.experts.18.gate_proj.weight": "model-00046-of-00101.safetensors", + "model.layers.43.mlp.experts.18.up_proj.weight": "model-00046-of-00101.safetensors", + "model.layers.43.mlp.experts.18.down_proj.weight": "model-00046-of-00101.safetensors", + "model.layers.43.mlp.experts.19.gate_proj.weight": "model-00046-of-00101.safetensors", + "model.layers.43.mlp.experts.19.up_proj.weight": "model-00046-of-00101.safetensors", + "model.layers.43.mlp.experts.19.down_proj.weight": "model-00046-of-00101.safetensors", + "model.layers.43.mlp.experts.20.gate_proj.weight": "model-00046-of-00101.safetensors", + "model.layers.43.mlp.experts.20.up_proj.weight": "model-00046-of-00101.safetensors", + "model.layers.43.mlp.experts.20.down_proj.weight": "model-00046-of-00101.safetensors", + "model.layers.43.mlp.experts.21.gate_proj.weight": "model-00046-of-00101.safetensors", + "model.layers.43.mlp.experts.21.up_proj.weight": "model-00046-of-00101.safetensors", + "model.layers.43.mlp.experts.21.down_proj.weight": "model-00046-of-00101.safetensors", + "model.layers.43.mlp.experts.22.gate_proj.weight": "model-00046-of-00101.safetensors", + "model.layers.43.mlp.experts.22.up_proj.weight": "model-00046-of-00101.safetensors", + "model.layers.43.mlp.experts.22.down_proj.weight": "model-00046-of-00101.safetensors", + "model.layers.43.mlp.experts.23.gate_proj.weight": "model-00046-of-00101.safetensors", + "model.layers.43.mlp.experts.23.up_proj.weight": "model-00046-of-00101.safetensors", + "model.layers.43.mlp.experts.23.down_proj.weight": "model-00046-of-00101.safetensors", + "model.layers.43.mlp.experts.24.gate_proj.weight": "model-00046-of-00101.safetensors", + "model.layers.43.mlp.experts.24.up_proj.weight": "model-00046-of-00101.safetensors", + "model.layers.43.mlp.experts.24.down_proj.weight": "model-00046-of-00101.safetensors", + "model.layers.43.mlp.experts.25.gate_proj.weight": "model-00046-of-00101.safetensors", + "model.layers.43.mlp.experts.25.up_proj.weight": "model-00046-of-00101.safetensors", + "model.layers.43.mlp.experts.25.down_proj.weight": "model-00046-of-00101.safetensors", + "model.layers.43.mlp.experts.26.gate_proj.weight": "model-00046-of-00101.safetensors", + "model.layers.43.mlp.experts.26.up_proj.weight": "model-00046-of-00101.safetensors", + "model.layers.43.mlp.experts.26.down_proj.weight": "model-00046-of-00101.safetensors", + "model.layers.43.mlp.experts.27.gate_proj.weight": "model-00046-of-00101.safetensors", + "model.layers.43.mlp.experts.27.up_proj.weight": "model-00046-of-00101.safetensors", + "model.layers.43.mlp.experts.27.down_proj.weight": "model-00046-of-00101.safetensors", + "model.layers.43.mlp.experts.28.gate_proj.weight": "model-00046-of-00101.safetensors", + "model.layers.43.mlp.experts.28.up_proj.weight": "model-00046-of-00101.safetensors", + "model.layers.43.mlp.experts.28.down_proj.weight": "model-00046-of-00101.safetensors", + "model.layers.43.mlp.experts.29.gate_proj.weight": "model-00046-of-00101.safetensors", + "model.layers.43.mlp.experts.29.up_proj.weight": "model-00046-of-00101.safetensors", + "model.layers.43.mlp.experts.29.down_proj.weight": "model-00046-of-00101.safetensors", + "model.layers.43.mlp.experts.30.gate_proj.weight": "model-00046-of-00101.safetensors", + "model.layers.43.mlp.experts.30.up_proj.weight": "model-00046-of-00101.safetensors", + "model.layers.43.mlp.experts.30.down_proj.weight": "model-00046-of-00101.safetensors", + "model.layers.43.mlp.experts.31.gate_proj.weight": "model-00046-of-00101.safetensors", + "model.layers.43.mlp.experts.31.up_proj.weight": "model-00046-of-00101.safetensors", + "model.layers.43.mlp.experts.31.down_proj.weight": "model-00046-of-00101.safetensors", + "model.layers.43.mlp.experts.32.gate_proj.weight": "model-00046-of-00101.safetensors", + "model.layers.43.mlp.experts.32.up_proj.weight": "model-00046-of-00101.safetensors", + "model.layers.43.mlp.experts.32.down_proj.weight": "model-00046-of-00101.safetensors", + "model.layers.43.mlp.experts.33.gate_proj.weight": "model-00046-of-00101.safetensors", + "model.layers.43.mlp.experts.33.up_proj.weight": "model-00046-of-00101.safetensors", + "model.layers.43.mlp.experts.33.down_proj.weight": "model-00046-of-00101.safetensors", + "model.layers.43.mlp.experts.34.gate_proj.weight": "model-00046-of-00101.safetensors", + "model.layers.43.mlp.experts.34.up_proj.weight": "model-00046-of-00101.safetensors", + "model.layers.43.mlp.experts.34.down_proj.weight": "model-00046-of-00101.safetensors", + "model.layers.43.mlp.experts.35.gate_proj.weight": "model-00046-of-00101.safetensors", + "model.layers.43.mlp.experts.35.up_proj.weight": "model-00046-of-00101.safetensors", + "model.layers.43.mlp.experts.35.down_proj.weight": "model-00046-of-00101.safetensors", + "model.layers.43.mlp.experts.36.gate_proj.weight": "model-00046-of-00101.safetensors", + "model.layers.43.mlp.experts.36.up_proj.weight": "model-00046-of-00101.safetensors", + "model.layers.43.mlp.experts.36.down_proj.weight": "model-00046-of-00101.safetensors", + "model.layers.43.mlp.experts.37.gate_proj.weight": "model-00046-of-00101.safetensors", + "model.layers.43.mlp.experts.37.up_proj.weight": "model-00046-of-00101.safetensors", + "model.layers.43.mlp.experts.37.down_proj.weight": "model-00046-of-00101.safetensors", + "model.layers.43.mlp.experts.38.gate_proj.weight": "model-00046-of-00101.safetensors", + "model.layers.43.mlp.experts.38.up_proj.weight": "model-00046-of-00101.safetensors", + "model.layers.43.mlp.experts.38.down_proj.weight": "model-00046-of-00101.safetensors", + "model.layers.43.mlp.experts.39.gate_proj.weight": "model-00046-of-00101.safetensors", + "model.layers.43.mlp.experts.39.up_proj.weight": "model-00046-of-00101.safetensors", + "model.layers.43.mlp.experts.39.down_proj.weight": "model-00046-of-00101.safetensors", + "model.layers.43.mlp.experts.40.gate_proj.weight": "model-00046-of-00101.safetensors", + "model.layers.43.mlp.experts.40.up_proj.weight": "model-00046-of-00101.safetensors", + "model.layers.43.mlp.experts.40.down_proj.weight": "model-00046-of-00101.safetensors", + "model.layers.43.mlp.experts.41.gate_proj.weight": "model-00046-of-00101.safetensors", + "model.layers.43.mlp.experts.41.up_proj.weight": "model-00046-of-00101.safetensors", + "model.layers.43.mlp.experts.41.down_proj.weight": "model-00046-of-00101.safetensors", + "model.layers.43.mlp.experts.42.gate_proj.weight": "model-00046-of-00101.safetensors", + "model.layers.43.mlp.experts.42.up_proj.weight": "model-00046-of-00101.safetensors", + "model.layers.43.mlp.experts.42.down_proj.weight": "model-00046-of-00101.safetensors", + "model.layers.43.mlp.experts.43.gate_proj.weight": "model-00046-of-00101.safetensors", + "model.layers.43.mlp.experts.43.up_proj.weight": "model-00046-of-00101.safetensors", + "model.layers.43.mlp.experts.43.down_proj.weight": "model-00046-of-00101.safetensors", + "model.layers.43.mlp.experts.44.gate_proj.weight": "model-00046-of-00101.safetensors", + "model.layers.43.mlp.experts.44.up_proj.weight": "model-00046-of-00101.safetensors", + "model.layers.43.mlp.experts.44.down_proj.weight": "model-00046-of-00101.safetensors", + "model.layers.43.mlp.experts.45.gate_proj.weight": "model-00046-of-00101.safetensors", + "model.layers.43.mlp.experts.45.up_proj.weight": "model-00046-of-00101.safetensors", + "model.layers.43.mlp.experts.45.down_proj.weight": "model-00046-of-00101.safetensors", + "model.layers.43.mlp.experts.46.gate_proj.weight": "model-00046-of-00101.safetensors", + "model.layers.43.mlp.experts.46.up_proj.weight": "model-00046-of-00101.safetensors", + "model.layers.43.mlp.experts.46.down_proj.weight": "model-00046-of-00101.safetensors", + "model.layers.43.mlp.experts.47.gate_proj.weight": "model-00046-of-00101.safetensors", + "model.layers.43.mlp.experts.47.up_proj.weight": "model-00046-of-00101.safetensors", + "model.layers.43.mlp.experts.47.down_proj.weight": "model-00046-of-00101.safetensors", + "model.layers.43.mlp.experts.48.gate_proj.weight": "model-00046-of-00101.safetensors", + "model.layers.43.mlp.experts.48.up_proj.weight": "model-00046-of-00101.safetensors", + "model.layers.43.mlp.experts.48.down_proj.weight": "model-00046-of-00101.safetensors", + "model.layers.43.mlp.experts.49.gate_proj.weight": "model-00046-of-00101.safetensors", + "model.layers.43.mlp.experts.49.up_proj.weight": "model-00046-of-00101.safetensors", + "model.layers.43.mlp.experts.49.down_proj.weight": "model-00046-of-00101.safetensors", + "model.layers.43.mlp.experts.50.gate_proj.weight": "model-00046-of-00101.safetensors", + "model.layers.43.mlp.experts.50.up_proj.weight": "model-00046-of-00101.safetensors", + "model.layers.43.mlp.experts.50.down_proj.weight": "model-00046-of-00101.safetensors", + "model.layers.43.mlp.experts.51.gate_proj.weight": "model-00046-of-00101.safetensors", + "model.layers.43.mlp.experts.51.up_proj.weight": "model-00046-of-00101.safetensors", + "model.layers.43.mlp.experts.51.down_proj.weight": "model-00046-of-00101.safetensors", + "model.layers.43.mlp.experts.52.gate_proj.weight": "model-00046-of-00101.safetensors", + "model.layers.43.mlp.experts.52.up_proj.weight": "model-00046-of-00101.safetensors", + "model.layers.43.mlp.experts.52.down_proj.weight": "model-00046-of-00101.safetensors", + "model.layers.43.mlp.experts.53.gate_proj.weight": "model-00046-of-00101.safetensors", + "model.layers.43.mlp.experts.53.up_proj.weight": "model-00046-of-00101.safetensors", + "model.layers.43.mlp.experts.53.down_proj.weight": "model-00046-of-00101.safetensors", + "model.layers.43.mlp.experts.54.gate_proj.weight": "model-00046-of-00101.safetensors", + "model.layers.43.mlp.experts.54.up_proj.weight": "model-00046-of-00101.safetensors", + "model.layers.43.mlp.experts.54.down_proj.weight": "model-00046-of-00101.safetensors", + "model.layers.43.mlp.experts.55.gate_proj.weight": "model-00046-of-00101.safetensors", + "model.layers.43.mlp.experts.55.up_proj.weight": "model-00046-of-00101.safetensors", + "model.layers.43.mlp.experts.55.down_proj.weight": "model-00046-of-00101.safetensors", + "model.layers.43.mlp.experts.56.gate_proj.weight": "model-00046-of-00101.safetensors", + "model.layers.43.mlp.experts.56.up_proj.weight": "model-00046-of-00101.safetensors", + "model.layers.43.mlp.experts.56.down_proj.weight": "model-00046-of-00101.safetensors", + "model.layers.43.mlp.experts.57.gate_proj.weight": "model-00046-of-00101.safetensors", + "model.layers.43.mlp.experts.57.up_proj.weight": "model-00046-of-00101.safetensors", + "model.layers.43.mlp.experts.57.down_proj.weight": "model-00046-of-00101.safetensors", + "model.layers.43.mlp.experts.58.gate_proj.weight": "model-00046-of-00101.safetensors", + "model.layers.43.mlp.experts.58.up_proj.weight": "model-00046-of-00101.safetensors", + "model.layers.43.mlp.experts.58.down_proj.weight": "model-00046-of-00101.safetensors", + "model.layers.43.mlp.experts.59.gate_proj.weight": "model-00046-of-00101.safetensors", + "model.layers.43.mlp.experts.59.up_proj.weight": "model-00046-of-00101.safetensors", + "model.layers.43.mlp.experts.59.down_proj.weight": "model-00046-of-00101.safetensors", + "model.layers.43.mlp.experts.60.gate_proj.weight": "model-00046-of-00101.safetensors", + "model.layers.43.mlp.experts.60.up_proj.weight": "model-00046-of-00101.safetensors", + "model.layers.43.mlp.experts.60.down_proj.weight": "model-00046-of-00101.safetensors", + "model.layers.43.mlp.experts.61.gate_proj.weight": "model-00046-of-00101.safetensors", + "model.layers.43.mlp.experts.61.up_proj.weight": "model-00046-of-00101.safetensors", + "model.layers.43.mlp.experts.61.down_proj.weight": "model-00046-of-00101.safetensors", + "model.layers.43.mlp.experts.62.gate_proj.weight": "model-00046-of-00101.safetensors", + "model.layers.43.mlp.experts.62.up_proj.weight": "model-00046-of-00101.safetensors", + "model.layers.43.mlp.experts.62.down_proj.weight": "model-00046-of-00101.safetensors", + "model.layers.43.mlp.experts.63.gate_proj.weight": "model-00046-of-00101.safetensors", + "model.layers.43.mlp.experts.63.up_proj.weight": "model-00046-of-00101.safetensors", + "model.layers.43.mlp.experts.63.down_proj.weight": "model-00046-of-00101.safetensors", + "model.layers.43.mlp.experts.64.gate_proj.weight": "model-00046-of-00101.safetensors", + "model.layers.43.mlp.experts.64.up_proj.weight": "model-00046-of-00101.safetensors", + "model.layers.43.mlp.experts.64.down_proj.weight": "model-00046-of-00101.safetensors", + "model.layers.43.mlp.experts.65.gate_proj.weight": "model-00046-of-00101.safetensors", + "model.layers.43.mlp.experts.65.up_proj.weight": "model-00046-of-00101.safetensors", + "model.layers.43.mlp.experts.65.down_proj.weight": "model-00046-of-00101.safetensors", + "model.layers.43.mlp.experts.66.gate_proj.weight": "model-00046-of-00101.safetensors", + "model.layers.43.mlp.experts.66.up_proj.weight": "model-00046-of-00101.safetensors", + "model.layers.43.mlp.experts.66.down_proj.weight": "model-00046-of-00101.safetensors", + "model.layers.43.mlp.experts.67.gate_proj.weight": "model-00046-of-00101.safetensors", + "model.layers.43.mlp.experts.67.up_proj.weight": "model-00046-of-00101.safetensors", + "model.layers.43.mlp.experts.67.down_proj.weight": "model-00047-of-00101.safetensors", + "model.layers.43.mlp.experts.68.gate_proj.weight": "model-00047-of-00101.safetensors", + "model.layers.43.mlp.experts.68.up_proj.weight": "model-00047-of-00101.safetensors", + "model.layers.43.mlp.experts.68.down_proj.weight": "model-00047-of-00101.safetensors", + "model.layers.43.mlp.experts.69.gate_proj.weight": "model-00047-of-00101.safetensors", + "model.layers.43.mlp.experts.69.up_proj.weight": "model-00047-of-00101.safetensors", + "model.layers.43.mlp.experts.69.down_proj.weight": "model-00047-of-00101.safetensors", + "model.layers.43.mlp.experts.70.gate_proj.weight": "model-00047-of-00101.safetensors", + "model.layers.43.mlp.experts.70.up_proj.weight": "model-00047-of-00101.safetensors", + "model.layers.43.mlp.experts.70.down_proj.weight": "model-00047-of-00101.safetensors", + "model.layers.43.mlp.experts.71.gate_proj.weight": "model-00047-of-00101.safetensors", + "model.layers.43.mlp.experts.71.up_proj.weight": "model-00047-of-00101.safetensors", + "model.layers.43.mlp.experts.71.down_proj.weight": "model-00047-of-00101.safetensors", + "model.layers.43.mlp.experts.72.gate_proj.weight": "model-00047-of-00101.safetensors", + "model.layers.43.mlp.experts.72.up_proj.weight": "model-00047-of-00101.safetensors", + "model.layers.43.mlp.experts.72.down_proj.weight": "model-00047-of-00101.safetensors", + "model.layers.43.mlp.experts.73.gate_proj.weight": "model-00047-of-00101.safetensors", + "model.layers.43.mlp.experts.73.up_proj.weight": "model-00047-of-00101.safetensors", + "model.layers.43.mlp.experts.73.down_proj.weight": "model-00047-of-00101.safetensors", + "model.layers.43.mlp.experts.74.gate_proj.weight": "model-00047-of-00101.safetensors", + "model.layers.43.mlp.experts.74.up_proj.weight": "model-00047-of-00101.safetensors", + "model.layers.43.mlp.experts.74.down_proj.weight": "model-00047-of-00101.safetensors", + "model.layers.43.mlp.experts.75.gate_proj.weight": "model-00047-of-00101.safetensors", + "model.layers.43.mlp.experts.75.up_proj.weight": "model-00047-of-00101.safetensors", + "model.layers.43.mlp.experts.75.down_proj.weight": "model-00047-of-00101.safetensors", + "model.layers.43.mlp.experts.76.gate_proj.weight": "model-00047-of-00101.safetensors", + "model.layers.43.mlp.experts.76.up_proj.weight": "model-00047-of-00101.safetensors", + "model.layers.43.mlp.experts.76.down_proj.weight": "model-00047-of-00101.safetensors", + "model.layers.43.mlp.experts.77.gate_proj.weight": "model-00047-of-00101.safetensors", + "model.layers.43.mlp.experts.77.up_proj.weight": "model-00047-of-00101.safetensors", + "model.layers.43.mlp.experts.77.down_proj.weight": "model-00047-of-00101.safetensors", + "model.layers.43.mlp.experts.78.gate_proj.weight": "model-00047-of-00101.safetensors", + "model.layers.43.mlp.experts.78.up_proj.weight": "model-00047-of-00101.safetensors", + "model.layers.43.mlp.experts.78.down_proj.weight": "model-00047-of-00101.safetensors", + "model.layers.43.mlp.experts.79.gate_proj.weight": "model-00047-of-00101.safetensors", + "model.layers.43.mlp.experts.79.up_proj.weight": "model-00047-of-00101.safetensors", + "model.layers.43.mlp.experts.79.down_proj.weight": "model-00047-of-00101.safetensors", + "model.layers.43.mlp.experts.80.gate_proj.weight": "model-00047-of-00101.safetensors", + "model.layers.43.mlp.experts.80.up_proj.weight": "model-00047-of-00101.safetensors", + "model.layers.43.mlp.experts.80.down_proj.weight": "model-00047-of-00101.safetensors", + "model.layers.43.mlp.experts.81.gate_proj.weight": "model-00047-of-00101.safetensors", + "model.layers.43.mlp.experts.81.up_proj.weight": "model-00047-of-00101.safetensors", + "model.layers.43.mlp.experts.81.down_proj.weight": "model-00047-of-00101.safetensors", + "model.layers.43.mlp.experts.82.gate_proj.weight": "model-00047-of-00101.safetensors", + "model.layers.43.mlp.experts.82.up_proj.weight": "model-00047-of-00101.safetensors", + "model.layers.43.mlp.experts.82.down_proj.weight": "model-00047-of-00101.safetensors", + "model.layers.43.mlp.experts.83.gate_proj.weight": "model-00047-of-00101.safetensors", + "model.layers.43.mlp.experts.83.up_proj.weight": "model-00047-of-00101.safetensors", + "model.layers.43.mlp.experts.83.down_proj.weight": "model-00047-of-00101.safetensors", + "model.layers.43.mlp.experts.84.gate_proj.weight": "model-00047-of-00101.safetensors", + "model.layers.43.mlp.experts.84.up_proj.weight": "model-00047-of-00101.safetensors", + "model.layers.43.mlp.experts.84.down_proj.weight": "model-00047-of-00101.safetensors", + "model.layers.43.mlp.experts.85.gate_proj.weight": "model-00047-of-00101.safetensors", + "model.layers.43.mlp.experts.85.up_proj.weight": "model-00047-of-00101.safetensors", + "model.layers.43.mlp.experts.85.down_proj.weight": "model-00047-of-00101.safetensors", + "model.layers.43.mlp.experts.86.gate_proj.weight": "model-00047-of-00101.safetensors", + "model.layers.43.mlp.experts.86.up_proj.weight": "model-00047-of-00101.safetensors", + "model.layers.43.mlp.experts.86.down_proj.weight": "model-00047-of-00101.safetensors", + "model.layers.43.mlp.experts.87.gate_proj.weight": "model-00047-of-00101.safetensors", + "model.layers.43.mlp.experts.87.up_proj.weight": "model-00047-of-00101.safetensors", + "model.layers.43.mlp.experts.87.down_proj.weight": "model-00047-of-00101.safetensors", + "model.layers.43.mlp.experts.88.gate_proj.weight": "model-00047-of-00101.safetensors", + "model.layers.43.mlp.experts.88.up_proj.weight": "model-00047-of-00101.safetensors", + "model.layers.43.mlp.experts.88.down_proj.weight": "model-00047-of-00101.safetensors", + "model.layers.43.mlp.experts.89.gate_proj.weight": "model-00047-of-00101.safetensors", + "model.layers.43.mlp.experts.89.up_proj.weight": "model-00047-of-00101.safetensors", + "model.layers.43.mlp.experts.89.down_proj.weight": "model-00047-of-00101.safetensors", + "model.layers.43.mlp.experts.90.gate_proj.weight": "model-00047-of-00101.safetensors", + "model.layers.43.mlp.experts.90.up_proj.weight": "model-00047-of-00101.safetensors", + "model.layers.43.mlp.experts.90.down_proj.weight": "model-00047-of-00101.safetensors", + "model.layers.43.mlp.experts.91.gate_proj.weight": "model-00047-of-00101.safetensors", + "model.layers.43.mlp.experts.91.up_proj.weight": "model-00047-of-00101.safetensors", + "model.layers.43.mlp.experts.91.down_proj.weight": "model-00047-of-00101.safetensors", + "model.layers.43.mlp.experts.92.gate_proj.weight": "model-00047-of-00101.safetensors", + "model.layers.43.mlp.experts.92.up_proj.weight": "model-00047-of-00101.safetensors", + "model.layers.43.mlp.experts.92.down_proj.weight": "model-00047-of-00101.safetensors", + "model.layers.43.mlp.experts.93.gate_proj.weight": "model-00047-of-00101.safetensors", + "model.layers.43.mlp.experts.93.up_proj.weight": "model-00047-of-00101.safetensors", + "model.layers.43.mlp.experts.93.down_proj.weight": "model-00047-of-00101.safetensors", + "model.layers.43.mlp.experts.94.gate_proj.weight": "model-00047-of-00101.safetensors", + "model.layers.43.mlp.experts.94.up_proj.weight": "model-00047-of-00101.safetensors", + "model.layers.43.mlp.experts.94.down_proj.weight": "model-00047-of-00101.safetensors", + "model.layers.43.mlp.experts.95.gate_proj.weight": "model-00047-of-00101.safetensors", + "model.layers.43.mlp.experts.95.up_proj.weight": "model-00047-of-00101.safetensors", + "model.layers.43.mlp.experts.95.down_proj.weight": "model-00047-of-00101.safetensors", + "model.layers.43.mlp.experts.96.gate_proj.weight": "model-00047-of-00101.safetensors", + "model.layers.43.mlp.experts.96.up_proj.weight": "model-00047-of-00101.safetensors", + "model.layers.43.mlp.experts.96.down_proj.weight": "model-00047-of-00101.safetensors", + "model.layers.43.mlp.experts.97.gate_proj.weight": "model-00047-of-00101.safetensors", + "model.layers.43.mlp.experts.97.up_proj.weight": "model-00047-of-00101.safetensors", + "model.layers.43.mlp.experts.97.down_proj.weight": "model-00047-of-00101.safetensors", + "model.layers.43.mlp.experts.98.gate_proj.weight": "model-00047-of-00101.safetensors", + "model.layers.43.mlp.experts.98.up_proj.weight": "model-00047-of-00101.safetensors", + "model.layers.43.mlp.experts.98.down_proj.weight": "model-00047-of-00101.safetensors", + "model.layers.43.mlp.experts.99.gate_proj.weight": "model-00047-of-00101.safetensors", + "model.layers.43.mlp.experts.99.up_proj.weight": "model-00047-of-00101.safetensors", + "model.layers.43.mlp.experts.99.down_proj.weight": "model-00047-of-00101.safetensors", + "model.layers.43.mlp.experts.100.gate_proj.weight": "model-00047-of-00101.safetensors", + "model.layers.43.mlp.experts.100.up_proj.weight": "model-00047-of-00101.safetensors", + "model.layers.43.mlp.experts.100.down_proj.weight": "model-00047-of-00101.safetensors", + "model.layers.43.mlp.experts.101.gate_proj.weight": "model-00047-of-00101.safetensors", + "model.layers.43.mlp.experts.101.up_proj.weight": "model-00047-of-00101.safetensors", + "model.layers.43.mlp.experts.101.down_proj.weight": "model-00047-of-00101.safetensors", + "model.layers.43.mlp.experts.102.gate_proj.weight": "model-00047-of-00101.safetensors", + "model.layers.43.mlp.experts.102.up_proj.weight": "model-00047-of-00101.safetensors", + "model.layers.43.mlp.experts.102.down_proj.weight": "model-00047-of-00101.safetensors", + "model.layers.43.mlp.experts.103.gate_proj.weight": "model-00047-of-00101.safetensors", + "model.layers.43.mlp.experts.103.up_proj.weight": "model-00047-of-00101.safetensors", + "model.layers.43.mlp.experts.103.down_proj.weight": "model-00047-of-00101.safetensors", + "model.layers.43.mlp.experts.104.gate_proj.weight": "model-00047-of-00101.safetensors", + "model.layers.43.mlp.experts.104.up_proj.weight": "model-00047-of-00101.safetensors", + "model.layers.43.mlp.experts.104.down_proj.weight": "model-00047-of-00101.safetensors", + "model.layers.43.mlp.experts.105.gate_proj.weight": "model-00047-of-00101.safetensors", + "model.layers.43.mlp.experts.105.up_proj.weight": "model-00047-of-00101.safetensors", + "model.layers.43.mlp.experts.105.down_proj.weight": "model-00047-of-00101.safetensors", + "model.layers.43.mlp.experts.106.gate_proj.weight": "model-00047-of-00101.safetensors", + "model.layers.43.mlp.experts.106.up_proj.weight": "model-00047-of-00101.safetensors", + "model.layers.43.mlp.experts.106.down_proj.weight": "model-00047-of-00101.safetensors", + "model.layers.43.mlp.experts.107.gate_proj.weight": "model-00047-of-00101.safetensors", + "model.layers.43.mlp.experts.107.up_proj.weight": "model-00047-of-00101.safetensors", + "model.layers.43.mlp.experts.107.down_proj.weight": "model-00047-of-00101.safetensors", + "model.layers.43.mlp.experts.108.gate_proj.weight": "model-00047-of-00101.safetensors", + "model.layers.43.mlp.experts.108.up_proj.weight": "model-00047-of-00101.safetensors", + "model.layers.43.mlp.experts.108.down_proj.weight": "model-00047-of-00101.safetensors", + "model.layers.43.mlp.experts.109.gate_proj.weight": "model-00047-of-00101.safetensors", + "model.layers.43.mlp.experts.109.up_proj.weight": "model-00047-of-00101.safetensors", + "model.layers.43.mlp.experts.109.down_proj.weight": "model-00047-of-00101.safetensors", + "model.layers.43.mlp.experts.110.gate_proj.weight": "model-00047-of-00101.safetensors", + "model.layers.43.mlp.experts.110.up_proj.weight": "model-00047-of-00101.safetensors", + "model.layers.43.mlp.experts.110.down_proj.weight": "model-00047-of-00101.safetensors", + "model.layers.43.mlp.experts.111.gate_proj.weight": "model-00047-of-00101.safetensors", + "model.layers.43.mlp.experts.111.up_proj.weight": "model-00047-of-00101.safetensors", + "model.layers.43.mlp.experts.111.down_proj.weight": "model-00047-of-00101.safetensors", + "model.layers.43.mlp.experts.112.gate_proj.weight": "model-00047-of-00101.safetensors", + "model.layers.43.mlp.experts.112.up_proj.weight": "model-00047-of-00101.safetensors", + "model.layers.43.mlp.experts.112.down_proj.weight": "model-00047-of-00101.safetensors", + "model.layers.43.mlp.experts.113.gate_proj.weight": "model-00047-of-00101.safetensors", + "model.layers.43.mlp.experts.113.up_proj.weight": "model-00047-of-00101.safetensors", + "model.layers.43.mlp.experts.113.down_proj.weight": "model-00047-of-00101.safetensors", + "model.layers.43.mlp.experts.114.gate_proj.weight": "model-00047-of-00101.safetensors", + "model.layers.43.mlp.experts.114.up_proj.weight": "model-00047-of-00101.safetensors", + "model.layers.43.mlp.experts.114.down_proj.weight": "model-00047-of-00101.safetensors", + "model.layers.43.mlp.experts.115.gate_proj.weight": "model-00047-of-00101.safetensors", + "model.layers.43.mlp.experts.115.up_proj.weight": "model-00047-of-00101.safetensors", + "model.layers.43.mlp.experts.115.down_proj.weight": "model-00047-of-00101.safetensors", + "model.layers.43.mlp.experts.116.gate_proj.weight": "model-00047-of-00101.safetensors", + "model.layers.43.mlp.experts.116.up_proj.weight": "model-00047-of-00101.safetensors", + "model.layers.43.mlp.experts.116.down_proj.weight": "model-00047-of-00101.safetensors", + "model.layers.43.mlp.experts.117.gate_proj.weight": "model-00047-of-00101.safetensors", + "model.layers.43.mlp.experts.117.up_proj.weight": "model-00047-of-00101.safetensors", + "model.layers.43.mlp.experts.117.down_proj.weight": "model-00047-of-00101.safetensors", + "model.layers.43.mlp.experts.118.gate_proj.weight": "model-00047-of-00101.safetensors", + "model.layers.43.mlp.experts.118.up_proj.weight": "model-00047-of-00101.safetensors", + "model.layers.43.mlp.experts.118.down_proj.weight": "model-00047-of-00101.safetensors", + "model.layers.43.mlp.experts.119.gate_proj.weight": "model-00047-of-00101.safetensors", + "model.layers.43.mlp.experts.119.up_proj.weight": "model-00047-of-00101.safetensors", + "model.layers.43.mlp.experts.119.down_proj.weight": "model-00047-of-00101.safetensors", + "model.layers.43.mlp.gate.weight": "model-00047-of-00101.safetensors", + "model.layers.43.mlp.gate.e_score_correction_bias": "model-00047-of-00101.safetensors", + "model.layers.43.mlp.shared_experts.gate_proj.weight": "model-00047-of-00101.safetensors", + "model.layers.43.mlp.shared_experts.up_proj.weight": "model-00047-of-00101.safetensors", + "model.layers.43.mlp.shared_experts.down_proj.weight": "model-00047-of-00101.safetensors", + "model.layers.43.input_layernorm.weight": "model-00047-of-00101.safetensors", + "model.layers.43.post_attention_layernorm.weight": "model-00047-of-00101.safetensors", + "model.layers.44.self_attn.q_proj.weight": "model-00047-of-00101.safetensors", + "model.layers.44.self_attn.q_proj.bias": "model-00047-of-00101.safetensors", + "model.layers.44.self_attn.k_proj.weight": "model-00047-of-00101.safetensors", + "model.layers.44.self_attn.k_proj.bias": "model-00047-of-00101.safetensors", + "model.layers.44.self_attn.v_proj.weight": "model-00047-of-00101.safetensors", + "model.layers.44.self_attn.v_proj.bias": "model-00047-of-00101.safetensors", + "model.layers.44.self_attn.o_proj.weight": "model-00047-of-00101.safetensors", + "model.layers.44.self_attn.q_norm.weight": "model-00047-of-00101.safetensors", + "model.layers.44.self_attn.k_norm.weight": "model-00047-of-00101.safetensors", + "model.layers.44.mlp.experts.0.gate_proj.weight": "model-00047-of-00101.safetensors", + "model.layers.44.mlp.experts.0.up_proj.weight": "model-00047-of-00101.safetensors", + "model.layers.44.mlp.experts.0.down_proj.weight": "model-00047-of-00101.safetensors", + "model.layers.44.mlp.experts.1.gate_proj.weight": "model-00047-of-00101.safetensors", + "model.layers.44.mlp.experts.1.up_proj.weight": "model-00047-of-00101.safetensors", + "model.layers.44.mlp.experts.1.down_proj.weight": "model-00047-of-00101.safetensors", + "model.layers.44.mlp.experts.2.gate_proj.weight": "model-00047-of-00101.safetensors", + "model.layers.44.mlp.experts.2.up_proj.weight": "model-00047-of-00101.safetensors", + "model.layers.44.mlp.experts.2.down_proj.weight": "model-00047-of-00101.safetensors", + "model.layers.44.mlp.experts.3.gate_proj.weight": "model-00047-of-00101.safetensors", + "model.layers.44.mlp.experts.3.up_proj.weight": "model-00047-of-00101.safetensors", + "model.layers.44.mlp.experts.3.down_proj.weight": "model-00047-of-00101.safetensors", + "model.layers.44.mlp.experts.4.gate_proj.weight": "model-00047-of-00101.safetensors", + "model.layers.44.mlp.experts.4.up_proj.weight": "model-00047-of-00101.safetensors", + "model.layers.44.mlp.experts.4.down_proj.weight": "model-00047-of-00101.safetensors", + "model.layers.44.mlp.experts.5.gate_proj.weight": "model-00047-of-00101.safetensors", + "model.layers.44.mlp.experts.5.up_proj.weight": "model-00047-of-00101.safetensors", + "model.layers.44.mlp.experts.5.down_proj.weight": "model-00047-of-00101.safetensors", + "model.layers.44.mlp.experts.6.gate_proj.weight": "model-00047-of-00101.safetensors", + "model.layers.44.mlp.experts.6.up_proj.weight": "model-00047-of-00101.safetensors", + "model.layers.44.mlp.experts.6.down_proj.weight": "model-00047-of-00101.safetensors", + "model.layers.44.mlp.experts.7.gate_proj.weight": "model-00047-of-00101.safetensors", + "model.layers.44.mlp.experts.7.up_proj.weight": "model-00047-of-00101.safetensors", + "model.layers.44.mlp.experts.7.down_proj.weight": "model-00047-of-00101.safetensors", + "model.layers.44.mlp.experts.8.gate_proj.weight": "model-00047-of-00101.safetensors", + "model.layers.44.mlp.experts.8.up_proj.weight": "model-00047-of-00101.safetensors", + "model.layers.44.mlp.experts.8.down_proj.weight": "model-00047-of-00101.safetensors", + "model.layers.44.mlp.experts.9.gate_proj.weight": "model-00047-of-00101.safetensors", + "model.layers.44.mlp.experts.9.up_proj.weight": "model-00047-of-00101.safetensors", + "model.layers.44.mlp.experts.9.down_proj.weight": "model-00047-of-00101.safetensors", + "model.layers.44.mlp.experts.10.gate_proj.weight": "model-00047-of-00101.safetensors", + "model.layers.44.mlp.experts.10.up_proj.weight": "model-00047-of-00101.safetensors", + "model.layers.44.mlp.experts.10.down_proj.weight": "model-00047-of-00101.safetensors", + "model.layers.44.mlp.experts.11.gate_proj.weight": "model-00047-of-00101.safetensors", + "model.layers.44.mlp.experts.11.up_proj.weight": "model-00047-of-00101.safetensors", + "model.layers.44.mlp.experts.11.down_proj.weight": "model-00047-of-00101.safetensors", + "model.layers.44.mlp.experts.12.gate_proj.weight": "model-00047-of-00101.safetensors", + "model.layers.44.mlp.experts.12.up_proj.weight": "model-00047-of-00101.safetensors", + "model.layers.44.mlp.experts.12.down_proj.weight": "model-00047-of-00101.safetensors", + "model.layers.44.mlp.experts.13.gate_proj.weight": "model-00047-of-00101.safetensors", + "model.layers.44.mlp.experts.13.up_proj.weight": "model-00047-of-00101.safetensors", + "model.layers.44.mlp.experts.13.down_proj.weight": "model-00047-of-00101.safetensors", + "model.layers.44.mlp.experts.14.gate_proj.weight": "model-00047-of-00101.safetensors", + "model.layers.44.mlp.experts.14.up_proj.weight": "model-00047-of-00101.safetensors", + "model.layers.44.mlp.experts.14.down_proj.weight": "model-00047-of-00101.safetensors", + "model.layers.44.mlp.experts.15.gate_proj.weight": "model-00047-of-00101.safetensors", + "model.layers.44.mlp.experts.15.up_proj.weight": "model-00047-of-00101.safetensors", + "model.layers.44.mlp.experts.15.down_proj.weight": "model-00047-of-00101.safetensors", + "model.layers.44.mlp.experts.16.gate_proj.weight": "model-00047-of-00101.safetensors", + "model.layers.44.mlp.experts.16.up_proj.weight": "model-00047-of-00101.safetensors", + "model.layers.44.mlp.experts.16.down_proj.weight": "model-00047-of-00101.safetensors", + "model.layers.44.mlp.experts.17.gate_proj.weight": "model-00047-of-00101.safetensors", + "model.layers.44.mlp.experts.17.up_proj.weight": "model-00047-of-00101.safetensors", + "model.layers.44.mlp.experts.17.down_proj.weight": "model-00047-of-00101.safetensors", + "model.layers.44.mlp.experts.18.gate_proj.weight": "model-00047-of-00101.safetensors", + "model.layers.44.mlp.experts.18.up_proj.weight": "model-00047-of-00101.safetensors", + "model.layers.44.mlp.experts.18.down_proj.weight": "model-00047-of-00101.safetensors", + "model.layers.44.mlp.experts.19.gate_proj.weight": "model-00047-of-00101.safetensors", + "model.layers.44.mlp.experts.19.up_proj.weight": "model-00047-of-00101.safetensors", + "model.layers.44.mlp.experts.19.down_proj.weight": "model-00047-of-00101.safetensors", + "model.layers.44.mlp.experts.20.gate_proj.weight": "model-00047-of-00101.safetensors", + "model.layers.44.mlp.experts.20.up_proj.weight": "model-00047-of-00101.safetensors", + "model.layers.44.mlp.experts.20.down_proj.weight": "model-00047-of-00101.safetensors", + "model.layers.44.mlp.experts.21.gate_proj.weight": "model-00047-of-00101.safetensors", + "model.layers.44.mlp.experts.21.up_proj.weight": "model-00047-of-00101.safetensors", + "model.layers.44.mlp.experts.21.down_proj.weight": "model-00047-of-00101.safetensors", + "model.layers.44.mlp.experts.22.gate_proj.weight": "model-00047-of-00101.safetensors", + "model.layers.44.mlp.experts.22.up_proj.weight": "model-00047-of-00101.safetensors", + "model.layers.44.mlp.experts.22.down_proj.weight": "model-00047-of-00101.safetensors", + "model.layers.44.mlp.experts.23.gate_proj.weight": "model-00047-of-00101.safetensors", + "model.layers.44.mlp.experts.23.up_proj.weight": "model-00047-of-00101.safetensors", + "model.layers.44.mlp.experts.23.down_proj.weight": "model-00047-of-00101.safetensors", + "model.layers.44.mlp.experts.24.gate_proj.weight": "model-00047-of-00101.safetensors", + "model.layers.44.mlp.experts.24.up_proj.weight": "model-00047-of-00101.safetensors", + "model.layers.44.mlp.experts.24.down_proj.weight": "model-00047-of-00101.safetensors", + "model.layers.44.mlp.experts.25.gate_proj.weight": "model-00047-of-00101.safetensors", + "model.layers.44.mlp.experts.25.up_proj.weight": "model-00047-of-00101.safetensors", + "model.layers.44.mlp.experts.25.down_proj.weight": "model-00047-of-00101.safetensors", + "model.layers.44.mlp.experts.26.gate_proj.weight": "model-00047-of-00101.safetensors", + "model.layers.44.mlp.experts.26.up_proj.weight": "model-00047-of-00101.safetensors", + "model.layers.44.mlp.experts.26.down_proj.weight": "model-00047-of-00101.safetensors", + "model.layers.44.mlp.experts.27.gate_proj.weight": "model-00047-of-00101.safetensors", + "model.layers.44.mlp.experts.27.up_proj.weight": "model-00047-of-00101.safetensors", + "model.layers.44.mlp.experts.27.down_proj.weight": "model-00047-of-00101.safetensors", + "model.layers.44.mlp.experts.28.gate_proj.weight": "model-00047-of-00101.safetensors", + "model.layers.44.mlp.experts.28.up_proj.weight": "model-00047-of-00101.safetensors", + "model.layers.44.mlp.experts.28.down_proj.weight": "model-00047-of-00101.safetensors", + "model.layers.44.mlp.experts.29.gate_proj.weight": "model-00047-of-00101.safetensors", + "model.layers.44.mlp.experts.29.up_proj.weight": "model-00047-of-00101.safetensors", + "model.layers.44.mlp.experts.29.down_proj.weight": "model-00047-of-00101.safetensors", + "model.layers.44.mlp.experts.30.gate_proj.weight": "model-00047-of-00101.safetensors", + "model.layers.44.mlp.experts.30.up_proj.weight": "model-00047-of-00101.safetensors", + "model.layers.44.mlp.experts.30.down_proj.weight": "model-00047-of-00101.safetensors", + "model.layers.44.mlp.experts.31.gate_proj.weight": "model-00047-of-00101.safetensors", + "model.layers.44.mlp.experts.31.up_proj.weight": "model-00047-of-00101.safetensors", + "model.layers.44.mlp.experts.31.down_proj.weight": "model-00047-of-00101.safetensors", + "model.layers.44.mlp.experts.32.gate_proj.weight": "model-00047-of-00101.safetensors", + "model.layers.44.mlp.experts.32.up_proj.weight": "model-00047-of-00101.safetensors", + "model.layers.44.mlp.experts.32.down_proj.weight": "model-00047-of-00101.safetensors", + "model.layers.44.mlp.experts.33.gate_proj.weight": "model-00047-of-00101.safetensors", + "model.layers.44.mlp.experts.33.up_proj.weight": "model-00047-of-00101.safetensors", + "model.layers.44.mlp.experts.33.down_proj.weight": "model-00047-of-00101.safetensors", + "model.layers.44.mlp.experts.34.gate_proj.weight": "model-00047-of-00101.safetensors", + "model.layers.44.mlp.experts.34.up_proj.weight": "model-00047-of-00101.safetensors", + "model.layers.44.mlp.experts.34.down_proj.weight": "model-00047-of-00101.safetensors", + "model.layers.44.mlp.experts.35.gate_proj.weight": "model-00047-of-00101.safetensors", + "model.layers.44.mlp.experts.35.up_proj.weight": "model-00047-of-00101.safetensors", + "model.layers.44.mlp.experts.35.down_proj.weight": "model-00047-of-00101.safetensors", + "model.layers.44.mlp.experts.36.gate_proj.weight": "model-00047-of-00101.safetensors", + "model.layers.44.mlp.experts.36.up_proj.weight": "model-00047-of-00101.safetensors", + "model.layers.44.mlp.experts.36.down_proj.weight": "model-00047-of-00101.safetensors", + "model.layers.44.mlp.experts.37.gate_proj.weight": "model-00047-of-00101.safetensors", + "model.layers.44.mlp.experts.37.up_proj.weight": "model-00047-of-00101.safetensors", + "model.layers.44.mlp.experts.37.down_proj.weight": "model-00047-of-00101.safetensors", + "model.layers.44.mlp.experts.38.gate_proj.weight": "model-00047-of-00101.safetensors", + "model.layers.44.mlp.experts.38.up_proj.weight": "model-00047-of-00101.safetensors", + "model.layers.44.mlp.experts.38.down_proj.weight": "model-00047-of-00101.safetensors", + "model.layers.44.mlp.experts.39.gate_proj.weight": "model-00047-of-00101.safetensors", + "model.layers.44.mlp.experts.39.up_proj.weight": "model-00047-of-00101.safetensors", + "model.layers.44.mlp.experts.39.down_proj.weight": "model-00047-of-00101.safetensors", + "model.layers.44.mlp.experts.40.gate_proj.weight": "model-00047-of-00101.safetensors", + "model.layers.44.mlp.experts.40.up_proj.weight": "model-00047-of-00101.safetensors", + "model.layers.44.mlp.experts.40.down_proj.weight": "model-00047-of-00101.safetensors", + "model.layers.44.mlp.experts.41.gate_proj.weight": "model-00047-of-00101.safetensors", + "model.layers.44.mlp.experts.41.up_proj.weight": "model-00047-of-00101.safetensors", + "model.layers.44.mlp.experts.41.down_proj.weight": "model-00047-of-00101.safetensors", + "model.layers.44.mlp.experts.42.gate_proj.weight": "model-00047-of-00101.safetensors", + "model.layers.44.mlp.experts.42.up_proj.weight": "model-00047-of-00101.safetensors", + "model.layers.44.mlp.experts.42.down_proj.weight": "model-00047-of-00101.safetensors", + "model.layers.44.mlp.experts.43.gate_proj.weight": "model-00047-of-00101.safetensors", + "model.layers.44.mlp.experts.43.up_proj.weight": "model-00047-of-00101.safetensors", + "model.layers.44.mlp.experts.43.down_proj.weight": "model-00047-of-00101.safetensors", + "model.layers.44.mlp.experts.44.gate_proj.weight": "model-00047-of-00101.safetensors", + "model.layers.44.mlp.experts.44.up_proj.weight": "model-00047-of-00101.safetensors", + "model.layers.44.mlp.experts.44.down_proj.weight": "model-00047-of-00101.safetensors", + "model.layers.44.mlp.experts.45.gate_proj.weight": "model-00047-of-00101.safetensors", + "model.layers.44.mlp.experts.45.up_proj.weight": "model-00047-of-00101.safetensors", + "model.layers.44.mlp.experts.45.down_proj.weight": "model-00047-of-00101.safetensors", + "model.layers.44.mlp.experts.46.gate_proj.weight": "model-00047-of-00101.safetensors", + "model.layers.44.mlp.experts.46.up_proj.weight": "model-00047-of-00101.safetensors", + "model.layers.44.mlp.experts.46.down_proj.weight": "model-00047-of-00101.safetensors", + "model.layers.44.mlp.experts.47.gate_proj.weight": "model-00047-of-00101.safetensors", + "model.layers.44.mlp.experts.47.up_proj.weight": "model-00047-of-00101.safetensors", + "model.layers.44.mlp.experts.47.down_proj.weight": "model-00047-of-00101.safetensors", + "model.layers.44.mlp.experts.48.gate_proj.weight": "model-00047-of-00101.safetensors", + "model.layers.44.mlp.experts.48.up_proj.weight": "model-00047-of-00101.safetensors", + "model.layers.44.mlp.experts.48.down_proj.weight": "model-00047-of-00101.safetensors", + "model.layers.44.mlp.experts.49.gate_proj.weight": "model-00047-of-00101.safetensors", + "model.layers.44.mlp.experts.49.up_proj.weight": "model-00047-of-00101.safetensors", + "model.layers.44.mlp.experts.49.down_proj.weight": "model-00047-of-00101.safetensors", + "model.layers.44.mlp.experts.50.gate_proj.weight": "model-00047-of-00101.safetensors", + "model.layers.44.mlp.experts.50.up_proj.weight": "model-00047-of-00101.safetensors", + "model.layers.44.mlp.experts.50.down_proj.weight": "model-00047-of-00101.safetensors", + "model.layers.44.mlp.experts.51.gate_proj.weight": "model-00047-of-00101.safetensors", + "model.layers.44.mlp.experts.51.up_proj.weight": "model-00047-of-00101.safetensors", + "model.layers.44.mlp.experts.51.down_proj.weight": "model-00047-of-00101.safetensors", + "model.layers.44.mlp.experts.52.gate_proj.weight": "model-00047-of-00101.safetensors", + "model.layers.44.mlp.experts.52.up_proj.weight": "model-00047-of-00101.safetensors", + "model.layers.44.mlp.experts.52.down_proj.weight": "model-00047-of-00101.safetensors", + "model.layers.44.mlp.experts.53.gate_proj.weight": "model-00047-of-00101.safetensors", + "model.layers.44.mlp.experts.53.up_proj.weight": "model-00047-of-00101.safetensors", + "model.layers.44.mlp.experts.53.down_proj.weight": "model-00047-of-00101.safetensors", + "model.layers.44.mlp.experts.54.gate_proj.weight": "model-00047-of-00101.safetensors", + "model.layers.44.mlp.experts.54.up_proj.weight": "model-00048-of-00101.safetensors", + "model.layers.44.mlp.experts.54.down_proj.weight": "model-00048-of-00101.safetensors", + "model.layers.44.mlp.experts.55.gate_proj.weight": "model-00048-of-00101.safetensors", + "model.layers.44.mlp.experts.55.up_proj.weight": "model-00048-of-00101.safetensors", + "model.layers.44.mlp.experts.55.down_proj.weight": "model-00048-of-00101.safetensors", + "model.layers.44.mlp.experts.56.gate_proj.weight": "model-00048-of-00101.safetensors", + "model.layers.44.mlp.experts.56.up_proj.weight": "model-00048-of-00101.safetensors", + "model.layers.44.mlp.experts.56.down_proj.weight": "model-00048-of-00101.safetensors", + "model.layers.44.mlp.experts.57.gate_proj.weight": "model-00048-of-00101.safetensors", + "model.layers.44.mlp.experts.57.up_proj.weight": "model-00048-of-00101.safetensors", + "model.layers.44.mlp.experts.57.down_proj.weight": "model-00048-of-00101.safetensors", + "model.layers.44.mlp.experts.58.gate_proj.weight": "model-00048-of-00101.safetensors", + "model.layers.44.mlp.experts.58.up_proj.weight": "model-00048-of-00101.safetensors", + "model.layers.44.mlp.experts.58.down_proj.weight": "model-00048-of-00101.safetensors", + "model.layers.44.mlp.experts.59.gate_proj.weight": "model-00048-of-00101.safetensors", + "model.layers.44.mlp.experts.59.up_proj.weight": "model-00048-of-00101.safetensors", + "model.layers.44.mlp.experts.59.down_proj.weight": "model-00048-of-00101.safetensors", + "model.layers.44.mlp.experts.60.gate_proj.weight": "model-00048-of-00101.safetensors", + "model.layers.44.mlp.experts.60.up_proj.weight": "model-00048-of-00101.safetensors", + "model.layers.44.mlp.experts.60.down_proj.weight": "model-00048-of-00101.safetensors", + "model.layers.44.mlp.experts.61.gate_proj.weight": "model-00048-of-00101.safetensors", + "model.layers.44.mlp.experts.61.up_proj.weight": "model-00048-of-00101.safetensors", + "model.layers.44.mlp.experts.61.down_proj.weight": "model-00048-of-00101.safetensors", + "model.layers.44.mlp.experts.62.gate_proj.weight": "model-00048-of-00101.safetensors", + "model.layers.44.mlp.experts.62.up_proj.weight": "model-00048-of-00101.safetensors", + "model.layers.44.mlp.experts.62.down_proj.weight": "model-00048-of-00101.safetensors", + "model.layers.44.mlp.experts.63.gate_proj.weight": "model-00048-of-00101.safetensors", + "model.layers.44.mlp.experts.63.up_proj.weight": "model-00048-of-00101.safetensors", + "model.layers.44.mlp.experts.63.down_proj.weight": "model-00048-of-00101.safetensors", + "model.layers.44.mlp.experts.64.gate_proj.weight": "model-00048-of-00101.safetensors", + "model.layers.44.mlp.experts.64.up_proj.weight": "model-00048-of-00101.safetensors", + "model.layers.44.mlp.experts.64.down_proj.weight": "model-00048-of-00101.safetensors", + "model.layers.44.mlp.experts.65.gate_proj.weight": "model-00048-of-00101.safetensors", + "model.layers.44.mlp.experts.65.up_proj.weight": "model-00048-of-00101.safetensors", + "model.layers.44.mlp.experts.65.down_proj.weight": "model-00048-of-00101.safetensors", + "model.layers.44.mlp.experts.66.gate_proj.weight": "model-00048-of-00101.safetensors", + "model.layers.44.mlp.experts.66.up_proj.weight": "model-00048-of-00101.safetensors", + "model.layers.44.mlp.experts.66.down_proj.weight": "model-00048-of-00101.safetensors", + "model.layers.44.mlp.experts.67.gate_proj.weight": "model-00048-of-00101.safetensors", + "model.layers.44.mlp.experts.67.up_proj.weight": "model-00048-of-00101.safetensors", + "model.layers.44.mlp.experts.67.down_proj.weight": "model-00048-of-00101.safetensors", + "model.layers.44.mlp.experts.68.gate_proj.weight": "model-00048-of-00101.safetensors", + "model.layers.44.mlp.experts.68.up_proj.weight": "model-00048-of-00101.safetensors", + "model.layers.44.mlp.experts.68.down_proj.weight": "model-00048-of-00101.safetensors", + "model.layers.44.mlp.experts.69.gate_proj.weight": "model-00048-of-00101.safetensors", + "model.layers.44.mlp.experts.69.up_proj.weight": "model-00048-of-00101.safetensors", + "model.layers.44.mlp.experts.69.down_proj.weight": "model-00048-of-00101.safetensors", + "model.layers.44.mlp.experts.70.gate_proj.weight": "model-00048-of-00101.safetensors", + "model.layers.44.mlp.experts.70.up_proj.weight": "model-00048-of-00101.safetensors", + "model.layers.44.mlp.experts.70.down_proj.weight": "model-00048-of-00101.safetensors", + "model.layers.44.mlp.experts.71.gate_proj.weight": "model-00048-of-00101.safetensors", + "model.layers.44.mlp.experts.71.up_proj.weight": "model-00048-of-00101.safetensors", + "model.layers.44.mlp.experts.71.down_proj.weight": "model-00048-of-00101.safetensors", + "model.layers.44.mlp.experts.72.gate_proj.weight": "model-00048-of-00101.safetensors", + "model.layers.44.mlp.experts.72.up_proj.weight": "model-00048-of-00101.safetensors", + "model.layers.44.mlp.experts.72.down_proj.weight": "model-00048-of-00101.safetensors", + "model.layers.44.mlp.experts.73.gate_proj.weight": "model-00048-of-00101.safetensors", + "model.layers.44.mlp.experts.73.up_proj.weight": "model-00048-of-00101.safetensors", + "model.layers.44.mlp.experts.73.down_proj.weight": "model-00048-of-00101.safetensors", + "model.layers.44.mlp.experts.74.gate_proj.weight": "model-00048-of-00101.safetensors", + "model.layers.44.mlp.experts.74.up_proj.weight": "model-00048-of-00101.safetensors", + "model.layers.44.mlp.experts.74.down_proj.weight": "model-00048-of-00101.safetensors", + "model.layers.44.mlp.experts.75.gate_proj.weight": "model-00048-of-00101.safetensors", + "model.layers.44.mlp.experts.75.up_proj.weight": "model-00048-of-00101.safetensors", + "model.layers.44.mlp.experts.75.down_proj.weight": "model-00048-of-00101.safetensors", + "model.layers.44.mlp.experts.76.gate_proj.weight": "model-00048-of-00101.safetensors", + "model.layers.44.mlp.experts.76.up_proj.weight": "model-00048-of-00101.safetensors", + "model.layers.44.mlp.experts.76.down_proj.weight": "model-00048-of-00101.safetensors", + "model.layers.44.mlp.experts.77.gate_proj.weight": "model-00048-of-00101.safetensors", + "model.layers.44.mlp.experts.77.up_proj.weight": "model-00048-of-00101.safetensors", + "model.layers.44.mlp.experts.77.down_proj.weight": "model-00048-of-00101.safetensors", + "model.layers.44.mlp.experts.78.gate_proj.weight": "model-00048-of-00101.safetensors", + "model.layers.44.mlp.experts.78.up_proj.weight": "model-00048-of-00101.safetensors", + "model.layers.44.mlp.experts.78.down_proj.weight": "model-00048-of-00101.safetensors", + "model.layers.44.mlp.experts.79.gate_proj.weight": "model-00048-of-00101.safetensors", + "model.layers.44.mlp.experts.79.up_proj.weight": "model-00048-of-00101.safetensors", + "model.layers.44.mlp.experts.79.down_proj.weight": "model-00048-of-00101.safetensors", + "model.layers.44.mlp.experts.80.gate_proj.weight": "model-00048-of-00101.safetensors", + "model.layers.44.mlp.experts.80.up_proj.weight": "model-00048-of-00101.safetensors", + "model.layers.44.mlp.experts.80.down_proj.weight": "model-00048-of-00101.safetensors", + "model.layers.44.mlp.experts.81.gate_proj.weight": "model-00048-of-00101.safetensors", + "model.layers.44.mlp.experts.81.up_proj.weight": "model-00048-of-00101.safetensors", + "model.layers.44.mlp.experts.81.down_proj.weight": "model-00048-of-00101.safetensors", + "model.layers.44.mlp.experts.82.gate_proj.weight": "model-00048-of-00101.safetensors", + "model.layers.44.mlp.experts.82.up_proj.weight": "model-00048-of-00101.safetensors", + "model.layers.44.mlp.experts.82.down_proj.weight": "model-00048-of-00101.safetensors", + "model.layers.44.mlp.experts.83.gate_proj.weight": "model-00048-of-00101.safetensors", + "model.layers.44.mlp.experts.83.up_proj.weight": "model-00048-of-00101.safetensors", + "model.layers.44.mlp.experts.83.down_proj.weight": "model-00048-of-00101.safetensors", + "model.layers.44.mlp.experts.84.gate_proj.weight": "model-00048-of-00101.safetensors", + "model.layers.44.mlp.experts.84.up_proj.weight": "model-00048-of-00101.safetensors", + "model.layers.44.mlp.experts.84.down_proj.weight": "model-00048-of-00101.safetensors", + "model.layers.44.mlp.experts.85.gate_proj.weight": "model-00048-of-00101.safetensors", + "model.layers.44.mlp.experts.85.up_proj.weight": "model-00048-of-00101.safetensors", + "model.layers.44.mlp.experts.85.down_proj.weight": "model-00048-of-00101.safetensors", + "model.layers.44.mlp.experts.86.gate_proj.weight": "model-00048-of-00101.safetensors", + "model.layers.44.mlp.experts.86.up_proj.weight": "model-00048-of-00101.safetensors", + "model.layers.44.mlp.experts.86.down_proj.weight": "model-00048-of-00101.safetensors", + "model.layers.44.mlp.experts.87.gate_proj.weight": "model-00048-of-00101.safetensors", + "model.layers.44.mlp.experts.87.up_proj.weight": "model-00048-of-00101.safetensors", + "model.layers.44.mlp.experts.87.down_proj.weight": "model-00048-of-00101.safetensors", + "model.layers.44.mlp.experts.88.gate_proj.weight": "model-00048-of-00101.safetensors", + "model.layers.44.mlp.experts.88.up_proj.weight": "model-00048-of-00101.safetensors", + "model.layers.44.mlp.experts.88.down_proj.weight": "model-00048-of-00101.safetensors", + "model.layers.44.mlp.experts.89.gate_proj.weight": "model-00048-of-00101.safetensors", + "model.layers.44.mlp.experts.89.up_proj.weight": "model-00048-of-00101.safetensors", + "model.layers.44.mlp.experts.89.down_proj.weight": "model-00048-of-00101.safetensors", + "model.layers.44.mlp.experts.90.gate_proj.weight": "model-00048-of-00101.safetensors", + "model.layers.44.mlp.experts.90.up_proj.weight": "model-00048-of-00101.safetensors", + "model.layers.44.mlp.experts.90.down_proj.weight": "model-00048-of-00101.safetensors", + "model.layers.44.mlp.experts.91.gate_proj.weight": "model-00048-of-00101.safetensors", + "model.layers.44.mlp.experts.91.up_proj.weight": "model-00048-of-00101.safetensors", + "model.layers.44.mlp.experts.91.down_proj.weight": "model-00048-of-00101.safetensors", + "model.layers.44.mlp.experts.92.gate_proj.weight": "model-00048-of-00101.safetensors", + "model.layers.44.mlp.experts.92.up_proj.weight": "model-00048-of-00101.safetensors", + "model.layers.44.mlp.experts.92.down_proj.weight": "model-00048-of-00101.safetensors", + "model.layers.44.mlp.experts.93.gate_proj.weight": "model-00048-of-00101.safetensors", + "model.layers.44.mlp.experts.93.up_proj.weight": "model-00048-of-00101.safetensors", + "model.layers.44.mlp.experts.93.down_proj.weight": "model-00048-of-00101.safetensors", + "model.layers.44.mlp.experts.94.gate_proj.weight": "model-00048-of-00101.safetensors", + "model.layers.44.mlp.experts.94.up_proj.weight": "model-00048-of-00101.safetensors", + "model.layers.44.mlp.experts.94.down_proj.weight": "model-00048-of-00101.safetensors", + "model.layers.44.mlp.experts.95.gate_proj.weight": "model-00048-of-00101.safetensors", + "model.layers.44.mlp.experts.95.up_proj.weight": "model-00048-of-00101.safetensors", + "model.layers.44.mlp.experts.95.down_proj.weight": "model-00048-of-00101.safetensors", + "model.layers.44.mlp.experts.96.gate_proj.weight": "model-00048-of-00101.safetensors", + "model.layers.44.mlp.experts.96.up_proj.weight": "model-00048-of-00101.safetensors", + "model.layers.44.mlp.experts.96.down_proj.weight": "model-00048-of-00101.safetensors", + "model.layers.44.mlp.experts.97.gate_proj.weight": "model-00048-of-00101.safetensors", + "model.layers.44.mlp.experts.97.up_proj.weight": "model-00048-of-00101.safetensors", + "model.layers.44.mlp.experts.97.down_proj.weight": "model-00048-of-00101.safetensors", + "model.layers.44.mlp.experts.98.gate_proj.weight": "model-00048-of-00101.safetensors", + "model.layers.44.mlp.experts.98.up_proj.weight": "model-00048-of-00101.safetensors", + "model.layers.44.mlp.experts.98.down_proj.weight": "model-00048-of-00101.safetensors", + "model.layers.44.mlp.experts.99.gate_proj.weight": "model-00048-of-00101.safetensors", + "model.layers.44.mlp.experts.99.up_proj.weight": "model-00048-of-00101.safetensors", + "model.layers.44.mlp.experts.99.down_proj.weight": "model-00048-of-00101.safetensors", + "model.layers.44.mlp.experts.100.gate_proj.weight": "model-00048-of-00101.safetensors", + "model.layers.44.mlp.experts.100.up_proj.weight": "model-00048-of-00101.safetensors", + "model.layers.44.mlp.experts.100.down_proj.weight": "model-00048-of-00101.safetensors", + "model.layers.44.mlp.experts.101.gate_proj.weight": "model-00048-of-00101.safetensors", + "model.layers.44.mlp.experts.101.up_proj.weight": "model-00048-of-00101.safetensors", + "model.layers.44.mlp.experts.101.down_proj.weight": "model-00048-of-00101.safetensors", + "model.layers.44.mlp.experts.102.gate_proj.weight": "model-00048-of-00101.safetensors", + "model.layers.44.mlp.experts.102.up_proj.weight": "model-00048-of-00101.safetensors", + "model.layers.44.mlp.experts.102.down_proj.weight": "model-00048-of-00101.safetensors", + "model.layers.44.mlp.experts.103.gate_proj.weight": "model-00048-of-00101.safetensors", + "model.layers.44.mlp.experts.103.up_proj.weight": "model-00048-of-00101.safetensors", + "model.layers.44.mlp.experts.103.down_proj.weight": "model-00048-of-00101.safetensors", + "model.layers.44.mlp.experts.104.gate_proj.weight": "model-00048-of-00101.safetensors", + "model.layers.44.mlp.experts.104.up_proj.weight": "model-00048-of-00101.safetensors", + "model.layers.44.mlp.experts.104.down_proj.weight": "model-00048-of-00101.safetensors", + "model.layers.44.mlp.experts.105.gate_proj.weight": "model-00048-of-00101.safetensors", + "model.layers.44.mlp.experts.105.up_proj.weight": "model-00048-of-00101.safetensors", + "model.layers.44.mlp.experts.105.down_proj.weight": "model-00048-of-00101.safetensors", + "model.layers.44.mlp.experts.106.gate_proj.weight": "model-00048-of-00101.safetensors", + "model.layers.44.mlp.experts.106.up_proj.weight": "model-00048-of-00101.safetensors", + "model.layers.44.mlp.experts.106.down_proj.weight": "model-00048-of-00101.safetensors", + "model.layers.44.mlp.experts.107.gate_proj.weight": "model-00048-of-00101.safetensors", + "model.layers.44.mlp.experts.107.up_proj.weight": "model-00048-of-00101.safetensors", + "model.layers.44.mlp.experts.107.down_proj.weight": "model-00048-of-00101.safetensors", + "model.layers.44.mlp.experts.108.gate_proj.weight": "model-00048-of-00101.safetensors", + "model.layers.44.mlp.experts.108.up_proj.weight": "model-00048-of-00101.safetensors", + "model.layers.44.mlp.experts.108.down_proj.weight": "model-00048-of-00101.safetensors", + "model.layers.44.mlp.experts.109.gate_proj.weight": "model-00048-of-00101.safetensors", + "model.layers.44.mlp.experts.109.up_proj.weight": "model-00048-of-00101.safetensors", + "model.layers.44.mlp.experts.109.down_proj.weight": "model-00048-of-00101.safetensors", + "model.layers.44.mlp.experts.110.gate_proj.weight": "model-00048-of-00101.safetensors", + "model.layers.44.mlp.experts.110.up_proj.weight": "model-00048-of-00101.safetensors", + "model.layers.44.mlp.experts.110.down_proj.weight": "model-00048-of-00101.safetensors", + "model.layers.44.mlp.experts.111.gate_proj.weight": "model-00048-of-00101.safetensors", + "model.layers.44.mlp.experts.111.up_proj.weight": "model-00048-of-00101.safetensors", + "model.layers.44.mlp.experts.111.down_proj.weight": "model-00048-of-00101.safetensors", + "model.layers.44.mlp.experts.112.gate_proj.weight": "model-00048-of-00101.safetensors", + "model.layers.44.mlp.experts.112.up_proj.weight": "model-00048-of-00101.safetensors", + "model.layers.44.mlp.experts.112.down_proj.weight": "model-00048-of-00101.safetensors", + "model.layers.44.mlp.experts.113.gate_proj.weight": "model-00048-of-00101.safetensors", + "model.layers.44.mlp.experts.113.up_proj.weight": "model-00048-of-00101.safetensors", + "model.layers.44.mlp.experts.113.down_proj.weight": "model-00048-of-00101.safetensors", + "model.layers.44.mlp.experts.114.gate_proj.weight": "model-00048-of-00101.safetensors", + "model.layers.44.mlp.experts.114.up_proj.weight": "model-00048-of-00101.safetensors", + "model.layers.44.mlp.experts.114.down_proj.weight": "model-00048-of-00101.safetensors", + "model.layers.44.mlp.experts.115.gate_proj.weight": "model-00048-of-00101.safetensors", + "model.layers.44.mlp.experts.115.up_proj.weight": "model-00048-of-00101.safetensors", + "model.layers.44.mlp.experts.115.down_proj.weight": "model-00048-of-00101.safetensors", + "model.layers.44.mlp.experts.116.gate_proj.weight": "model-00048-of-00101.safetensors", + "model.layers.44.mlp.experts.116.up_proj.weight": "model-00048-of-00101.safetensors", + "model.layers.44.mlp.experts.116.down_proj.weight": "model-00048-of-00101.safetensors", + "model.layers.44.mlp.experts.117.gate_proj.weight": "model-00048-of-00101.safetensors", + "model.layers.44.mlp.experts.117.up_proj.weight": "model-00048-of-00101.safetensors", + "model.layers.44.mlp.experts.117.down_proj.weight": "model-00048-of-00101.safetensors", + "model.layers.44.mlp.experts.118.gate_proj.weight": "model-00048-of-00101.safetensors", + "model.layers.44.mlp.experts.118.up_proj.weight": "model-00048-of-00101.safetensors", + "model.layers.44.mlp.experts.118.down_proj.weight": "model-00048-of-00101.safetensors", + "model.layers.44.mlp.experts.119.gate_proj.weight": "model-00048-of-00101.safetensors", + "model.layers.44.mlp.experts.119.up_proj.weight": "model-00048-of-00101.safetensors", + "model.layers.44.mlp.experts.119.down_proj.weight": "model-00048-of-00101.safetensors", + "model.layers.44.mlp.gate.weight": "model-00048-of-00101.safetensors", + "model.layers.44.mlp.gate.e_score_correction_bias": "model-00048-of-00101.safetensors", + "model.layers.44.mlp.shared_experts.gate_proj.weight": "model-00048-of-00101.safetensors", + "model.layers.44.mlp.shared_experts.up_proj.weight": "model-00048-of-00101.safetensors", + "model.layers.44.mlp.shared_experts.down_proj.weight": "model-00048-of-00101.safetensors", + "model.layers.44.input_layernorm.weight": "model-00048-of-00101.safetensors", + "model.layers.44.post_attention_layernorm.weight": "model-00048-of-00101.safetensors", + "model.layers.45.self_attn.q_proj.weight": "model-00048-of-00101.safetensors", + "model.layers.45.self_attn.q_proj.bias": "model-00048-of-00101.safetensors", + "model.layers.45.self_attn.k_proj.weight": "model-00048-of-00101.safetensors", + "model.layers.45.self_attn.k_proj.bias": "model-00048-of-00101.safetensors", + "model.layers.45.self_attn.v_proj.weight": "model-00048-of-00101.safetensors", + "model.layers.45.self_attn.v_proj.bias": "model-00048-of-00101.safetensors", + "model.layers.45.self_attn.o_proj.weight": "model-00048-of-00101.safetensors", + "model.layers.45.self_attn.q_norm.weight": "model-00048-of-00101.safetensors", + "model.layers.45.self_attn.k_norm.weight": "model-00048-of-00101.safetensors", + "model.layers.45.mlp.experts.0.gate_proj.weight": "model-00048-of-00101.safetensors", + "model.layers.45.mlp.experts.0.up_proj.weight": "model-00048-of-00101.safetensors", + "model.layers.45.mlp.experts.0.down_proj.weight": "model-00048-of-00101.safetensors", + "model.layers.45.mlp.experts.1.gate_proj.weight": "model-00048-of-00101.safetensors", + "model.layers.45.mlp.experts.1.up_proj.weight": "model-00048-of-00101.safetensors", + "model.layers.45.mlp.experts.1.down_proj.weight": "model-00048-of-00101.safetensors", + "model.layers.45.mlp.experts.2.gate_proj.weight": "model-00048-of-00101.safetensors", + "model.layers.45.mlp.experts.2.up_proj.weight": "model-00048-of-00101.safetensors", + "model.layers.45.mlp.experts.2.down_proj.weight": "model-00048-of-00101.safetensors", + "model.layers.45.mlp.experts.3.gate_proj.weight": "model-00048-of-00101.safetensors", + "model.layers.45.mlp.experts.3.up_proj.weight": "model-00048-of-00101.safetensors", + "model.layers.45.mlp.experts.3.down_proj.weight": "model-00048-of-00101.safetensors", + "model.layers.45.mlp.experts.4.gate_proj.weight": "model-00048-of-00101.safetensors", + "model.layers.45.mlp.experts.4.up_proj.weight": "model-00048-of-00101.safetensors", + "model.layers.45.mlp.experts.4.down_proj.weight": "model-00048-of-00101.safetensors", + "model.layers.45.mlp.experts.5.gate_proj.weight": "model-00048-of-00101.safetensors", + "model.layers.45.mlp.experts.5.up_proj.weight": "model-00048-of-00101.safetensors", + "model.layers.45.mlp.experts.5.down_proj.weight": "model-00048-of-00101.safetensors", + "model.layers.45.mlp.experts.6.gate_proj.weight": "model-00048-of-00101.safetensors", + "model.layers.45.mlp.experts.6.up_proj.weight": "model-00048-of-00101.safetensors", + "model.layers.45.mlp.experts.6.down_proj.weight": "model-00048-of-00101.safetensors", + "model.layers.45.mlp.experts.7.gate_proj.weight": "model-00048-of-00101.safetensors", + "model.layers.45.mlp.experts.7.up_proj.weight": "model-00048-of-00101.safetensors", + "model.layers.45.mlp.experts.7.down_proj.weight": "model-00048-of-00101.safetensors", + "model.layers.45.mlp.experts.8.gate_proj.weight": "model-00048-of-00101.safetensors", + "model.layers.45.mlp.experts.8.up_proj.weight": "model-00048-of-00101.safetensors", + "model.layers.45.mlp.experts.8.down_proj.weight": "model-00048-of-00101.safetensors", + "model.layers.45.mlp.experts.9.gate_proj.weight": "model-00048-of-00101.safetensors", + "model.layers.45.mlp.experts.9.up_proj.weight": "model-00048-of-00101.safetensors", + "model.layers.45.mlp.experts.9.down_proj.weight": "model-00048-of-00101.safetensors", + "model.layers.45.mlp.experts.10.gate_proj.weight": "model-00048-of-00101.safetensors", + "model.layers.45.mlp.experts.10.up_proj.weight": "model-00048-of-00101.safetensors", + "model.layers.45.mlp.experts.10.down_proj.weight": "model-00048-of-00101.safetensors", + "model.layers.45.mlp.experts.11.gate_proj.weight": "model-00048-of-00101.safetensors", + "model.layers.45.mlp.experts.11.up_proj.weight": "model-00048-of-00101.safetensors", + "model.layers.45.mlp.experts.11.down_proj.weight": "model-00048-of-00101.safetensors", + "model.layers.45.mlp.experts.12.gate_proj.weight": "model-00048-of-00101.safetensors", + "model.layers.45.mlp.experts.12.up_proj.weight": "model-00048-of-00101.safetensors", + "model.layers.45.mlp.experts.12.down_proj.weight": "model-00048-of-00101.safetensors", + "model.layers.45.mlp.experts.13.gate_proj.weight": "model-00048-of-00101.safetensors", + "model.layers.45.mlp.experts.13.up_proj.weight": "model-00048-of-00101.safetensors", + "model.layers.45.mlp.experts.13.down_proj.weight": "model-00048-of-00101.safetensors", + "model.layers.45.mlp.experts.14.gate_proj.weight": "model-00048-of-00101.safetensors", + "model.layers.45.mlp.experts.14.up_proj.weight": "model-00048-of-00101.safetensors", + "model.layers.45.mlp.experts.14.down_proj.weight": "model-00048-of-00101.safetensors", + "model.layers.45.mlp.experts.15.gate_proj.weight": "model-00048-of-00101.safetensors", + "model.layers.45.mlp.experts.15.up_proj.weight": "model-00048-of-00101.safetensors", + "model.layers.45.mlp.experts.15.down_proj.weight": "model-00048-of-00101.safetensors", + "model.layers.45.mlp.experts.16.gate_proj.weight": "model-00048-of-00101.safetensors", + "model.layers.45.mlp.experts.16.up_proj.weight": "model-00048-of-00101.safetensors", + "model.layers.45.mlp.experts.16.down_proj.weight": "model-00048-of-00101.safetensors", + "model.layers.45.mlp.experts.17.gate_proj.weight": "model-00048-of-00101.safetensors", + "model.layers.45.mlp.experts.17.up_proj.weight": "model-00048-of-00101.safetensors", + "model.layers.45.mlp.experts.17.down_proj.weight": "model-00048-of-00101.safetensors", + "model.layers.45.mlp.experts.18.gate_proj.weight": "model-00048-of-00101.safetensors", + "model.layers.45.mlp.experts.18.up_proj.weight": "model-00048-of-00101.safetensors", + "model.layers.45.mlp.experts.18.down_proj.weight": "model-00048-of-00101.safetensors", + "model.layers.45.mlp.experts.19.gate_proj.weight": "model-00048-of-00101.safetensors", + "model.layers.45.mlp.experts.19.up_proj.weight": "model-00048-of-00101.safetensors", + "model.layers.45.mlp.experts.19.down_proj.weight": "model-00048-of-00101.safetensors", + "model.layers.45.mlp.experts.20.gate_proj.weight": "model-00048-of-00101.safetensors", + "model.layers.45.mlp.experts.20.up_proj.weight": "model-00048-of-00101.safetensors", + "model.layers.45.mlp.experts.20.down_proj.weight": "model-00048-of-00101.safetensors", + "model.layers.45.mlp.experts.21.gate_proj.weight": "model-00048-of-00101.safetensors", + "model.layers.45.mlp.experts.21.up_proj.weight": "model-00048-of-00101.safetensors", + "model.layers.45.mlp.experts.21.down_proj.weight": "model-00048-of-00101.safetensors", + "model.layers.45.mlp.experts.22.gate_proj.weight": "model-00048-of-00101.safetensors", + "model.layers.45.mlp.experts.22.up_proj.weight": "model-00048-of-00101.safetensors", + "model.layers.45.mlp.experts.22.down_proj.weight": "model-00048-of-00101.safetensors", + "model.layers.45.mlp.experts.23.gate_proj.weight": "model-00048-of-00101.safetensors", + "model.layers.45.mlp.experts.23.up_proj.weight": "model-00048-of-00101.safetensors", + "model.layers.45.mlp.experts.23.down_proj.weight": "model-00048-of-00101.safetensors", + "model.layers.45.mlp.experts.24.gate_proj.weight": "model-00048-of-00101.safetensors", + "model.layers.45.mlp.experts.24.up_proj.weight": "model-00048-of-00101.safetensors", + "model.layers.45.mlp.experts.24.down_proj.weight": "model-00048-of-00101.safetensors", + "model.layers.45.mlp.experts.25.gate_proj.weight": "model-00048-of-00101.safetensors", + "model.layers.45.mlp.experts.25.up_proj.weight": "model-00048-of-00101.safetensors", + "model.layers.45.mlp.experts.25.down_proj.weight": "model-00048-of-00101.safetensors", + "model.layers.45.mlp.experts.26.gate_proj.weight": "model-00048-of-00101.safetensors", + "model.layers.45.mlp.experts.26.up_proj.weight": "model-00048-of-00101.safetensors", + "model.layers.45.mlp.experts.26.down_proj.weight": "model-00048-of-00101.safetensors", + "model.layers.45.mlp.experts.27.gate_proj.weight": "model-00048-of-00101.safetensors", + "model.layers.45.mlp.experts.27.up_proj.weight": "model-00048-of-00101.safetensors", + "model.layers.45.mlp.experts.27.down_proj.weight": "model-00048-of-00101.safetensors", + "model.layers.45.mlp.experts.28.gate_proj.weight": "model-00048-of-00101.safetensors", + "model.layers.45.mlp.experts.28.up_proj.weight": "model-00048-of-00101.safetensors", + "model.layers.45.mlp.experts.28.down_proj.weight": "model-00048-of-00101.safetensors", + "model.layers.45.mlp.experts.29.gate_proj.weight": "model-00048-of-00101.safetensors", + "model.layers.45.mlp.experts.29.up_proj.weight": "model-00048-of-00101.safetensors", + "model.layers.45.mlp.experts.29.down_proj.weight": "model-00048-of-00101.safetensors", + "model.layers.45.mlp.experts.30.gate_proj.weight": "model-00048-of-00101.safetensors", + "model.layers.45.mlp.experts.30.up_proj.weight": "model-00048-of-00101.safetensors", + "model.layers.45.mlp.experts.30.down_proj.weight": "model-00048-of-00101.safetensors", + "model.layers.45.mlp.experts.31.gate_proj.weight": "model-00048-of-00101.safetensors", + "model.layers.45.mlp.experts.31.up_proj.weight": "model-00048-of-00101.safetensors", + "model.layers.45.mlp.experts.31.down_proj.weight": "model-00048-of-00101.safetensors", + "model.layers.45.mlp.experts.32.gate_proj.weight": "model-00048-of-00101.safetensors", + "model.layers.45.mlp.experts.32.up_proj.weight": "model-00048-of-00101.safetensors", + "model.layers.45.mlp.experts.32.down_proj.weight": "model-00048-of-00101.safetensors", + "model.layers.45.mlp.experts.33.gate_proj.weight": "model-00048-of-00101.safetensors", + "model.layers.45.mlp.experts.33.up_proj.weight": "model-00048-of-00101.safetensors", + "model.layers.45.mlp.experts.33.down_proj.weight": "model-00048-of-00101.safetensors", + "model.layers.45.mlp.experts.34.gate_proj.weight": "model-00048-of-00101.safetensors", + "model.layers.45.mlp.experts.34.up_proj.weight": "model-00048-of-00101.safetensors", + "model.layers.45.mlp.experts.34.down_proj.weight": "model-00048-of-00101.safetensors", + "model.layers.45.mlp.experts.35.gate_proj.weight": "model-00048-of-00101.safetensors", + "model.layers.45.mlp.experts.35.up_proj.weight": "model-00048-of-00101.safetensors", + "model.layers.45.mlp.experts.35.down_proj.weight": "model-00048-of-00101.safetensors", + "model.layers.45.mlp.experts.36.gate_proj.weight": "model-00048-of-00101.safetensors", + "model.layers.45.mlp.experts.36.up_proj.weight": "model-00048-of-00101.safetensors", + "model.layers.45.mlp.experts.36.down_proj.weight": "model-00048-of-00101.safetensors", + "model.layers.45.mlp.experts.37.gate_proj.weight": "model-00048-of-00101.safetensors", + "model.layers.45.mlp.experts.37.up_proj.weight": "model-00048-of-00101.safetensors", + "model.layers.45.mlp.experts.37.down_proj.weight": "model-00048-of-00101.safetensors", + "model.layers.45.mlp.experts.38.gate_proj.weight": "model-00048-of-00101.safetensors", + "model.layers.45.mlp.experts.38.up_proj.weight": "model-00048-of-00101.safetensors", + "model.layers.45.mlp.experts.38.down_proj.weight": "model-00048-of-00101.safetensors", + "model.layers.45.mlp.experts.39.gate_proj.weight": "model-00048-of-00101.safetensors", + "model.layers.45.mlp.experts.39.up_proj.weight": "model-00048-of-00101.safetensors", + "model.layers.45.mlp.experts.39.down_proj.weight": "model-00048-of-00101.safetensors", + "model.layers.45.mlp.experts.40.gate_proj.weight": "model-00048-of-00101.safetensors", + "model.layers.45.mlp.experts.40.up_proj.weight": "model-00048-of-00101.safetensors", + "model.layers.45.mlp.experts.40.down_proj.weight": "model-00048-of-00101.safetensors", + "model.layers.45.mlp.experts.41.gate_proj.weight": "model-00049-of-00101.safetensors", + "model.layers.45.mlp.experts.41.up_proj.weight": "model-00049-of-00101.safetensors", + "model.layers.45.mlp.experts.41.down_proj.weight": "model-00049-of-00101.safetensors", + "model.layers.45.mlp.experts.42.gate_proj.weight": "model-00049-of-00101.safetensors", + "model.layers.45.mlp.experts.42.up_proj.weight": "model-00049-of-00101.safetensors", + "model.layers.45.mlp.experts.42.down_proj.weight": "model-00049-of-00101.safetensors", + "model.layers.45.mlp.experts.43.gate_proj.weight": "model-00049-of-00101.safetensors", + "model.layers.45.mlp.experts.43.up_proj.weight": "model-00049-of-00101.safetensors", + "model.layers.45.mlp.experts.43.down_proj.weight": "model-00049-of-00101.safetensors", + "model.layers.45.mlp.experts.44.gate_proj.weight": "model-00049-of-00101.safetensors", + "model.layers.45.mlp.experts.44.up_proj.weight": "model-00049-of-00101.safetensors", + "model.layers.45.mlp.experts.44.down_proj.weight": "model-00049-of-00101.safetensors", + "model.layers.45.mlp.experts.45.gate_proj.weight": "model-00049-of-00101.safetensors", + "model.layers.45.mlp.experts.45.up_proj.weight": "model-00049-of-00101.safetensors", + "model.layers.45.mlp.experts.45.down_proj.weight": "model-00049-of-00101.safetensors", + "model.layers.45.mlp.experts.46.gate_proj.weight": "model-00049-of-00101.safetensors", + "model.layers.45.mlp.experts.46.up_proj.weight": "model-00049-of-00101.safetensors", + "model.layers.45.mlp.experts.46.down_proj.weight": "model-00049-of-00101.safetensors", + "model.layers.45.mlp.experts.47.gate_proj.weight": "model-00049-of-00101.safetensors", + "model.layers.45.mlp.experts.47.up_proj.weight": "model-00049-of-00101.safetensors", + "model.layers.45.mlp.experts.47.down_proj.weight": "model-00049-of-00101.safetensors", + "model.layers.45.mlp.experts.48.gate_proj.weight": "model-00049-of-00101.safetensors", + "model.layers.45.mlp.experts.48.up_proj.weight": "model-00049-of-00101.safetensors", + "model.layers.45.mlp.experts.48.down_proj.weight": "model-00049-of-00101.safetensors", + "model.layers.45.mlp.experts.49.gate_proj.weight": "model-00049-of-00101.safetensors", + "model.layers.45.mlp.experts.49.up_proj.weight": "model-00049-of-00101.safetensors", + "model.layers.45.mlp.experts.49.down_proj.weight": "model-00049-of-00101.safetensors", + "model.layers.45.mlp.experts.50.gate_proj.weight": "model-00049-of-00101.safetensors", + "model.layers.45.mlp.experts.50.up_proj.weight": "model-00049-of-00101.safetensors", + "model.layers.45.mlp.experts.50.down_proj.weight": "model-00049-of-00101.safetensors", + "model.layers.45.mlp.experts.51.gate_proj.weight": "model-00049-of-00101.safetensors", + "model.layers.45.mlp.experts.51.up_proj.weight": "model-00049-of-00101.safetensors", + "model.layers.45.mlp.experts.51.down_proj.weight": "model-00049-of-00101.safetensors", + "model.layers.45.mlp.experts.52.gate_proj.weight": "model-00049-of-00101.safetensors", + "model.layers.45.mlp.experts.52.up_proj.weight": "model-00049-of-00101.safetensors", + "model.layers.45.mlp.experts.52.down_proj.weight": "model-00049-of-00101.safetensors", + "model.layers.45.mlp.experts.53.gate_proj.weight": "model-00049-of-00101.safetensors", + "model.layers.45.mlp.experts.53.up_proj.weight": "model-00049-of-00101.safetensors", + "model.layers.45.mlp.experts.53.down_proj.weight": "model-00049-of-00101.safetensors", + "model.layers.45.mlp.experts.54.gate_proj.weight": "model-00049-of-00101.safetensors", + "model.layers.45.mlp.experts.54.up_proj.weight": "model-00049-of-00101.safetensors", + "model.layers.45.mlp.experts.54.down_proj.weight": "model-00049-of-00101.safetensors", + "model.layers.45.mlp.experts.55.gate_proj.weight": "model-00049-of-00101.safetensors", + "model.layers.45.mlp.experts.55.up_proj.weight": "model-00049-of-00101.safetensors", + "model.layers.45.mlp.experts.55.down_proj.weight": "model-00049-of-00101.safetensors", + "model.layers.45.mlp.experts.56.gate_proj.weight": "model-00049-of-00101.safetensors", + "model.layers.45.mlp.experts.56.up_proj.weight": "model-00049-of-00101.safetensors", + "model.layers.45.mlp.experts.56.down_proj.weight": "model-00049-of-00101.safetensors", + "model.layers.45.mlp.experts.57.gate_proj.weight": "model-00049-of-00101.safetensors", + "model.layers.45.mlp.experts.57.up_proj.weight": "model-00049-of-00101.safetensors", + "model.layers.45.mlp.experts.57.down_proj.weight": "model-00049-of-00101.safetensors", + "model.layers.45.mlp.experts.58.gate_proj.weight": "model-00049-of-00101.safetensors", + "model.layers.45.mlp.experts.58.up_proj.weight": "model-00049-of-00101.safetensors", + "model.layers.45.mlp.experts.58.down_proj.weight": "model-00049-of-00101.safetensors", + "model.layers.45.mlp.experts.59.gate_proj.weight": "model-00049-of-00101.safetensors", + "model.layers.45.mlp.experts.59.up_proj.weight": "model-00049-of-00101.safetensors", + "model.layers.45.mlp.experts.59.down_proj.weight": "model-00049-of-00101.safetensors", + "model.layers.45.mlp.experts.60.gate_proj.weight": "model-00049-of-00101.safetensors", + "model.layers.45.mlp.experts.60.up_proj.weight": "model-00049-of-00101.safetensors", + "model.layers.45.mlp.experts.60.down_proj.weight": "model-00049-of-00101.safetensors", + "model.layers.45.mlp.experts.61.gate_proj.weight": "model-00049-of-00101.safetensors", + "model.layers.45.mlp.experts.61.up_proj.weight": "model-00049-of-00101.safetensors", + "model.layers.45.mlp.experts.61.down_proj.weight": "model-00049-of-00101.safetensors", + "model.layers.45.mlp.experts.62.gate_proj.weight": "model-00049-of-00101.safetensors", + "model.layers.45.mlp.experts.62.up_proj.weight": "model-00049-of-00101.safetensors", + "model.layers.45.mlp.experts.62.down_proj.weight": "model-00049-of-00101.safetensors", + "model.layers.45.mlp.experts.63.gate_proj.weight": "model-00049-of-00101.safetensors", + "model.layers.45.mlp.experts.63.up_proj.weight": "model-00049-of-00101.safetensors", + "model.layers.45.mlp.experts.63.down_proj.weight": "model-00049-of-00101.safetensors", + "model.layers.45.mlp.experts.64.gate_proj.weight": "model-00049-of-00101.safetensors", + "model.layers.45.mlp.experts.64.up_proj.weight": "model-00049-of-00101.safetensors", + "model.layers.45.mlp.experts.64.down_proj.weight": "model-00049-of-00101.safetensors", + "model.layers.45.mlp.experts.65.gate_proj.weight": "model-00049-of-00101.safetensors", + "model.layers.45.mlp.experts.65.up_proj.weight": "model-00049-of-00101.safetensors", + "model.layers.45.mlp.experts.65.down_proj.weight": "model-00049-of-00101.safetensors", + "model.layers.45.mlp.experts.66.gate_proj.weight": "model-00049-of-00101.safetensors", + "model.layers.45.mlp.experts.66.up_proj.weight": "model-00049-of-00101.safetensors", + "model.layers.45.mlp.experts.66.down_proj.weight": "model-00049-of-00101.safetensors", + "model.layers.45.mlp.experts.67.gate_proj.weight": "model-00049-of-00101.safetensors", + "model.layers.45.mlp.experts.67.up_proj.weight": "model-00049-of-00101.safetensors", + "model.layers.45.mlp.experts.67.down_proj.weight": "model-00049-of-00101.safetensors", + "model.layers.45.mlp.experts.68.gate_proj.weight": "model-00049-of-00101.safetensors", + "model.layers.45.mlp.experts.68.up_proj.weight": "model-00049-of-00101.safetensors", + "model.layers.45.mlp.experts.68.down_proj.weight": "model-00049-of-00101.safetensors", + "model.layers.45.mlp.experts.69.gate_proj.weight": "model-00049-of-00101.safetensors", + "model.layers.45.mlp.experts.69.up_proj.weight": "model-00049-of-00101.safetensors", + "model.layers.45.mlp.experts.69.down_proj.weight": "model-00049-of-00101.safetensors", + "model.layers.45.mlp.experts.70.gate_proj.weight": "model-00049-of-00101.safetensors", + "model.layers.45.mlp.experts.70.up_proj.weight": "model-00049-of-00101.safetensors", + "model.layers.45.mlp.experts.70.down_proj.weight": "model-00049-of-00101.safetensors", + "model.layers.45.mlp.experts.71.gate_proj.weight": "model-00049-of-00101.safetensors", + "model.layers.45.mlp.experts.71.up_proj.weight": "model-00049-of-00101.safetensors", + "model.layers.45.mlp.experts.71.down_proj.weight": "model-00049-of-00101.safetensors", + "model.layers.45.mlp.experts.72.gate_proj.weight": "model-00049-of-00101.safetensors", + "model.layers.45.mlp.experts.72.up_proj.weight": "model-00049-of-00101.safetensors", + "model.layers.45.mlp.experts.72.down_proj.weight": "model-00049-of-00101.safetensors", + "model.layers.45.mlp.experts.73.gate_proj.weight": "model-00049-of-00101.safetensors", + "model.layers.45.mlp.experts.73.up_proj.weight": "model-00049-of-00101.safetensors", + "model.layers.45.mlp.experts.73.down_proj.weight": "model-00049-of-00101.safetensors", + "model.layers.45.mlp.experts.74.gate_proj.weight": "model-00049-of-00101.safetensors", + "model.layers.45.mlp.experts.74.up_proj.weight": "model-00049-of-00101.safetensors", + "model.layers.45.mlp.experts.74.down_proj.weight": "model-00049-of-00101.safetensors", + "model.layers.45.mlp.experts.75.gate_proj.weight": "model-00049-of-00101.safetensors", + "model.layers.45.mlp.experts.75.up_proj.weight": "model-00049-of-00101.safetensors", + "model.layers.45.mlp.experts.75.down_proj.weight": "model-00049-of-00101.safetensors", + "model.layers.45.mlp.experts.76.gate_proj.weight": "model-00049-of-00101.safetensors", + "model.layers.45.mlp.experts.76.up_proj.weight": "model-00049-of-00101.safetensors", + "model.layers.45.mlp.experts.76.down_proj.weight": "model-00049-of-00101.safetensors", + "model.layers.45.mlp.experts.77.gate_proj.weight": "model-00049-of-00101.safetensors", + "model.layers.45.mlp.experts.77.up_proj.weight": "model-00049-of-00101.safetensors", + "model.layers.45.mlp.experts.77.down_proj.weight": "model-00049-of-00101.safetensors", + "model.layers.45.mlp.experts.78.gate_proj.weight": "model-00049-of-00101.safetensors", + "model.layers.45.mlp.experts.78.up_proj.weight": "model-00049-of-00101.safetensors", + "model.layers.45.mlp.experts.78.down_proj.weight": "model-00049-of-00101.safetensors", + "model.layers.45.mlp.experts.79.gate_proj.weight": "model-00049-of-00101.safetensors", + "model.layers.45.mlp.experts.79.up_proj.weight": "model-00049-of-00101.safetensors", + "model.layers.45.mlp.experts.79.down_proj.weight": "model-00049-of-00101.safetensors", + "model.layers.45.mlp.experts.80.gate_proj.weight": "model-00049-of-00101.safetensors", + "model.layers.45.mlp.experts.80.up_proj.weight": "model-00049-of-00101.safetensors", + "model.layers.45.mlp.experts.80.down_proj.weight": "model-00049-of-00101.safetensors", + "model.layers.45.mlp.experts.81.gate_proj.weight": "model-00049-of-00101.safetensors", + "model.layers.45.mlp.experts.81.up_proj.weight": "model-00049-of-00101.safetensors", + "model.layers.45.mlp.experts.81.down_proj.weight": "model-00049-of-00101.safetensors", + "model.layers.45.mlp.experts.82.gate_proj.weight": "model-00049-of-00101.safetensors", + "model.layers.45.mlp.experts.82.up_proj.weight": "model-00049-of-00101.safetensors", + "model.layers.45.mlp.experts.82.down_proj.weight": "model-00049-of-00101.safetensors", + "model.layers.45.mlp.experts.83.gate_proj.weight": "model-00049-of-00101.safetensors", + "model.layers.45.mlp.experts.83.up_proj.weight": "model-00049-of-00101.safetensors", + "model.layers.45.mlp.experts.83.down_proj.weight": "model-00049-of-00101.safetensors", + "model.layers.45.mlp.experts.84.gate_proj.weight": "model-00049-of-00101.safetensors", + "model.layers.45.mlp.experts.84.up_proj.weight": "model-00049-of-00101.safetensors", + "model.layers.45.mlp.experts.84.down_proj.weight": "model-00049-of-00101.safetensors", + "model.layers.45.mlp.experts.85.gate_proj.weight": "model-00049-of-00101.safetensors", + "model.layers.45.mlp.experts.85.up_proj.weight": "model-00049-of-00101.safetensors", + "model.layers.45.mlp.experts.85.down_proj.weight": "model-00049-of-00101.safetensors", + "model.layers.45.mlp.experts.86.gate_proj.weight": "model-00049-of-00101.safetensors", + "model.layers.45.mlp.experts.86.up_proj.weight": "model-00049-of-00101.safetensors", + "model.layers.45.mlp.experts.86.down_proj.weight": "model-00049-of-00101.safetensors", + "model.layers.45.mlp.experts.87.gate_proj.weight": "model-00049-of-00101.safetensors", + "model.layers.45.mlp.experts.87.up_proj.weight": "model-00049-of-00101.safetensors", + "model.layers.45.mlp.experts.87.down_proj.weight": "model-00049-of-00101.safetensors", + "model.layers.45.mlp.experts.88.gate_proj.weight": "model-00049-of-00101.safetensors", + "model.layers.45.mlp.experts.88.up_proj.weight": "model-00049-of-00101.safetensors", + "model.layers.45.mlp.experts.88.down_proj.weight": "model-00049-of-00101.safetensors", + "model.layers.45.mlp.experts.89.gate_proj.weight": "model-00049-of-00101.safetensors", + "model.layers.45.mlp.experts.89.up_proj.weight": "model-00049-of-00101.safetensors", + "model.layers.45.mlp.experts.89.down_proj.weight": "model-00049-of-00101.safetensors", + "model.layers.45.mlp.experts.90.gate_proj.weight": "model-00049-of-00101.safetensors", + "model.layers.45.mlp.experts.90.up_proj.weight": "model-00049-of-00101.safetensors", + "model.layers.45.mlp.experts.90.down_proj.weight": "model-00049-of-00101.safetensors", + "model.layers.45.mlp.experts.91.gate_proj.weight": "model-00049-of-00101.safetensors", + "model.layers.45.mlp.experts.91.up_proj.weight": "model-00049-of-00101.safetensors", + "model.layers.45.mlp.experts.91.down_proj.weight": "model-00049-of-00101.safetensors", + "model.layers.45.mlp.experts.92.gate_proj.weight": "model-00049-of-00101.safetensors", + "model.layers.45.mlp.experts.92.up_proj.weight": "model-00049-of-00101.safetensors", + "model.layers.45.mlp.experts.92.down_proj.weight": "model-00049-of-00101.safetensors", + "model.layers.45.mlp.experts.93.gate_proj.weight": "model-00049-of-00101.safetensors", + "model.layers.45.mlp.experts.93.up_proj.weight": "model-00049-of-00101.safetensors", + "model.layers.45.mlp.experts.93.down_proj.weight": "model-00049-of-00101.safetensors", + "model.layers.45.mlp.experts.94.gate_proj.weight": "model-00049-of-00101.safetensors", + "model.layers.45.mlp.experts.94.up_proj.weight": "model-00049-of-00101.safetensors", + "model.layers.45.mlp.experts.94.down_proj.weight": "model-00049-of-00101.safetensors", + "model.layers.45.mlp.experts.95.gate_proj.weight": "model-00049-of-00101.safetensors", + "model.layers.45.mlp.experts.95.up_proj.weight": "model-00049-of-00101.safetensors", + "model.layers.45.mlp.experts.95.down_proj.weight": "model-00049-of-00101.safetensors", + "model.layers.45.mlp.experts.96.gate_proj.weight": "model-00049-of-00101.safetensors", + "model.layers.45.mlp.experts.96.up_proj.weight": "model-00049-of-00101.safetensors", + "model.layers.45.mlp.experts.96.down_proj.weight": "model-00049-of-00101.safetensors", + "model.layers.45.mlp.experts.97.gate_proj.weight": "model-00049-of-00101.safetensors", + "model.layers.45.mlp.experts.97.up_proj.weight": "model-00049-of-00101.safetensors", + "model.layers.45.mlp.experts.97.down_proj.weight": "model-00049-of-00101.safetensors", + "model.layers.45.mlp.experts.98.gate_proj.weight": "model-00049-of-00101.safetensors", + "model.layers.45.mlp.experts.98.up_proj.weight": "model-00049-of-00101.safetensors", + "model.layers.45.mlp.experts.98.down_proj.weight": "model-00049-of-00101.safetensors", + "model.layers.45.mlp.experts.99.gate_proj.weight": "model-00049-of-00101.safetensors", + "model.layers.45.mlp.experts.99.up_proj.weight": "model-00049-of-00101.safetensors", + "model.layers.45.mlp.experts.99.down_proj.weight": "model-00049-of-00101.safetensors", + "model.layers.45.mlp.experts.100.gate_proj.weight": "model-00049-of-00101.safetensors", + "model.layers.45.mlp.experts.100.up_proj.weight": "model-00049-of-00101.safetensors", + "model.layers.45.mlp.experts.100.down_proj.weight": "model-00049-of-00101.safetensors", + "model.layers.45.mlp.experts.101.gate_proj.weight": "model-00049-of-00101.safetensors", + "model.layers.45.mlp.experts.101.up_proj.weight": "model-00049-of-00101.safetensors", + "model.layers.45.mlp.experts.101.down_proj.weight": "model-00049-of-00101.safetensors", + "model.layers.45.mlp.experts.102.gate_proj.weight": "model-00049-of-00101.safetensors", + "model.layers.45.mlp.experts.102.up_proj.weight": "model-00049-of-00101.safetensors", + "model.layers.45.mlp.experts.102.down_proj.weight": "model-00049-of-00101.safetensors", + "model.layers.45.mlp.experts.103.gate_proj.weight": "model-00049-of-00101.safetensors", + "model.layers.45.mlp.experts.103.up_proj.weight": "model-00049-of-00101.safetensors", + "model.layers.45.mlp.experts.103.down_proj.weight": "model-00049-of-00101.safetensors", + "model.layers.45.mlp.experts.104.gate_proj.weight": "model-00049-of-00101.safetensors", + "model.layers.45.mlp.experts.104.up_proj.weight": "model-00049-of-00101.safetensors", + "model.layers.45.mlp.experts.104.down_proj.weight": "model-00049-of-00101.safetensors", + "model.layers.45.mlp.experts.105.gate_proj.weight": "model-00049-of-00101.safetensors", + "model.layers.45.mlp.experts.105.up_proj.weight": "model-00049-of-00101.safetensors", + "model.layers.45.mlp.experts.105.down_proj.weight": "model-00049-of-00101.safetensors", + "model.layers.45.mlp.experts.106.gate_proj.weight": "model-00049-of-00101.safetensors", + "model.layers.45.mlp.experts.106.up_proj.weight": "model-00049-of-00101.safetensors", + "model.layers.45.mlp.experts.106.down_proj.weight": "model-00049-of-00101.safetensors", + "model.layers.45.mlp.experts.107.gate_proj.weight": "model-00049-of-00101.safetensors", + "model.layers.45.mlp.experts.107.up_proj.weight": "model-00049-of-00101.safetensors", + "model.layers.45.mlp.experts.107.down_proj.weight": "model-00049-of-00101.safetensors", + "model.layers.45.mlp.experts.108.gate_proj.weight": "model-00049-of-00101.safetensors", + "model.layers.45.mlp.experts.108.up_proj.weight": "model-00049-of-00101.safetensors", + "model.layers.45.mlp.experts.108.down_proj.weight": "model-00049-of-00101.safetensors", + "model.layers.45.mlp.experts.109.gate_proj.weight": "model-00049-of-00101.safetensors", + "model.layers.45.mlp.experts.109.up_proj.weight": "model-00049-of-00101.safetensors", + "model.layers.45.mlp.experts.109.down_proj.weight": "model-00049-of-00101.safetensors", + "model.layers.45.mlp.experts.110.gate_proj.weight": "model-00049-of-00101.safetensors", + "model.layers.45.mlp.experts.110.up_proj.weight": "model-00049-of-00101.safetensors", + "model.layers.45.mlp.experts.110.down_proj.weight": "model-00049-of-00101.safetensors", + "model.layers.45.mlp.experts.111.gate_proj.weight": "model-00049-of-00101.safetensors", + "model.layers.45.mlp.experts.111.up_proj.weight": "model-00049-of-00101.safetensors", + "model.layers.45.mlp.experts.111.down_proj.weight": "model-00049-of-00101.safetensors", + "model.layers.45.mlp.experts.112.gate_proj.weight": "model-00049-of-00101.safetensors", + "model.layers.45.mlp.experts.112.up_proj.weight": "model-00049-of-00101.safetensors", + "model.layers.45.mlp.experts.112.down_proj.weight": "model-00049-of-00101.safetensors", + "model.layers.45.mlp.experts.113.gate_proj.weight": "model-00049-of-00101.safetensors", + "model.layers.45.mlp.experts.113.up_proj.weight": "model-00049-of-00101.safetensors", + "model.layers.45.mlp.experts.113.down_proj.weight": "model-00049-of-00101.safetensors", + "model.layers.45.mlp.experts.114.gate_proj.weight": "model-00049-of-00101.safetensors", + "model.layers.45.mlp.experts.114.up_proj.weight": "model-00049-of-00101.safetensors", + "model.layers.45.mlp.experts.114.down_proj.weight": "model-00049-of-00101.safetensors", + "model.layers.45.mlp.experts.115.gate_proj.weight": "model-00049-of-00101.safetensors", + "model.layers.45.mlp.experts.115.up_proj.weight": "model-00049-of-00101.safetensors", + "model.layers.45.mlp.experts.115.down_proj.weight": "model-00049-of-00101.safetensors", + "model.layers.45.mlp.experts.116.gate_proj.weight": "model-00049-of-00101.safetensors", + "model.layers.45.mlp.experts.116.up_proj.weight": "model-00049-of-00101.safetensors", + "model.layers.45.mlp.experts.116.down_proj.weight": "model-00049-of-00101.safetensors", + "model.layers.45.mlp.experts.117.gate_proj.weight": "model-00049-of-00101.safetensors", + "model.layers.45.mlp.experts.117.up_proj.weight": "model-00049-of-00101.safetensors", + "model.layers.45.mlp.experts.117.down_proj.weight": "model-00049-of-00101.safetensors", + "model.layers.45.mlp.experts.118.gate_proj.weight": "model-00049-of-00101.safetensors", + "model.layers.45.mlp.experts.118.up_proj.weight": "model-00049-of-00101.safetensors", + "model.layers.45.mlp.experts.118.down_proj.weight": "model-00049-of-00101.safetensors", + "model.layers.45.mlp.experts.119.gate_proj.weight": "model-00049-of-00101.safetensors", + "model.layers.45.mlp.experts.119.up_proj.weight": "model-00049-of-00101.safetensors", + "model.layers.45.mlp.experts.119.down_proj.weight": "model-00049-of-00101.safetensors", + "model.layers.45.mlp.gate.weight": "model-00049-of-00101.safetensors", + "model.layers.45.mlp.gate.e_score_correction_bias": "model-00049-of-00101.safetensors", + "model.layers.45.mlp.shared_experts.gate_proj.weight": "model-00049-of-00101.safetensors", + "model.layers.45.mlp.shared_experts.up_proj.weight": "model-00049-of-00101.safetensors", + "model.layers.45.mlp.shared_experts.down_proj.weight": "model-00049-of-00101.safetensors", + "model.layers.45.input_layernorm.weight": "model-00049-of-00101.safetensors", + "model.layers.45.post_attention_layernorm.weight": "model-00049-of-00101.safetensors", + "model.layers.46.self_attn.q_proj.weight": "model-00049-of-00101.safetensors", + "model.layers.46.self_attn.q_proj.bias": "model-00049-of-00101.safetensors", + "model.layers.46.self_attn.k_proj.weight": "model-00049-of-00101.safetensors", + "model.layers.46.self_attn.k_proj.bias": "model-00049-of-00101.safetensors", + "model.layers.46.self_attn.v_proj.weight": "model-00049-of-00101.safetensors", + "model.layers.46.self_attn.v_proj.bias": "model-00049-of-00101.safetensors", + "model.layers.46.self_attn.o_proj.weight": "model-00049-of-00101.safetensors", + "model.layers.46.self_attn.q_norm.weight": "model-00049-of-00101.safetensors", + "model.layers.46.self_attn.k_norm.weight": "model-00049-of-00101.safetensors", + "model.layers.46.mlp.experts.0.gate_proj.weight": "model-00049-of-00101.safetensors", + "model.layers.46.mlp.experts.0.up_proj.weight": "model-00049-of-00101.safetensors", + "model.layers.46.mlp.experts.0.down_proj.weight": "model-00049-of-00101.safetensors", + "model.layers.46.mlp.experts.1.gate_proj.weight": "model-00049-of-00101.safetensors", + "model.layers.46.mlp.experts.1.up_proj.weight": "model-00049-of-00101.safetensors", + "model.layers.46.mlp.experts.1.down_proj.weight": "model-00049-of-00101.safetensors", + "model.layers.46.mlp.experts.2.gate_proj.weight": "model-00049-of-00101.safetensors", + "model.layers.46.mlp.experts.2.up_proj.weight": "model-00049-of-00101.safetensors", + "model.layers.46.mlp.experts.2.down_proj.weight": "model-00049-of-00101.safetensors", + "model.layers.46.mlp.experts.3.gate_proj.weight": "model-00049-of-00101.safetensors", + "model.layers.46.mlp.experts.3.up_proj.weight": "model-00049-of-00101.safetensors", + "model.layers.46.mlp.experts.3.down_proj.weight": "model-00049-of-00101.safetensors", + "model.layers.46.mlp.experts.4.gate_proj.weight": "model-00049-of-00101.safetensors", + "model.layers.46.mlp.experts.4.up_proj.weight": "model-00049-of-00101.safetensors", + "model.layers.46.mlp.experts.4.down_proj.weight": "model-00049-of-00101.safetensors", + "model.layers.46.mlp.experts.5.gate_proj.weight": "model-00049-of-00101.safetensors", + "model.layers.46.mlp.experts.5.up_proj.weight": "model-00049-of-00101.safetensors", + "model.layers.46.mlp.experts.5.down_proj.weight": "model-00049-of-00101.safetensors", + "model.layers.46.mlp.experts.6.gate_proj.weight": "model-00049-of-00101.safetensors", + "model.layers.46.mlp.experts.6.up_proj.weight": "model-00049-of-00101.safetensors", + "model.layers.46.mlp.experts.6.down_proj.weight": "model-00049-of-00101.safetensors", + "model.layers.46.mlp.experts.7.gate_proj.weight": "model-00049-of-00101.safetensors", + "model.layers.46.mlp.experts.7.up_proj.weight": "model-00049-of-00101.safetensors", + "model.layers.46.mlp.experts.7.down_proj.weight": "model-00049-of-00101.safetensors", + "model.layers.46.mlp.experts.8.gate_proj.weight": "model-00049-of-00101.safetensors", + "model.layers.46.mlp.experts.8.up_proj.weight": "model-00049-of-00101.safetensors", + "model.layers.46.mlp.experts.8.down_proj.weight": "model-00049-of-00101.safetensors", + "model.layers.46.mlp.experts.9.gate_proj.weight": "model-00049-of-00101.safetensors", + "model.layers.46.mlp.experts.9.up_proj.weight": "model-00049-of-00101.safetensors", + "model.layers.46.mlp.experts.9.down_proj.weight": "model-00049-of-00101.safetensors", + "model.layers.46.mlp.experts.10.gate_proj.weight": "model-00049-of-00101.safetensors", + "model.layers.46.mlp.experts.10.up_proj.weight": "model-00049-of-00101.safetensors", + "model.layers.46.mlp.experts.10.down_proj.weight": "model-00049-of-00101.safetensors", + "model.layers.46.mlp.experts.11.gate_proj.weight": "model-00049-of-00101.safetensors", + "model.layers.46.mlp.experts.11.up_proj.weight": "model-00049-of-00101.safetensors", + "model.layers.46.mlp.experts.11.down_proj.weight": "model-00049-of-00101.safetensors", + "model.layers.46.mlp.experts.12.gate_proj.weight": "model-00049-of-00101.safetensors", + "model.layers.46.mlp.experts.12.up_proj.weight": "model-00049-of-00101.safetensors", + "model.layers.46.mlp.experts.12.down_proj.weight": "model-00049-of-00101.safetensors", + "model.layers.46.mlp.experts.13.gate_proj.weight": "model-00049-of-00101.safetensors", + "model.layers.46.mlp.experts.13.up_proj.weight": "model-00049-of-00101.safetensors", + "model.layers.46.mlp.experts.13.down_proj.weight": "model-00049-of-00101.safetensors", + "model.layers.46.mlp.experts.14.gate_proj.weight": "model-00049-of-00101.safetensors", + "model.layers.46.mlp.experts.14.up_proj.weight": "model-00049-of-00101.safetensors", + "model.layers.46.mlp.experts.14.down_proj.weight": "model-00049-of-00101.safetensors", + "model.layers.46.mlp.experts.15.gate_proj.weight": "model-00049-of-00101.safetensors", + "model.layers.46.mlp.experts.15.up_proj.weight": "model-00049-of-00101.safetensors", + "model.layers.46.mlp.experts.15.down_proj.weight": "model-00049-of-00101.safetensors", + "model.layers.46.mlp.experts.16.gate_proj.weight": "model-00049-of-00101.safetensors", + "model.layers.46.mlp.experts.16.up_proj.weight": "model-00049-of-00101.safetensors", + "model.layers.46.mlp.experts.16.down_proj.weight": "model-00049-of-00101.safetensors", + "model.layers.46.mlp.experts.17.gate_proj.weight": "model-00049-of-00101.safetensors", + "model.layers.46.mlp.experts.17.up_proj.weight": "model-00049-of-00101.safetensors", + "model.layers.46.mlp.experts.17.down_proj.weight": "model-00049-of-00101.safetensors", + "model.layers.46.mlp.experts.18.gate_proj.weight": "model-00049-of-00101.safetensors", + "model.layers.46.mlp.experts.18.up_proj.weight": "model-00049-of-00101.safetensors", + "model.layers.46.mlp.experts.18.down_proj.weight": "model-00049-of-00101.safetensors", + "model.layers.46.mlp.experts.19.gate_proj.weight": "model-00049-of-00101.safetensors", + "model.layers.46.mlp.experts.19.up_proj.weight": "model-00049-of-00101.safetensors", + "model.layers.46.mlp.experts.19.down_proj.weight": "model-00049-of-00101.safetensors", + "model.layers.46.mlp.experts.20.gate_proj.weight": "model-00049-of-00101.safetensors", + "model.layers.46.mlp.experts.20.up_proj.weight": "model-00049-of-00101.safetensors", + "model.layers.46.mlp.experts.20.down_proj.weight": "model-00049-of-00101.safetensors", + "model.layers.46.mlp.experts.21.gate_proj.weight": "model-00049-of-00101.safetensors", + "model.layers.46.mlp.experts.21.up_proj.weight": "model-00049-of-00101.safetensors", + "model.layers.46.mlp.experts.21.down_proj.weight": "model-00049-of-00101.safetensors", + "model.layers.46.mlp.experts.22.gate_proj.weight": "model-00049-of-00101.safetensors", + "model.layers.46.mlp.experts.22.up_proj.weight": "model-00049-of-00101.safetensors", + "model.layers.46.mlp.experts.22.down_proj.weight": "model-00049-of-00101.safetensors", + "model.layers.46.mlp.experts.23.gate_proj.weight": "model-00049-of-00101.safetensors", + "model.layers.46.mlp.experts.23.up_proj.weight": "model-00049-of-00101.safetensors", + "model.layers.46.mlp.experts.23.down_proj.weight": "model-00049-of-00101.safetensors", + "model.layers.46.mlp.experts.24.gate_proj.weight": "model-00049-of-00101.safetensors", + "model.layers.46.mlp.experts.24.up_proj.weight": "model-00049-of-00101.safetensors", + "model.layers.46.mlp.experts.24.down_proj.weight": "model-00049-of-00101.safetensors", + "model.layers.46.mlp.experts.25.gate_proj.weight": "model-00049-of-00101.safetensors", + "model.layers.46.mlp.experts.25.up_proj.weight": "model-00049-of-00101.safetensors", + "model.layers.46.mlp.experts.25.down_proj.weight": "model-00049-of-00101.safetensors", + "model.layers.46.mlp.experts.26.gate_proj.weight": "model-00049-of-00101.safetensors", + "model.layers.46.mlp.experts.26.up_proj.weight": "model-00049-of-00101.safetensors", + "model.layers.46.mlp.experts.26.down_proj.weight": "model-00049-of-00101.safetensors", + "model.layers.46.mlp.experts.27.gate_proj.weight": "model-00049-of-00101.safetensors", + "model.layers.46.mlp.experts.27.up_proj.weight": "model-00049-of-00101.safetensors", + "model.layers.46.mlp.experts.27.down_proj.weight": "model-00050-of-00101.safetensors", + "model.layers.46.mlp.experts.28.gate_proj.weight": "model-00050-of-00101.safetensors", + "model.layers.46.mlp.experts.28.up_proj.weight": "model-00050-of-00101.safetensors", + "model.layers.46.mlp.experts.28.down_proj.weight": "model-00050-of-00101.safetensors", + "model.layers.46.mlp.experts.29.gate_proj.weight": "model-00050-of-00101.safetensors", + "model.layers.46.mlp.experts.29.up_proj.weight": "model-00050-of-00101.safetensors", + "model.layers.46.mlp.experts.29.down_proj.weight": "model-00050-of-00101.safetensors", + "model.layers.46.mlp.experts.30.gate_proj.weight": "model-00050-of-00101.safetensors", + "model.layers.46.mlp.experts.30.up_proj.weight": "model-00050-of-00101.safetensors", + "model.layers.46.mlp.experts.30.down_proj.weight": "model-00050-of-00101.safetensors", + "model.layers.46.mlp.experts.31.gate_proj.weight": "model-00050-of-00101.safetensors", + "model.layers.46.mlp.experts.31.up_proj.weight": "model-00050-of-00101.safetensors", + "model.layers.46.mlp.experts.31.down_proj.weight": "model-00050-of-00101.safetensors", + "model.layers.46.mlp.experts.32.gate_proj.weight": "model-00050-of-00101.safetensors", + "model.layers.46.mlp.experts.32.up_proj.weight": "model-00050-of-00101.safetensors", + "model.layers.46.mlp.experts.32.down_proj.weight": "model-00050-of-00101.safetensors", + "model.layers.46.mlp.experts.33.gate_proj.weight": "model-00050-of-00101.safetensors", + "model.layers.46.mlp.experts.33.up_proj.weight": "model-00050-of-00101.safetensors", + "model.layers.46.mlp.experts.33.down_proj.weight": "model-00050-of-00101.safetensors", + "model.layers.46.mlp.experts.34.gate_proj.weight": "model-00050-of-00101.safetensors", + "model.layers.46.mlp.experts.34.up_proj.weight": "model-00050-of-00101.safetensors", + "model.layers.46.mlp.experts.34.down_proj.weight": "model-00050-of-00101.safetensors", + "model.layers.46.mlp.experts.35.gate_proj.weight": "model-00050-of-00101.safetensors", + "model.layers.46.mlp.experts.35.up_proj.weight": "model-00050-of-00101.safetensors", + "model.layers.46.mlp.experts.35.down_proj.weight": "model-00050-of-00101.safetensors", + "model.layers.46.mlp.experts.36.gate_proj.weight": "model-00050-of-00101.safetensors", + "model.layers.46.mlp.experts.36.up_proj.weight": "model-00050-of-00101.safetensors", + "model.layers.46.mlp.experts.36.down_proj.weight": "model-00050-of-00101.safetensors", + "model.layers.46.mlp.experts.37.gate_proj.weight": "model-00050-of-00101.safetensors", + "model.layers.46.mlp.experts.37.up_proj.weight": "model-00050-of-00101.safetensors", + "model.layers.46.mlp.experts.37.down_proj.weight": "model-00050-of-00101.safetensors", + "model.layers.46.mlp.experts.38.gate_proj.weight": "model-00050-of-00101.safetensors", + "model.layers.46.mlp.experts.38.up_proj.weight": "model-00050-of-00101.safetensors", + "model.layers.46.mlp.experts.38.down_proj.weight": "model-00050-of-00101.safetensors", + "model.layers.46.mlp.experts.39.gate_proj.weight": "model-00050-of-00101.safetensors", + "model.layers.46.mlp.experts.39.up_proj.weight": "model-00050-of-00101.safetensors", + "model.layers.46.mlp.experts.39.down_proj.weight": "model-00050-of-00101.safetensors", + "model.layers.46.mlp.experts.40.gate_proj.weight": "model-00050-of-00101.safetensors", + "model.layers.46.mlp.experts.40.up_proj.weight": "model-00050-of-00101.safetensors", + "model.layers.46.mlp.experts.40.down_proj.weight": "model-00050-of-00101.safetensors", + "model.layers.46.mlp.experts.41.gate_proj.weight": "model-00050-of-00101.safetensors", + "model.layers.46.mlp.experts.41.up_proj.weight": "model-00050-of-00101.safetensors", + "model.layers.46.mlp.experts.41.down_proj.weight": "model-00050-of-00101.safetensors", + "model.layers.46.mlp.experts.42.gate_proj.weight": "model-00050-of-00101.safetensors", + "model.layers.46.mlp.experts.42.up_proj.weight": "model-00050-of-00101.safetensors", + "model.layers.46.mlp.experts.42.down_proj.weight": "model-00050-of-00101.safetensors", + "model.layers.46.mlp.experts.43.gate_proj.weight": "model-00050-of-00101.safetensors", + "model.layers.46.mlp.experts.43.up_proj.weight": "model-00050-of-00101.safetensors", + "model.layers.46.mlp.experts.43.down_proj.weight": "model-00050-of-00101.safetensors", + "model.layers.46.mlp.experts.44.gate_proj.weight": "model-00050-of-00101.safetensors", + "model.layers.46.mlp.experts.44.up_proj.weight": "model-00050-of-00101.safetensors", + "model.layers.46.mlp.experts.44.down_proj.weight": "model-00050-of-00101.safetensors", + "model.layers.46.mlp.experts.45.gate_proj.weight": "model-00050-of-00101.safetensors", + "model.layers.46.mlp.experts.45.up_proj.weight": "model-00050-of-00101.safetensors", + "model.layers.46.mlp.experts.45.down_proj.weight": "model-00050-of-00101.safetensors", + "model.layers.46.mlp.experts.46.gate_proj.weight": "model-00050-of-00101.safetensors", + "model.layers.46.mlp.experts.46.up_proj.weight": "model-00050-of-00101.safetensors", + "model.layers.46.mlp.experts.46.down_proj.weight": "model-00050-of-00101.safetensors", + "model.layers.46.mlp.experts.47.gate_proj.weight": "model-00050-of-00101.safetensors", + "model.layers.46.mlp.experts.47.up_proj.weight": "model-00050-of-00101.safetensors", + "model.layers.46.mlp.experts.47.down_proj.weight": "model-00050-of-00101.safetensors", + "model.layers.46.mlp.experts.48.gate_proj.weight": "model-00050-of-00101.safetensors", + "model.layers.46.mlp.experts.48.up_proj.weight": "model-00050-of-00101.safetensors", + "model.layers.46.mlp.experts.48.down_proj.weight": "model-00050-of-00101.safetensors", + "model.layers.46.mlp.experts.49.gate_proj.weight": "model-00050-of-00101.safetensors", + "model.layers.46.mlp.experts.49.up_proj.weight": "model-00050-of-00101.safetensors", + "model.layers.46.mlp.experts.49.down_proj.weight": "model-00050-of-00101.safetensors", + "model.layers.46.mlp.experts.50.gate_proj.weight": "model-00050-of-00101.safetensors", + "model.layers.46.mlp.experts.50.up_proj.weight": "model-00050-of-00101.safetensors", + "model.layers.46.mlp.experts.50.down_proj.weight": "model-00050-of-00101.safetensors", + "model.layers.46.mlp.experts.51.gate_proj.weight": "model-00050-of-00101.safetensors", + "model.layers.46.mlp.experts.51.up_proj.weight": "model-00050-of-00101.safetensors", + "model.layers.46.mlp.experts.51.down_proj.weight": "model-00050-of-00101.safetensors", + "model.layers.46.mlp.experts.52.gate_proj.weight": "model-00050-of-00101.safetensors", + "model.layers.46.mlp.experts.52.up_proj.weight": "model-00050-of-00101.safetensors", + "model.layers.46.mlp.experts.52.down_proj.weight": "model-00050-of-00101.safetensors", + "model.layers.46.mlp.experts.53.gate_proj.weight": "model-00050-of-00101.safetensors", + "model.layers.46.mlp.experts.53.up_proj.weight": "model-00050-of-00101.safetensors", + "model.layers.46.mlp.experts.53.down_proj.weight": "model-00050-of-00101.safetensors", + "model.layers.46.mlp.experts.54.gate_proj.weight": "model-00050-of-00101.safetensors", + "model.layers.46.mlp.experts.54.up_proj.weight": "model-00050-of-00101.safetensors", + "model.layers.46.mlp.experts.54.down_proj.weight": "model-00050-of-00101.safetensors", + "model.layers.46.mlp.experts.55.gate_proj.weight": "model-00050-of-00101.safetensors", + "model.layers.46.mlp.experts.55.up_proj.weight": "model-00050-of-00101.safetensors", + "model.layers.46.mlp.experts.55.down_proj.weight": "model-00050-of-00101.safetensors", + "model.layers.46.mlp.experts.56.gate_proj.weight": "model-00050-of-00101.safetensors", + "model.layers.46.mlp.experts.56.up_proj.weight": "model-00050-of-00101.safetensors", + "model.layers.46.mlp.experts.56.down_proj.weight": "model-00050-of-00101.safetensors", + "model.layers.46.mlp.experts.57.gate_proj.weight": "model-00050-of-00101.safetensors", + "model.layers.46.mlp.experts.57.up_proj.weight": "model-00050-of-00101.safetensors", + "model.layers.46.mlp.experts.57.down_proj.weight": "model-00050-of-00101.safetensors", + "model.layers.46.mlp.experts.58.gate_proj.weight": "model-00050-of-00101.safetensors", + "model.layers.46.mlp.experts.58.up_proj.weight": "model-00050-of-00101.safetensors", + "model.layers.46.mlp.experts.58.down_proj.weight": "model-00050-of-00101.safetensors", + "model.layers.46.mlp.experts.59.gate_proj.weight": "model-00050-of-00101.safetensors", + "model.layers.46.mlp.experts.59.up_proj.weight": "model-00050-of-00101.safetensors", + "model.layers.46.mlp.experts.59.down_proj.weight": "model-00050-of-00101.safetensors", + "model.layers.46.mlp.experts.60.gate_proj.weight": "model-00050-of-00101.safetensors", + "model.layers.46.mlp.experts.60.up_proj.weight": "model-00050-of-00101.safetensors", + "model.layers.46.mlp.experts.60.down_proj.weight": "model-00050-of-00101.safetensors", + "model.layers.46.mlp.experts.61.gate_proj.weight": "model-00050-of-00101.safetensors", + "model.layers.46.mlp.experts.61.up_proj.weight": "model-00050-of-00101.safetensors", + "model.layers.46.mlp.experts.61.down_proj.weight": "model-00050-of-00101.safetensors", + "model.layers.46.mlp.experts.62.gate_proj.weight": "model-00050-of-00101.safetensors", + "model.layers.46.mlp.experts.62.up_proj.weight": "model-00050-of-00101.safetensors", + "model.layers.46.mlp.experts.62.down_proj.weight": "model-00050-of-00101.safetensors", + "model.layers.46.mlp.experts.63.gate_proj.weight": "model-00050-of-00101.safetensors", + "model.layers.46.mlp.experts.63.up_proj.weight": "model-00050-of-00101.safetensors", + "model.layers.46.mlp.experts.63.down_proj.weight": "model-00050-of-00101.safetensors", + "model.layers.46.mlp.experts.64.gate_proj.weight": "model-00050-of-00101.safetensors", + "model.layers.46.mlp.experts.64.up_proj.weight": "model-00050-of-00101.safetensors", + "model.layers.46.mlp.experts.64.down_proj.weight": "model-00050-of-00101.safetensors", + "model.layers.46.mlp.experts.65.gate_proj.weight": "model-00050-of-00101.safetensors", + "model.layers.46.mlp.experts.65.up_proj.weight": "model-00050-of-00101.safetensors", + "model.layers.46.mlp.experts.65.down_proj.weight": "model-00050-of-00101.safetensors", + "model.layers.46.mlp.experts.66.gate_proj.weight": "model-00050-of-00101.safetensors", + "model.layers.46.mlp.experts.66.up_proj.weight": "model-00050-of-00101.safetensors", + "model.layers.46.mlp.experts.66.down_proj.weight": "model-00050-of-00101.safetensors", + "model.layers.46.mlp.experts.67.gate_proj.weight": "model-00050-of-00101.safetensors", + "model.layers.46.mlp.experts.67.up_proj.weight": "model-00050-of-00101.safetensors", + "model.layers.46.mlp.experts.67.down_proj.weight": "model-00050-of-00101.safetensors", + "model.layers.46.mlp.experts.68.gate_proj.weight": "model-00050-of-00101.safetensors", + "model.layers.46.mlp.experts.68.up_proj.weight": "model-00050-of-00101.safetensors", + "model.layers.46.mlp.experts.68.down_proj.weight": "model-00050-of-00101.safetensors", + "model.layers.46.mlp.experts.69.gate_proj.weight": "model-00050-of-00101.safetensors", + "model.layers.46.mlp.experts.69.up_proj.weight": "model-00050-of-00101.safetensors", + "model.layers.46.mlp.experts.69.down_proj.weight": "model-00050-of-00101.safetensors", + "model.layers.46.mlp.experts.70.gate_proj.weight": "model-00050-of-00101.safetensors", + "model.layers.46.mlp.experts.70.up_proj.weight": "model-00050-of-00101.safetensors", + "model.layers.46.mlp.experts.70.down_proj.weight": "model-00050-of-00101.safetensors", + "model.layers.46.mlp.experts.71.gate_proj.weight": "model-00050-of-00101.safetensors", + "model.layers.46.mlp.experts.71.up_proj.weight": "model-00050-of-00101.safetensors", + "model.layers.46.mlp.experts.71.down_proj.weight": "model-00050-of-00101.safetensors", + "model.layers.46.mlp.experts.72.gate_proj.weight": "model-00050-of-00101.safetensors", + "model.layers.46.mlp.experts.72.up_proj.weight": "model-00050-of-00101.safetensors", + "model.layers.46.mlp.experts.72.down_proj.weight": "model-00050-of-00101.safetensors", + "model.layers.46.mlp.experts.73.gate_proj.weight": "model-00050-of-00101.safetensors", + "model.layers.46.mlp.experts.73.up_proj.weight": "model-00050-of-00101.safetensors", + "model.layers.46.mlp.experts.73.down_proj.weight": "model-00050-of-00101.safetensors", + "model.layers.46.mlp.experts.74.gate_proj.weight": "model-00050-of-00101.safetensors", + "model.layers.46.mlp.experts.74.up_proj.weight": "model-00050-of-00101.safetensors", + "model.layers.46.mlp.experts.74.down_proj.weight": "model-00050-of-00101.safetensors", + "model.layers.46.mlp.experts.75.gate_proj.weight": "model-00050-of-00101.safetensors", + "model.layers.46.mlp.experts.75.up_proj.weight": "model-00050-of-00101.safetensors", + "model.layers.46.mlp.experts.75.down_proj.weight": "model-00050-of-00101.safetensors", + "model.layers.46.mlp.experts.76.gate_proj.weight": "model-00050-of-00101.safetensors", + "model.layers.46.mlp.experts.76.up_proj.weight": "model-00050-of-00101.safetensors", + "model.layers.46.mlp.experts.76.down_proj.weight": "model-00050-of-00101.safetensors", + "model.layers.46.mlp.experts.77.gate_proj.weight": "model-00050-of-00101.safetensors", + "model.layers.46.mlp.experts.77.up_proj.weight": "model-00050-of-00101.safetensors", + "model.layers.46.mlp.experts.77.down_proj.weight": "model-00050-of-00101.safetensors", + "model.layers.46.mlp.experts.78.gate_proj.weight": "model-00050-of-00101.safetensors", + "model.layers.46.mlp.experts.78.up_proj.weight": "model-00050-of-00101.safetensors", + "model.layers.46.mlp.experts.78.down_proj.weight": "model-00050-of-00101.safetensors", + "model.layers.46.mlp.experts.79.gate_proj.weight": "model-00050-of-00101.safetensors", + "model.layers.46.mlp.experts.79.up_proj.weight": "model-00050-of-00101.safetensors", + "model.layers.46.mlp.experts.79.down_proj.weight": "model-00050-of-00101.safetensors", + "model.layers.46.mlp.experts.80.gate_proj.weight": "model-00050-of-00101.safetensors", + "model.layers.46.mlp.experts.80.up_proj.weight": "model-00050-of-00101.safetensors", + "model.layers.46.mlp.experts.80.down_proj.weight": "model-00050-of-00101.safetensors", + "model.layers.46.mlp.experts.81.gate_proj.weight": "model-00050-of-00101.safetensors", + "model.layers.46.mlp.experts.81.up_proj.weight": "model-00050-of-00101.safetensors", + "model.layers.46.mlp.experts.81.down_proj.weight": "model-00050-of-00101.safetensors", + "model.layers.46.mlp.experts.82.gate_proj.weight": "model-00050-of-00101.safetensors", + "model.layers.46.mlp.experts.82.up_proj.weight": "model-00050-of-00101.safetensors", + "model.layers.46.mlp.experts.82.down_proj.weight": "model-00050-of-00101.safetensors", + "model.layers.46.mlp.experts.83.gate_proj.weight": "model-00050-of-00101.safetensors", + "model.layers.46.mlp.experts.83.up_proj.weight": "model-00050-of-00101.safetensors", + "model.layers.46.mlp.experts.83.down_proj.weight": "model-00050-of-00101.safetensors", + "model.layers.46.mlp.experts.84.gate_proj.weight": "model-00050-of-00101.safetensors", + "model.layers.46.mlp.experts.84.up_proj.weight": "model-00050-of-00101.safetensors", + "model.layers.46.mlp.experts.84.down_proj.weight": "model-00050-of-00101.safetensors", + "model.layers.46.mlp.experts.85.gate_proj.weight": "model-00050-of-00101.safetensors", + "model.layers.46.mlp.experts.85.up_proj.weight": "model-00050-of-00101.safetensors", + "model.layers.46.mlp.experts.85.down_proj.weight": "model-00050-of-00101.safetensors", + "model.layers.46.mlp.experts.86.gate_proj.weight": "model-00050-of-00101.safetensors", + "model.layers.46.mlp.experts.86.up_proj.weight": "model-00050-of-00101.safetensors", + "model.layers.46.mlp.experts.86.down_proj.weight": "model-00050-of-00101.safetensors", + "model.layers.46.mlp.experts.87.gate_proj.weight": "model-00050-of-00101.safetensors", + "model.layers.46.mlp.experts.87.up_proj.weight": "model-00050-of-00101.safetensors", + "model.layers.46.mlp.experts.87.down_proj.weight": "model-00050-of-00101.safetensors", + "model.layers.46.mlp.experts.88.gate_proj.weight": "model-00050-of-00101.safetensors", + "model.layers.46.mlp.experts.88.up_proj.weight": "model-00050-of-00101.safetensors", + "model.layers.46.mlp.experts.88.down_proj.weight": "model-00050-of-00101.safetensors", + "model.layers.46.mlp.experts.89.gate_proj.weight": "model-00050-of-00101.safetensors", + "model.layers.46.mlp.experts.89.up_proj.weight": "model-00050-of-00101.safetensors", + "model.layers.46.mlp.experts.89.down_proj.weight": "model-00050-of-00101.safetensors", + "model.layers.46.mlp.experts.90.gate_proj.weight": "model-00050-of-00101.safetensors", + "model.layers.46.mlp.experts.90.up_proj.weight": "model-00050-of-00101.safetensors", + "model.layers.46.mlp.experts.90.down_proj.weight": "model-00050-of-00101.safetensors", + "model.layers.46.mlp.experts.91.gate_proj.weight": "model-00050-of-00101.safetensors", + "model.layers.46.mlp.experts.91.up_proj.weight": "model-00050-of-00101.safetensors", + "model.layers.46.mlp.experts.91.down_proj.weight": "model-00050-of-00101.safetensors", + "model.layers.46.mlp.experts.92.gate_proj.weight": "model-00050-of-00101.safetensors", + "model.layers.46.mlp.experts.92.up_proj.weight": "model-00050-of-00101.safetensors", + "model.layers.46.mlp.experts.92.down_proj.weight": "model-00050-of-00101.safetensors", + "model.layers.46.mlp.experts.93.gate_proj.weight": "model-00050-of-00101.safetensors", + "model.layers.46.mlp.experts.93.up_proj.weight": "model-00050-of-00101.safetensors", + "model.layers.46.mlp.experts.93.down_proj.weight": "model-00050-of-00101.safetensors", + "model.layers.46.mlp.experts.94.gate_proj.weight": "model-00050-of-00101.safetensors", + "model.layers.46.mlp.experts.94.up_proj.weight": "model-00050-of-00101.safetensors", + "model.layers.46.mlp.experts.94.down_proj.weight": "model-00050-of-00101.safetensors", + "model.layers.46.mlp.experts.95.gate_proj.weight": "model-00050-of-00101.safetensors", + "model.layers.46.mlp.experts.95.up_proj.weight": "model-00050-of-00101.safetensors", + "model.layers.46.mlp.experts.95.down_proj.weight": "model-00050-of-00101.safetensors", + "model.layers.46.mlp.experts.96.gate_proj.weight": "model-00050-of-00101.safetensors", + "model.layers.46.mlp.experts.96.up_proj.weight": "model-00050-of-00101.safetensors", + "model.layers.46.mlp.experts.96.down_proj.weight": "model-00050-of-00101.safetensors", + "model.layers.46.mlp.experts.97.gate_proj.weight": "model-00050-of-00101.safetensors", + "model.layers.46.mlp.experts.97.up_proj.weight": "model-00050-of-00101.safetensors", + "model.layers.46.mlp.experts.97.down_proj.weight": "model-00050-of-00101.safetensors", + "model.layers.46.mlp.experts.98.gate_proj.weight": "model-00050-of-00101.safetensors", + "model.layers.46.mlp.experts.98.up_proj.weight": "model-00050-of-00101.safetensors", + "model.layers.46.mlp.experts.98.down_proj.weight": "model-00050-of-00101.safetensors", + "model.layers.46.mlp.experts.99.gate_proj.weight": "model-00050-of-00101.safetensors", + "model.layers.46.mlp.experts.99.up_proj.weight": "model-00050-of-00101.safetensors", + "model.layers.46.mlp.experts.99.down_proj.weight": "model-00050-of-00101.safetensors", + "model.layers.46.mlp.experts.100.gate_proj.weight": "model-00050-of-00101.safetensors", + "model.layers.46.mlp.experts.100.up_proj.weight": "model-00050-of-00101.safetensors", + "model.layers.46.mlp.experts.100.down_proj.weight": "model-00050-of-00101.safetensors", + "model.layers.46.mlp.experts.101.gate_proj.weight": "model-00050-of-00101.safetensors", + "model.layers.46.mlp.experts.101.up_proj.weight": "model-00050-of-00101.safetensors", + "model.layers.46.mlp.experts.101.down_proj.weight": "model-00050-of-00101.safetensors", + "model.layers.46.mlp.experts.102.gate_proj.weight": "model-00050-of-00101.safetensors", + "model.layers.46.mlp.experts.102.up_proj.weight": "model-00050-of-00101.safetensors", + "model.layers.46.mlp.experts.102.down_proj.weight": "model-00050-of-00101.safetensors", + "model.layers.46.mlp.experts.103.gate_proj.weight": "model-00050-of-00101.safetensors", + "model.layers.46.mlp.experts.103.up_proj.weight": "model-00050-of-00101.safetensors", + "model.layers.46.mlp.experts.103.down_proj.weight": "model-00050-of-00101.safetensors", + "model.layers.46.mlp.experts.104.gate_proj.weight": "model-00050-of-00101.safetensors", + "model.layers.46.mlp.experts.104.up_proj.weight": "model-00050-of-00101.safetensors", + "model.layers.46.mlp.experts.104.down_proj.weight": "model-00050-of-00101.safetensors", + "model.layers.46.mlp.experts.105.gate_proj.weight": "model-00050-of-00101.safetensors", + "model.layers.46.mlp.experts.105.up_proj.weight": "model-00050-of-00101.safetensors", + "model.layers.46.mlp.experts.105.down_proj.weight": "model-00050-of-00101.safetensors", + "model.layers.46.mlp.experts.106.gate_proj.weight": "model-00050-of-00101.safetensors", + "model.layers.46.mlp.experts.106.up_proj.weight": "model-00050-of-00101.safetensors", + "model.layers.46.mlp.experts.106.down_proj.weight": "model-00050-of-00101.safetensors", + "model.layers.46.mlp.experts.107.gate_proj.weight": "model-00050-of-00101.safetensors", + "model.layers.46.mlp.experts.107.up_proj.weight": "model-00050-of-00101.safetensors", + "model.layers.46.mlp.experts.107.down_proj.weight": "model-00050-of-00101.safetensors", + "model.layers.46.mlp.experts.108.gate_proj.weight": "model-00050-of-00101.safetensors", + "model.layers.46.mlp.experts.108.up_proj.weight": "model-00050-of-00101.safetensors", + "model.layers.46.mlp.experts.108.down_proj.weight": "model-00050-of-00101.safetensors", + "model.layers.46.mlp.experts.109.gate_proj.weight": "model-00050-of-00101.safetensors", + "model.layers.46.mlp.experts.109.up_proj.weight": "model-00050-of-00101.safetensors", + "model.layers.46.mlp.experts.109.down_proj.weight": "model-00050-of-00101.safetensors", + "model.layers.46.mlp.experts.110.gate_proj.weight": "model-00050-of-00101.safetensors", + "model.layers.46.mlp.experts.110.up_proj.weight": "model-00050-of-00101.safetensors", + "model.layers.46.mlp.experts.110.down_proj.weight": "model-00050-of-00101.safetensors", + "model.layers.46.mlp.experts.111.gate_proj.weight": "model-00050-of-00101.safetensors", + "model.layers.46.mlp.experts.111.up_proj.weight": "model-00050-of-00101.safetensors", + "model.layers.46.mlp.experts.111.down_proj.weight": "model-00050-of-00101.safetensors", + "model.layers.46.mlp.experts.112.gate_proj.weight": "model-00050-of-00101.safetensors", + "model.layers.46.mlp.experts.112.up_proj.weight": "model-00050-of-00101.safetensors", + "model.layers.46.mlp.experts.112.down_proj.weight": "model-00050-of-00101.safetensors", + "model.layers.46.mlp.experts.113.gate_proj.weight": "model-00050-of-00101.safetensors", + "model.layers.46.mlp.experts.113.up_proj.weight": "model-00050-of-00101.safetensors", + "model.layers.46.mlp.experts.113.down_proj.weight": "model-00050-of-00101.safetensors", + "model.layers.46.mlp.experts.114.gate_proj.weight": "model-00050-of-00101.safetensors", + "model.layers.46.mlp.experts.114.up_proj.weight": "model-00050-of-00101.safetensors", + "model.layers.46.mlp.experts.114.down_proj.weight": "model-00050-of-00101.safetensors", + "model.layers.46.mlp.experts.115.gate_proj.weight": "model-00050-of-00101.safetensors", + "model.layers.46.mlp.experts.115.up_proj.weight": "model-00050-of-00101.safetensors", + "model.layers.46.mlp.experts.115.down_proj.weight": "model-00050-of-00101.safetensors", + "model.layers.46.mlp.experts.116.gate_proj.weight": "model-00050-of-00101.safetensors", + "model.layers.46.mlp.experts.116.up_proj.weight": "model-00050-of-00101.safetensors", + "model.layers.46.mlp.experts.116.down_proj.weight": "model-00050-of-00101.safetensors", + "model.layers.46.mlp.experts.117.gate_proj.weight": "model-00050-of-00101.safetensors", + "model.layers.46.mlp.experts.117.up_proj.weight": "model-00050-of-00101.safetensors", + "model.layers.46.mlp.experts.117.down_proj.weight": "model-00050-of-00101.safetensors", + "model.layers.46.mlp.experts.118.gate_proj.weight": "model-00050-of-00101.safetensors", + "model.layers.46.mlp.experts.118.up_proj.weight": "model-00050-of-00101.safetensors", + "model.layers.46.mlp.experts.118.down_proj.weight": "model-00050-of-00101.safetensors", + "model.layers.46.mlp.experts.119.gate_proj.weight": "model-00050-of-00101.safetensors", + "model.layers.46.mlp.experts.119.up_proj.weight": "model-00050-of-00101.safetensors", + "model.layers.46.mlp.experts.119.down_proj.weight": "model-00050-of-00101.safetensors", + "model.layers.46.mlp.gate.weight": "model-00050-of-00101.safetensors", + "model.layers.46.mlp.gate.e_score_correction_bias": "model-00050-of-00101.safetensors", + "model.layers.46.mlp.shared_experts.gate_proj.weight": "model-00050-of-00101.safetensors", + "model.layers.46.mlp.shared_experts.up_proj.weight": "model-00050-of-00101.safetensors", + "model.layers.46.mlp.shared_experts.down_proj.weight": "model-00050-of-00101.safetensors", + "model.layers.46.input_layernorm.weight": "model-00050-of-00101.safetensors", + "model.layers.46.post_attention_layernorm.weight": "model-00050-of-00101.safetensors", + "model.layers.47.self_attn.q_proj.weight": "model-00050-of-00101.safetensors", + "model.layers.47.self_attn.q_proj.bias": "model-00050-of-00101.safetensors", + "model.layers.47.self_attn.k_proj.weight": "model-00050-of-00101.safetensors", + "model.layers.47.self_attn.k_proj.bias": "model-00050-of-00101.safetensors", + "model.layers.47.self_attn.v_proj.weight": "model-00050-of-00101.safetensors", + "model.layers.47.self_attn.v_proj.bias": "model-00050-of-00101.safetensors", + "model.layers.47.self_attn.o_proj.weight": "model-00050-of-00101.safetensors", + "model.layers.47.self_attn.q_norm.weight": "model-00050-of-00101.safetensors", + "model.layers.47.self_attn.k_norm.weight": "model-00050-of-00101.safetensors", + "model.layers.47.mlp.experts.0.gate_proj.weight": "model-00050-of-00101.safetensors", + "model.layers.47.mlp.experts.0.up_proj.weight": "model-00050-of-00101.safetensors", + "model.layers.47.mlp.experts.0.down_proj.weight": "model-00050-of-00101.safetensors", + "model.layers.47.mlp.experts.1.gate_proj.weight": "model-00050-of-00101.safetensors", + "model.layers.47.mlp.experts.1.up_proj.weight": "model-00050-of-00101.safetensors", + "model.layers.47.mlp.experts.1.down_proj.weight": "model-00050-of-00101.safetensors", + "model.layers.47.mlp.experts.2.gate_proj.weight": "model-00050-of-00101.safetensors", + "model.layers.47.mlp.experts.2.up_proj.weight": "model-00050-of-00101.safetensors", + "model.layers.47.mlp.experts.2.down_proj.weight": "model-00050-of-00101.safetensors", + "model.layers.47.mlp.experts.3.gate_proj.weight": "model-00050-of-00101.safetensors", + "model.layers.47.mlp.experts.3.up_proj.weight": "model-00050-of-00101.safetensors", + "model.layers.47.mlp.experts.3.down_proj.weight": "model-00050-of-00101.safetensors", + "model.layers.47.mlp.experts.4.gate_proj.weight": "model-00050-of-00101.safetensors", + "model.layers.47.mlp.experts.4.up_proj.weight": "model-00050-of-00101.safetensors", + "model.layers.47.mlp.experts.4.down_proj.weight": "model-00050-of-00101.safetensors", + "model.layers.47.mlp.experts.5.gate_proj.weight": "model-00050-of-00101.safetensors", + "model.layers.47.mlp.experts.5.up_proj.weight": "model-00050-of-00101.safetensors", + "model.layers.47.mlp.experts.5.down_proj.weight": "model-00050-of-00101.safetensors", + "model.layers.47.mlp.experts.6.gate_proj.weight": "model-00050-of-00101.safetensors", + "model.layers.47.mlp.experts.6.up_proj.weight": "model-00050-of-00101.safetensors", + "model.layers.47.mlp.experts.6.down_proj.weight": "model-00050-of-00101.safetensors", + "model.layers.47.mlp.experts.7.gate_proj.weight": "model-00050-of-00101.safetensors", + "model.layers.47.mlp.experts.7.up_proj.weight": "model-00050-of-00101.safetensors", + "model.layers.47.mlp.experts.7.down_proj.weight": "model-00050-of-00101.safetensors", + "model.layers.47.mlp.experts.8.gate_proj.weight": "model-00050-of-00101.safetensors", + "model.layers.47.mlp.experts.8.up_proj.weight": "model-00050-of-00101.safetensors", + "model.layers.47.mlp.experts.8.down_proj.weight": "model-00050-of-00101.safetensors", + "model.layers.47.mlp.experts.9.gate_proj.weight": "model-00050-of-00101.safetensors", + "model.layers.47.mlp.experts.9.up_proj.weight": "model-00050-of-00101.safetensors", + "model.layers.47.mlp.experts.9.down_proj.weight": "model-00050-of-00101.safetensors", + "model.layers.47.mlp.experts.10.gate_proj.weight": "model-00050-of-00101.safetensors", + "model.layers.47.mlp.experts.10.up_proj.weight": "model-00050-of-00101.safetensors", + "model.layers.47.mlp.experts.10.down_proj.weight": "model-00050-of-00101.safetensors", + "model.layers.47.mlp.experts.11.gate_proj.weight": "model-00050-of-00101.safetensors", + "model.layers.47.mlp.experts.11.up_proj.weight": "model-00050-of-00101.safetensors", + "model.layers.47.mlp.experts.11.down_proj.weight": "model-00050-of-00101.safetensors", + "model.layers.47.mlp.experts.12.gate_proj.weight": "model-00050-of-00101.safetensors", + "model.layers.47.mlp.experts.12.up_proj.weight": "model-00050-of-00101.safetensors", + "model.layers.47.mlp.experts.12.down_proj.weight": "model-00050-of-00101.safetensors", + "model.layers.47.mlp.experts.13.gate_proj.weight": "model-00050-of-00101.safetensors", + "model.layers.47.mlp.experts.13.up_proj.weight": "model-00050-of-00101.safetensors", + "model.layers.47.mlp.experts.13.down_proj.weight": "model-00050-of-00101.safetensors", + "model.layers.47.mlp.experts.14.gate_proj.weight": "model-00050-of-00101.safetensors", + "model.layers.47.mlp.experts.14.up_proj.weight": "model-00051-of-00101.safetensors", + "model.layers.47.mlp.experts.14.down_proj.weight": "model-00051-of-00101.safetensors", + "model.layers.47.mlp.experts.15.gate_proj.weight": "model-00051-of-00101.safetensors", + "model.layers.47.mlp.experts.15.up_proj.weight": "model-00051-of-00101.safetensors", + "model.layers.47.mlp.experts.15.down_proj.weight": "model-00051-of-00101.safetensors", + "model.layers.47.mlp.experts.16.gate_proj.weight": "model-00051-of-00101.safetensors", + "model.layers.47.mlp.experts.16.up_proj.weight": "model-00051-of-00101.safetensors", + "model.layers.47.mlp.experts.16.down_proj.weight": "model-00051-of-00101.safetensors", + "model.layers.47.mlp.experts.17.gate_proj.weight": "model-00051-of-00101.safetensors", + "model.layers.47.mlp.experts.17.up_proj.weight": "model-00051-of-00101.safetensors", + "model.layers.47.mlp.experts.17.down_proj.weight": "model-00051-of-00101.safetensors", + "model.layers.47.mlp.experts.18.gate_proj.weight": "model-00051-of-00101.safetensors", + "model.layers.47.mlp.experts.18.up_proj.weight": "model-00051-of-00101.safetensors", + "model.layers.47.mlp.experts.18.down_proj.weight": "model-00051-of-00101.safetensors", + "model.layers.47.mlp.experts.19.gate_proj.weight": "model-00051-of-00101.safetensors", + "model.layers.47.mlp.experts.19.up_proj.weight": "model-00051-of-00101.safetensors", + "model.layers.47.mlp.experts.19.down_proj.weight": "model-00051-of-00101.safetensors", + "model.layers.47.mlp.experts.20.gate_proj.weight": "model-00051-of-00101.safetensors", + "model.layers.47.mlp.experts.20.up_proj.weight": "model-00051-of-00101.safetensors", + "model.layers.47.mlp.experts.20.down_proj.weight": "model-00051-of-00101.safetensors", + "model.layers.47.mlp.experts.21.gate_proj.weight": "model-00051-of-00101.safetensors", + "model.layers.47.mlp.experts.21.up_proj.weight": "model-00051-of-00101.safetensors", + "model.layers.47.mlp.experts.21.down_proj.weight": "model-00051-of-00101.safetensors", + "model.layers.47.mlp.experts.22.gate_proj.weight": "model-00051-of-00101.safetensors", + "model.layers.47.mlp.experts.22.up_proj.weight": "model-00051-of-00101.safetensors", + "model.layers.47.mlp.experts.22.down_proj.weight": "model-00051-of-00101.safetensors", + "model.layers.47.mlp.experts.23.gate_proj.weight": "model-00051-of-00101.safetensors", + "model.layers.47.mlp.experts.23.up_proj.weight": "model-00051-of-00101.safetensors", + "model.layers.47.mlp.experts.23.down_proj.weight": "model-00051-of-00101.safetensors", + "model.layers.47.mlp.experts.24.gate_proj.weight": "model-00051-of-00101.safetensors", + "model.layers.47.mlp.experts.24.up_proj.weight": "model-00051-of-00101.safetensors", + "model.layers.47.mlp.experts.24.down_proj.weight": "model-00051-of-00101.safetensors", + "model.layers.47.mlp.experts.25.gate_proj.weight": "model-00051-of-00101.safetensors", + "model.layers.47.mlp.experts.25.up_proj.weight": "model-00051-of-00101.safetensors", + "model.layers.47.mlp.experts.25.down_proj.weight": "model-00051-of-00101.safetensors", + "model.layers.47.mlp.experts.26.gate_proj.weight": "model-00051-of-00101.safetensors", + "model.layers.47.mlp.experts.26.up_proj.weight": "model-00051-of-00101.safetensors", + "model.layers.47.mlp.experts.26.down_proj.weight": "model-00051-of-00101.safetensors", + "model.layers.47.mlp.experts.27.gate_proj.weight": "model-00051-of-00101.safetensors", + "model.layers.47.mlp.experts.27.up_proj.weight": "model-00051-of-00101.safetensors", + "model.layers.47.mlp.experts.27.down_proj.weight": "model-00051-of-00101.safetensors", + "model.layers.47.mlp.experts.28.gate_proj.weight": "model-00051-of-00101.safetensors", + "model.layers.47.mlp.experts.28.up_proj.weight": "model-00051-of-00101.safetensors", + "model.layers.47.mlp.experts.28.down_proj.weight": "model-00051-of-00101.safetensors", + "model.layers.47.mlp.experts.29.gate_proj.weight": "model-00051-of-00101.safetensors", + "model.layers.47.mlp.experts.29.up_proj.weight": "model-00051-of-00101.safetensors", + "model.layers.47.mlp.experts.29.down_proj.weight": "model-00051-of-00101.safetensors", + "model.layers.47.mlp.experts.30.gate_proj.weight": "model-00051-of-00101.safetensors", + "model.layers.47.mlp.experts.30.up_proj.weight": "model-00051-of-00101.safetensors", + "model.layers.47.mlp.experts.30.down_proj.weight": "model-00051-of-00101.safetensors", + "model.layers.47.mlp.experts.31.gate_proj.weight": "model-00051-of-00101.safetensors", + "model.layers.47.mlp.experts.31.up_proj.weight": "model-00051-of-00101.safetensors", + "model.layers.47.mlp.experts.31.down_proj.weight": "model-00051-of-00101.safetensors", + "model.layers.47.mlp.experts.32.gate_proj.weight": "model-00051-of-00101.safetensors", + "model.layers.47.mlp.experts.32.up_proj.weight": "model-00051-of-00101.safetensors", + "model.layers.47.mlp.experts.32.down_proj.weight": "model-00051-of-00101.safetensors", + "model.layers.47.mlp.experts.33.gate_proj.weight": "model-00051-of-00101.safetensors", + "model.layers.47.mlp.experts.33.up_proj.weight": "model-00051-of-00101.safetensors", + "model.layers.47.mlp.experts.33.down_proj.weight": "model-00051-of-00101.safetensors", + "model.layers.47.mlp.experts.34.gate_proj.weight": "model-00051-of-00101.safetensors", + "model.layers.47.mlp.experts.34.up_proj.weight": "model-00051-of-00101.safetensors", + "model.layers.47.mlp.experts.34.down_proj.weight": "model-00051-of-00101.safetensors", + "model.layers.47.mlp.experts.35.gate_proj.weight": "model-00051-of-00101.safetensors", + "model.layers.47.mlp.experts.35.up_proj.weight": "model-00051-of-00101.safetensors", + "model.layers.47.mlp.experts.35.down_proj.weight": "model-00051-of-00101.safetensors", + "model.layers.47.mlp.experts.36.gate_proj.weight": "model-00051-of-00101.safetensors", + "model.layers.47.mlp.experts.36.up_proj.weight": "model-00051-of-00101.safetensors", + "model.layers.47.mlp.experts.36.down_proj.weight": "model-00051-of-00101.safetensors", + "model.layers.47.mlp.experts.37.gate_proj.weight": "model-00051-of-00101.safetensors", + "model.layers.47.mlp.experts.37.up_proj.weight": "model-00051-of-00101.safetensors", + "model.layers.47.mlp.experts.37.down_proj.weight": "model-00051-of-00101.safetensors", + "model.layers.47.mlp.experts.38.gate_proj.weight": "model-00051-of-00101.safetensors", + "model.layers.47.mlp.experts.38.up_proj.weight": "model-00051-of-00101.safetensors", + "model.layers.47.mlp.experts.38.down_proj.weight": "model-00051-of-00101.safetensors", + "model.layers.47.mlp.experts.39.gate_proj.weight": "model-00051-of-00101.safetensors", + "model.layers.47.mlp.experts.39.up_proj.weight": "model-00051-of-00101.safetensors", + "model.layers.47.mlp.experts.39.down_proj.weight": "model-00051-of-00101.safetensors", + "model.layers.47.mlp.experts.40.gate_proj.weight": "model-00051-of-00101.safetensors", + "model.layers.47.mlp.experts.40.up_proj.weight": "model-00051-of-00101.safetensors", + "model.layers.47.mlp.experts.40.down_proj.weight": "model-00051-of-00101.safetensors", + "model.layers.47.mlp.experts.41.gate_proj.weight": "model-00051-of-00101.safetensors", + "model.layers.47.mlp.experts.41.up_proj.weight": "model-00051-of-00101.safetensors", + "model.layers.47.mlp.experts.41.down_proj.weight": "model-00051-of-00101.safetensors", + "model.layers.47.mlp.experts.42.gate_proj.weight": "model-00051-of-00101.safetensors", + "model.layers.47.mlp.experts.42.up_proj.weight": "model-00051-of-00101.safetensors", + "model.layers.47.mlp.experts.42.down_proj.weight": "model-00051-of-00101.safetensors", + "model.layers.47.mlp.experts.43.gate_proj.weight": "model-00051-of-00101.safetensors", + "model.layers.47.mlp.experts.43.up_proj.weight": "model-00051-of-00101.safetensors", + "model.layers.47.mlp.experts.43.down_proj.weight": "model-00051-of-00101.safetensors", + "model.layers.47.mlp.experts.44.gate_proj.weight": "model-00051-of-00101.safetensors", + "model.layers.47.mlp.experts.44.up_proj.weight": "model-00051-of-00101.safetensors", + "model.layers.47.mlp.experts.44.down_proj.weight": "model-00051-of-00101.safetensors", + "model.layers.47.mlp.experts.45.gate_proj.weight": "model-00051-of-00101.safetensors", + "model.layers.47.mlp.experts.45.up_proj.weight": "model-00051-of-00101.safetensors", + "model.layers.47.mlp.experts.45.down_proj.weight": "model-00051-of-00101.safetensors", + "model.layers.47.mlp.experts.46.gate_proj.weight": "model-00051-of-00101.safetensors", + "model.layers.47.mlp.experts.46.up_proj.weight": "model-00051-of-00101.safetensors", + "model.layers.47.mlp.experts.46.down_proj.weight": "model-00051-of-00101.safetensors", + "model.layers.47.mlp.experts.47.gate_proj.weight": "model-00051-of-00101.safetensors", + "model.layers.47.mlp.experts.47.up_proj.weight": "model-00051-of-00101.safetensors", + "model.layers.47.mlp.experts.47.down_proj.weight": "model-00051-of-00101.safetensors", + "model.layers.47.mlp.experts.48.gate_proj.weight": "model-00051-of-00101.safetensors", + "model.layers.47.mlp.experts.48.up_proj.weight": "model-00051-of-00101.safetensors", + "model.layers.47.mlp.experts.48.down_proj.weight": "model-00051-of-00101.safetensors", + "model.layers.47.mlp.experts.49.gate_proj.weight": "model-00051-of-00101.safetensors", + "model.layers.47.mlp.experts.49.up_proj.weight": "model-00051-of-00101.safetensors", + "model.layers.47.mlp.experts.49.down_proj.weight": "model-00051-of-00101.safetensors", + "model.layers.47.mlp.experts.50.gate_proj.weight": "model-00051-of-00101.safetensors", + "model.layers.47.mlp.experts.50.up_proj.weight": "model-00051-of-00101.safetensors", + "model.layers.47.mlp.experts.50.down_proj.weight": "model-00051-of-00101.safetensors", + "model.layers.47.mlp.experts.51.gate_proj.weight": "model-00051-of-00101.safetensors", + "model.layers.47.mlp.experts.51.up_proj.weight": "model-00051-of-00101.safetensors", + "model.layers.47.mlp.experts.51.down_proj.weight": "model-00051-of-00101.safetensors", + "model.layers.47.mlp.experts.52.gate_proj.weight": "model-00051-of-00101.safetensors", + "model.layers.47.mlp.experts.52.up_proj.weight": "model-00051-of-00101.safetensors", + "model.layers.47.mlp.experts.52.down_proj.weight": "model-00051-of-00101.safetensors", + "model.layers.47.mlp.experts.53.gate_proj.weight": "model-00051-of-00101.safetensors", + "model.layers.47.mlp.experts.53.up_proj.weight": "model-00051-of-00101.safetensors", + "model.layers.47.mlp.experts.53.down_proj.weight": "model-00051-of-00101.safetensors", + "model.layers.47.mlp.experts.54.gate_proj.weight": "model-00051-of-00101.safetensors", + "model.layers.47.mlp.experts.54.up_proj.weight": "model-00051-of-00101.safetensors", + "model.layers.47.mlp.experts.54.down_proj.weight": "model-00051-of-00101.safetensors", + "model.layers.47.mlp.experts.55.gate_proj.weight": "model-00051-of-00101.safetensors", + "model.layers.47.mlp.experts.55.up_proj.weight": "model-00051-of-00101.safetensors", + "model.layers.47.mlp.experts.55.down_proj.weight": "model-00051-of-00101.safetensors", + "model.layers.47.mlp.experts.56.gate_proj.weight": "model-00051-of-00101.safetensors", + "model.layers.47.mlp.experts.56.up_proj.weight": "model-00051-of-00101.safetensors", + "model.layers.47.mlp.experts.56.down_proj.weight": "model-00051-of-00101.safetensors", + "model.layers.47.mlp.experts.57.gate_proj.weight": "model-00051-of-00101.safetensors", + "model.layers.47.mlp.experts.57.up_proj.weight": "model-00051-of-00101.safetensors", + "model.layers.47.mlp.experts.57.down_proj.weight": "model-00051-of-00101.safetensors", + "model.layers.47.mlp.experts.58.gate_proj.weight": "model-00051-of-00101.safetensors", + "model.layers.47.mlp.experts.58.up_proj.weight": "model-00051-of-00101.safetensors", + "model.layers.47.mlp.experts.58.down_proj.weight": "model-00051-of-00101.safetensors", + "model.layers.47.mlp.experts.59.gate_proj.weight": "model-00051-of-00101.safetensors", + "model.layers.47.mlp.experts.59.up_proj.weight": "model-00051-of-00101.safetensors", + "model.layers.47.mlp.experts.59.down_proj.weight": "model-00051-of-00101.safetensors", + "model.layers.47.mlp.experts.60.gate_proj.weight": "model-00051-of-00101.safetensors", + "model.layers.47.mlp.experts.60.up_proj.weight": "model-00051-of-00101.safetensors", + "model.layers.47.mlp.experts.60.down_proj.weight": "model-00051-of-00101.safetensors", + "model.layers.47.mlp.experts.61.gate_proj.weight": "model-00051-of-00101.safetensors", + "model.layers.47.mlp.experts.61.up_proj.weight": "model-00051-of-00101.safetensors", + "model.layers.47.mlp.experts.61.down_proj.weight": "model-00051-of-00101.safetensors", + "model.layers.47.mlp.experts.62.gate_proj.weight": "model-00051-of-00101.safetensors", + "model.layers.47.mlp.experts.62.up_proj.weight": "model-00051-of-00101.safetensors", + "model.layers.47.mlp.experts.62.down_proj.weight": "model-00051-of-00101.safetensors", + "model.layers.47.mlp.experts.63.gate_proj.weight": "model-00051-of-00101.safetensors", + "model.layers.47.mlp.experts.63.up_proj.weight": "model-00051-of-00101.safetensors", + "model.layers.47.mlp.experts.63.down_proj.weight": "model-00051-of-00101.safetensors", + "model.layers.47.mlp.experts.64.gate_proj.weight": "model-00051-of-00101.safetensors", + "model.layers.47.mlp.experts.64.up_proj.weight": "model-00051-of-00101.safetensors", + "model.layers.47.mlp.experts.64.down_proj.weight": "model-00051-of-00101.safetensors", + "model.layers.47.mlp.experts.65.gate_proj.weight": "model-00051-of-00101.safetensors", + "model.layers.47.mlp.experts.65.up_proj.weight": "model-00051-of-00101.safetensors", + "model.layers.47.mlp.experts.65.down_proj.weight": "model-00051-of-00101.safetensors", + "model.layers.47.mlp.experts.66.gate_proj.weight": "model-00051-of-00101.safetensors", + "model.layers.47.mlp.experts.66.up_proj.weight": "model-00051-of-00101.safetensors", + "model.layers.47.mlp.experts.66.down_proj.weight": "model-00051-of-00101.safetensors", + "model.layers.47.mlp.experts.67.gate_proj.weight": "model-00051-of-00101.safetensors", + "model.layers.47.mlp.experts.67.up_proj.weight": "model-00051-of-00101.safetensors", + "model.layers.47.mlp.experts.67.down_proj.weight": "model-00051-of-00101.safetensors", + "model.layers.47.mlp.experts.68.gate_proj.weight": "model-00051-of-00101.safetensors", + "model.layers.47.mlp.experts.68.up_proj.weight": "model-00051-of-00101.safetensors", + "model.layers.47.mlp.experts.68.down_proj.weight": "model-00051-of-00101.safetensors", + "model.layers.47.mlp.experts.69.gate_proj.weight": "model-00051-of-00101.safetensors", + "model.layers.47.mlp.experts.69.up_proj.weight": "model-00051-of-00101.safetensors", + "model.layers.47.mlp.experts.69.down_proj.weight": "model-00051-of-00101.safetensors", + "model.layers.47.mlp.experts.70.gate_proj.weight": "model-00051-of-00101.safetensors", + "model.layers.47.mlp.experts.70.up_proj.weight": "model-00051-of-00101.safetensors", + "model.layers.47.mlp.experts.70.down_proj.weight": "model-00051-of-00101.safetensors", + "model.layers.47.mlp.experts.71.gate_proj.weight": "model-00051-of-00101.safetensors", + "model.layers.47.mlp.experts.71.up_proj.weight": "model-00051-of-00101.safetensors", + "model.layers.47.mlp.experts.71.down_proj.weight": "model-00051-of-00101.safetensors", + "model.layers.47.mlp.experts.72.gate_proj.weight": "model-00051-of-00101.safetensors", + "model.layers.47.mlp.experts.72.up_proj.weight": "model-00051-of-00101.safetensors", + "model.layers.47.mlp.experts.72.down_proj.weight": "model-00051-of-00101.safetensors", + "model.layers.47.mlp.experts.73.gate_proj.weight": "model-00051-of-00101.safetensors", + "model.layers.47.mlp.experts.73.up_proj.weight": "model-00051-of-00101.safetensors", + "model.layers.47.mlp.experts.73.down_proj.weight": "model-00051-of-00101.safetensors", + "model.layers.47.mlp.experts.74.gate_proj.weight": "model-00051-of-00101.safetensors", + "model.layers.47.mlp.experts.74.up_proj.weight": "model-00051-of-00101.safetensors", + "model.layers.47.mlp.experts.74.down_proj.weight": "model-00051-of-00101.safetensors", + "model.layers.47.mlp.experts.75.gate_proj.weight": "model-00051-of-00101.safetensors", + "model.layers.47.mlp.experts.75.up_proj.weight": "model-00051-of-00101.safetensors", + "model.layers.47.mlp.experts.75.down_proj.weight": "model-00051-of-00101.safetensors", + "model.layers.47.mlp.experts.76.gate_proj.weight": "model-00051-of-00101.safetensors", + "model.layers.47.mlp.experts.76.up_proj.weight": "model-00051-of-00101.safetensors", + "model.layers.47.mlp.experts.76.down_proj.weight": "model-00051-of-00101.safetensors", + "model.layers.47.mlp.experts.77.gate_proj.weight": "model-00051-of-00101.safetensors", + "model.layers.47.mlp.experts.77.up_proj.weight": "model-00051-of-00101.safetensors", + "model.layers.47.mlp.experts.77.down_proj.weight": "model-00051-of-00101.safetensors", + "model.layers.47.mlp.experts.78.gate_proj.weight": "model-00051-of-00101.safetensors", + "model.layers.47.mlp.experts.78.up_proj.weight": "model-00051-of-00101.safetensors", + "model.layers.47.mlp.experts.78.down_proj.weight": "model-00051-of-00101.safetensors", + "model.layers.47.mlp.experts.79.gate_proj.weight": "model-00051-of-00101.safetensors", + "model.layers.47.mlp.experts.79.up_proj.weight": "model-00051-of-00101.safetensors", + "model.layers.47.mlp.experts.79.down_proj.weight": "model-00051-of-00101.safetensors", + "model.layers.47.mlp.experts.80.gate_proj.weight": "model-00051-of-00101.safetensors", + "model.layers.47.mlp.experts.80.up_proj.weight": "model-00051-of-00101.safetensors", + "model.layers.47.mlp.experts.80.down_proj.weight": "model-00051-of-00101.safetensors", + "model.layers.47.mlp.experts.81.gate_proj.weight": "model-00051-of-00101.safetensors", + "model.layers.47.mlp.experts.81.up_proj.weight": "model-00051-of-00101.safetensors", + "model.layers.47.mlp.experts.81.down_proj.weight": "model-00051-of-00101.safetensors", + "model.layers.47.mlp.experts.82.gate_proj.weight": "model-00051-of-00101.safetensors", + "model.layers.47.mlp.experts.82.up_proj.weight": "model-00051-of-00101.safetensors", + "model.layers.47.mlp.experts.82.down_proj.weight": "model-00051-of-00101.safetensors", + "model.layers.47.mlp.experts.83.gate_proj.weight": "model-00051-of-00101.safetensors", + "model.layers.47.mlp.experts.83.up_proj.weight": "model-00051-of-00101.safetensors", + "model.layers.47.mlp.experts.83.down_proj.weight": "model-00051-of-00101.safetensors", + "model.layers.47.mlp.experts.84.gate_proj.weight": "model-00051-of-00101.safetensors", + "model.layers.47.mlp.experts.84.up_proj.weight": "model-00051-of-00101.safetensors", + "model.layers.47.mlp.experts.84.down_proj.weight": "model-00051-of-00101.safetensors", + "model.layers.47.mlp.experts.85.gate_proj.weight": "model-00051-of-00101.safetensors", + "model.layers.47.mlp.experts.85.up_proj.weight": "model-00051-of-00101.safetensors", + "model.layers.47.mlp.experts.85.down_proj.weight": "model-00051-of-00101.safetensors", + "model.layers.47.mlp.experts.86.gate_proj.weight": "model-00051-of-00101.safetensors", + "model.layers.47.mlp.experts.86.up_proj.weight": "model-00051-of-00101.safetensors", + "model.layers.47.mlp.experts.86.down_proj.weight": "model-00051-of-00101.safetensors", + "model.layers.47.mlp.experts.87.gate_proj.weight": "model-00051-of-00101.safetensors", + "model.layers.47.mlp.experts.87.up_proj.weight": "model-00051-of-00101.safetensors", + "model.layers.47.mlp.experts.87.down_proj.weight": "model-00051-of-00101.safetensors", + "model.layers.47.mlp.experts.88.gate_proj.weight": "model-00051-of-00101.safetensors", + "model.layers.47.mlp.experts.88.up_proj.weight": "model-00051-of-00101.safetensors", + "model.layers.47.mlp.experts.88.down_proj.weight": "model-00051-of-00101.safetensors", + "model.layers.47.mlp.experts.89.gate_proj.weight": "model-00051-of-00101.safetensors", + "model.layers.47.mlp.experts.89.up_proj.weight": "model-00051-of-00101.safetensors", + "model.layers.47.mlp.experts.89.down_proj.weight": "model-00051-of-00101.safetensors", + "model.layers.47.mlp.experts.90.gate_proj.weight": "model-00051-of-00101.safetensors", + "model.layers.47.mlp.experts.90.up_proj.weight": "model-00051-of-00101.safetensors", + "model.layers.47.mlp.experts.90.down_proj.weight": "model-00051-of-00101.safetensors", + "model.layers.47.mlp.experts.91.gate_proj.weight": "model-00051-of-00101.safetensors", + "model.layers.47.mlp.experts.91.up_proj.weight": "model-00051-of-00101.safetensors", + "model.layers.47.mlp.experts.91.down_proj.weight": "model-00051-of-00101.safetensors", + "model.layers.47.mlp.experts.92.gate_proj.weight": "model-00051-of-00101.safetensors", + "model.layers.47.mlp.experts.92.up_proj.weight": "model-00051-of-00101.safetensors", + "model.layers.47.mlp.experts.92.down_proj.weight": "model-00051-of-00101.safetensors", + "model.layers.47.mlp.experts.93.gate_proj.weight": "model-00051-of-00101.safetensors", + "model.layers.47.mlp.experts.93.up_proj.weight": "model-00051-of-00101.safetensors", + "model.layers.47.mlp.experts.93.down_proj.weight": "model-00051-of-00101.safetensors", + "model.layers.47.mlp.experts.94.gate_proj.weight": "model-00051-of-00101.safetensors", + "model.layers.47.mlp.experts.94.up_proj.weight": "model-00051-of-00101.safetensors", + "model.layers.47.mlp.experts.94.down_proj.weight": "model-00051-of-00101.safetensors", + "model.layers.47.mlp.experts.95.gate_proj.weight": "model-00051-of-00101.safetensors", + "model.layers.47.mlp.experts.95.up_proj.weight": "model-00051-of-00101.safetensors", + "model.layers.47.mlp.experts.95.down_proj.weight": "model-00051-of-00101.safetensors", + "model.layers.47.mlp.experts.96.gate_proj.weight": "model-00051-of-00101.safetensors", + "model.layers.47.mlp.experts.96.up_proj.weight": "model-00051-of-00101.safetensors", + "model.layers.47.mlp.experts.96.down_proj.weight": "model-00051-of-00101.safetensors", + "model.layers.47.mlp.experts.97.gate_proj.weight": "model-00051-of-00101.safetensors", + "model.layers.47.mlp.experts.97.up_proj.weight": "model-00051-of-00101.safetensors", + "model.layers.47.mlp.experts.97.down_proj.weight": "model-00051-of-00101.safetensors", + "model.layers.47.mlp.experts.98.gate_proj.weight": "model-00051-of-00101.safetensors", + "model.layers.47.mlp.experts.98.up_proj.weight": "model-00051-of-00101.safetensors", + "model.layers.47.mlp.experts.98.down_proj.weight": "model-00051-of-00101.safetensors", + "model.layers.47.mlp.experts.99.gate_proj.weight": "model-00051-of-00101.safetensors", + "model.layers.47.mlp.experts.99.up_proj.weight": "model-00051-of-00101.safetensors", + "model.layers.47.mlp.experts.99.down_proj.weight": "model-00051-of-00101.safetensors", + "model.layers.47.mlp.experts.100.gate_proj.weight": "model-00051-of-00101.safetensors", + "model.layers.47.mlp.experts.100.up_proj.weight": "model-00051-of-00101.safetensors", + "model.layers.47.mlp.experts.100.down_proj.weight": "model-00051-of-00101.safetensors", + "model.layers.47.mlp.experts.101.gate_proj.weight": "model-00051-of-00101.safetensors", + "model.layers.47.mlp.experts.101.up_proj.weight": "model-00051-of-00101.safetensors", + "model.layers.47.mlp.experts.101.down_proj.weight": "model-00051-of-00101.safetensors", + "model.layers.47.mlp.experts.102.gate_proj.weight": "model-00051-of-00101.safetensors", + "model.layers.47.mlp.experts.102.up_proj.weight": "model-00051-of-00101.safetensors", + "model.layers.47.mlp.experts.102.down_proj.weight": "model-00051-of-00101.safetensors", + "model.layers.47.mlp.experts.103.gate_proj.weight": "model-00051-of-00101.safetensors", + "model.layers.47.mlp.experts.103.up_proj.weight": "model-00051-of-00101.safetensors", + "model.layers.47.mlp.experts.103.down_proj.weight": "model-00051-of-00101.safetensors", + "model.layers.47.mlp.experts.104.gate_proj.weight": "model-00051-of-00101.safetensors", + "model.layers.47.mlp.experts.104.up_proj.weight": "model-00051-of-00101.safetensors", + "model.layers.47.mlp.experts.104.down_proj.weight": "model-00051-of-00101.safetensors", + "model.layers.47.mlp.experts.105.gate_proj.weight": "model-00051-of-00101.safetensors", + "model.layers.47.mlp.experts.105.up_proj.weight": "model-00051-of-00101.safetensors", + "model.layers.47.mlp.experts.105.down_proj.weight": "model-00051-of-00101.safetensors", + "model.layers.47.mlp.experts.106.gate_proj.weight": "model-00051-of-00101.safetensors", + "model.layers.47.mlp.experts.106.up_proj.weight": "model-00051-of-00101.safetensors", + "model.layers.47.mlp.experts.106.down_proj.weight": "model-00051-of-00101.safetensors", + "model.layers.47.mlp.experts.107.gate_proj.weight": "model-00051-of-00101.safetensors", + "model.layers.47.mlp.experts.107.up_proj.weight": "model-00051-of-00101.safetensors", + "model.layers.47.mlp.experts.107.down_proj.weight": "model-00051-of-00101.safetensors", + "model.layers.47.mlp.experts.108.gate_proj.weight": "model-00051-of-00101.safetensors", + "model.layers.47.mlp.experts.108.up_proj.weight": "model-00051-of-00101.safetensors", + "model.layers.47.mlp.experts.108.down_proj.weight": "model-00051-of-00101.safetensors", + "model.layers.47.mlp.experts.109.gate_proj.weight": "model-00051-of-00101.safetensors", + "model.layers.47.mlp.experts.109.up_proj.weight": "model-00051-of-00101.safetensors", + "model.layers.47.mlp.experts.109.down_proj.weight": "model-00051-of-00101.safetensors", + "model.layers.47.mlp.experts.110.gate_proj.weight": "model-00051-of-00101.safetensors", + "model.layers.47.mlp.experts.110.up_proj.weight": "model-00051-of-00101.safetensors", + "model.layers.47.mlp.experts.110.down_proj.weight": "model-00051-of-00101.safetensors", + "model.layers.47.mlp.experts.111.gate_proj.weight": "model-00051-of-00101.safetensors", + "model.layers.47.mlp.experts.111.up_proj.weight": "model-00051-of-00101.safetensors", + "model.layers.47.mlp.experts.111.down_proj.weight": "model-00051-of-00101.safetensors", + "model.layers.47.mlp.experts.112.gate_proj.weight": "model-00051-of-00101.safetensors", + "model.layers.47.mlp.experts.112.up_proj.weight": "model-00051-of-00101.safetensors", + "model.layers.47.mlp.experts.112.down_proj.weight": "model-00051-of-00101.safetensors", + "model.layers.47.mlp.experts.113.gate_proj.weight": "model-00051-of-00101.safetensors", + "model.layers.47.mlp.experts.113.up_proj.weight": "model-00051-of-00101.safetensors", + "model.layers.47.mlp.experts.113.down_proj.weight": "model-00051-of-00101.safetensors", + "model.layers.47.mlp.experts.114.gate_proj.weight": "model-00051-of-00101.safetensors", + "model.layers.47.mlp.experts.114.up_proj.weight": "model-00051-of-00101.safetensors", + "model.layers.47.mlp.experts.114.down_proj.weight": "model-00051-of-00101.safetensors", + "model.layers.47.mlp.experts.115.gate_proj.weight": "model-00051-of-00101.safetensors", + "model.layers.47.mlp.experts.115.up_proj.weight": "model-00051-of-00101.safetensors", + "model.layers.47.mlp.experts.115.down_proj.weight": "model-00051-of-00101.safetensors", + "model.layers.47.mlp.experts.116.gate_proj.weight": "model-00051-of-00101.safetensors", + "model.layers.47.mlp.experts.116.up_proj.weight": "model-00051-of-00101.safetensors", + "model.layers.47.mlp.experts.116.down_proj.weight": "model-00051-of-00101.safetensors", + "model.layers.47.mlp.experts.117.gate_proj.weight": "model-00051-of-00101.safetensors", + "model.layers.47.mlp.experts.117.up_proj.weight": "model-00051-of-00101.safetensors", + "model.layers.47.mlp.experts.117.down_proj.weight": "model-00051-of-00101.safetensors", + "model.layers.47.mlp.experts.118.gate_proj.weight": "model-00051-of-00101.safetensors", + "model.layers.47.mlp.experts.118.up_proj.weight": "model-00051-of-00101.safetensors", + "model.layers.47.mlp.experts.118.down_proj.weight": "model-00051-of-00101.safetensors", + "model.layers.47.mlp.experts.119.gate_proj.weight": "model-00051-of-00101.safetensors", + "model.layers.47.mlp.experts.119.up_proj.weight": "model-00051-of-00101.safetensors", + "model.layers.47.mlp.experts.119.down_proj.weight": "model-00051-of-00101.safetensors", + "model.layers.47.mlp.gate.weight": "model-00051-of-00101.safetensors", + "model.layers.47.mlp.gate.e_score_correction_bias": "model-00051-of-00101.safetensors", + "model.layers.47.mlp.shared_experts.gate_proj.weight": "model-00051-of-00101.safetensors", + "model.layers.47.mlp.shared_experts.up_proj.weight": "model-00051-of-00101.safetensors", + "model.layers.47.mlp.shared_experts.down_proj.weight": "model-00051-of-00101.safetensors", + "model.layers.47.input_layernorm.weight": "model-00051-of-00101.safetensors", + "model.layers.47.post_attention_layernorm.weight": "model-00051-of-00101.safetensors", + "model.layers.48.self_attn.q_proj.weight": "model-00051-of-00101.safetensors", + "model.layers.48.self_attn.q_proj.bias": "model-00051-of-00101.safetensors", + "model.layers.48.self_attn.k_proj.weight": "model-00051-of-00101.safetensors", + "model.layers.48.self_attn.k_proj.bias": "model-00051-of-00101.safetensors", + "model.layers.48.self_attn.v_proj.weight": "model-00051-of-00101.safetensors", + "model.layers.48.self_attn.v_proj.bias": "model-00051-of-00101.safetensors", + "model.layers.48.self_attn.o_proj.weight": "model-00051-of-00101.safetensors", + "model.layers.48.self_attn.q_norm.weight": "model-00051-of-00101.safetensors", + "model.layers.48.self_attn.k_norm.weight": "model-00051-of-00101.safetensors", + "model.layers.48.mlp.experts.0.gate_proj.weight": "model-00051-of-00101.safetensors", + "model.layers.48.mlp.experts.0.up_proj.weight": "model-00051-of-00101.safetensors", + "model.layers.48.mlp.experts.0.down_proj.weight": "model-00051-of-00101.safetensors", + "model.layers.48.mlp.experts.1.gate_proj.weight": "model-00052-of-00101.safetensors", + "model.layers.48.mlp.experts.1.up_proj.weight": "model-00052-of-00101.safetensors", + "model.layers.48.mlp.experts.1.down_proj.weight": "model-00052-of-00101.safetensors", + "model.layers.48.mlp.experts.2.gate_proj.weight": "model-00052-of-00101.safetensors", + "model.layers.48.mlp.experts.2.up_proj.weight": "model-00052-of-00101.safetensors", + "model.layers.48.mlp.experts.2.down_proj.weight": "model-00052-of-00101.safetensors", + "model.layers.48.mlp.experts.3.gate_proj.weight": "model-00052-of-00101.safetensors", + "model.layers.48.mlp.experts.3.up_proj.weight": "model-00052-of-00101.safetensors", + "model.layers.48.mlp.experts.3.down_proj.weight": "model-00052-of-00101.safetensors", + "model.layers.48.mlp.experts.4.gate_proj.weight": "model-00052-of-00101.safetensors", + "model.layers.48.mlp.experts.4.up_proj.weight": "model-00052-of-00101.safetensors", + "model.layers.48.mlp.experts.4.down_proj.weight": "model-00052-of-00101.safetensors", + "model.layers.48.mlp.experts.5.gate_proj.weight": "model-00052-of-00101.safetensors", + "model.layers.48.mlp.experts.5.up_proj.weight": "model-00052-of-00101.safetensors", + "model.layers.48.mlp.experts.5.down_proj.weight": "model-00052-of-00101.safetensors", + "model.layers.48.mlp.experts.6.gate_proj.weight": "model-00052-of-00101.safetensors", + "model.layers.48.mlp.experts.6.up_proj.weight": "model-00052-of-00101.safetensors", + "model.layers.48.mlp.experts.6.down_proj.weight": "model-00052-of-00101.safetensors", + "model.layers.48.mlp.experts.7.gate_proj.weight": "model-00052-of-00101.safetensors", + "model.layers.48.mlp.experts.7.up_proj.weight": "model-00052-of-00101.safetensors", + "model.layers.48.mlp.experts.7.down_proj.weight": "model-00052-of-00101.safetensors", + "model.layers.48.mlp.experts.8.gate_proj.weight": "model-00052-of-00101.safetensors", + "model.layers.48.mlp.experts.8.up_proj.weight": "model-00052-of-00101.safetensors", + "model.layers.48.mlp.experts.8.down_proj.weight": "model-00052-of-00101.safetensors", + "model.layers.48.mlp.experts.9.gate_proj.weight": "model-00052-of-00101.safetensors", + "model.layers.48.mlp.experts.9.up_proj.weight": "model-00052-of-00101.safetensors", + "model.layers.48.mlp.experts.9.down_proj.weight": "model-00052-of-00101.safetensors", + "model.layers.48.mlp.experts.10.gate_proj.weight": "model-00052-of-00101.safetensors", + "model.layers.48.mlp.experts.10.up_proj.weight": "model-00052-of-00101.safetensors", + "model.layers.48.mlp.experts.10.down_proj.weight": "model-00052-of-00101.safetensors", + "model.layers.48.mlp.experts.11.gate_proj.weight": "model-00052-of-00101.safetensors", + "model.layers.48.mlp.experts.11.up_proj.weight": "model-00052-of-00101.safetensors", + "model.layers.48.mlp.experts.11.down_proj.weight": "model-00052-of-00101.safetensors", + "model.layers.48.mlp.experts.12.gate_proj.weight": "model-00052-of-00101.safetensors", + "model.layers.48.mlp.experts.12.up_proj.weight": "model-00052-of-00101.safetensors", + "model.layers.48.mlp.experts.12.down_proj.weight": "model-00052-of-00101.safetensors", + "model.layers.48.mlp.experts.13.gate_proj.weight": "model-00052-of-00101.safetensors", + "model.layers.48.mlp.experts.13.up_proj.weight": "model-00052-of-00101.safetensors", + "model.layers.48.mlp.experts.13.down_proj.weight": "model-00052-of-00101.safetensors", + "model.layers.48.mlp.experts.14.gate_proj.weight": "model-00052-of-00101.safetensors", + "model.layers.48.mlp.experts.14.up_proj.weight": "model-00052-of-00101.safetensors", + "model.layers.48.mlp.experts.14.down_proj.weight": "model-00052-of-00101.safetensors", + "model.layers.48.mlp.experts.15.gate_proj.weight": "model-00052-of-00101.safetensors", + "model.layers.48.mlp.experts.15.up_proj.weight": "model-00052-of-00101.safetensors", + "model.layers.48.mlp.experts.15.down_proj.weight": "model-00052-of-00101.safetensors", + "model.layers.48.mlp.experts.16.gate_proj.weight": "model-00052-of-00101.safetensors", + "model.layers.48.mlp.experts.16.up_proj.weight": "model-00052-of-00101.safetensors", + "model.layers.48.mlp.experts.16.down_proj.weight": "model-00052-of-00101.safetensors", + "model.layers.48.mlp.experts.17.gate_proj.weight": "model-00052-of-00101.safetensors", + "model.layers.48.mlp.experts.17.up_proj.weight": "model-00052-of-00101.safetensors", + "model.layers.48.mlp.experts.17.down_proj.weight": "model-00052-of-00101.safetensors", + "model.layers.48.mlp.experts.18.gate_proj.weight": "model-00052-of-00101.safetensors", + "model.layers.48.mlp.experts.18.up_proj.weight": "model-00052-of-00101.safetensors", + "model.layers.48.mlp.experts.18.down_proj.weight": "model-00052-of-00101.safetensors", + "model.layers.48.mlp.experts.19.gate_proj.weight": "model-00052-of-00101.safetensors", + "model.layers.48.mlp.experts.19.up_proj.weight": "model-00052-of-00101.safetensors", + "model.layers.48.mlp.experts.19.down_proj.weight": "model-00052-of-00101.safetensors", + "model.layers.48.mlp.experts.20.gate_proj.weight": "model-00052-of-00101.safetensors", + "model.layers.48.mlp.experts.20.up_proj.weight": "model-00052-of-00101.safetensors", + "model.layers.48.mlp.experts.20.down_proj.weight": "model-00052-of-00101.safetensors", + "model.layers.48.mlp.experts.21.gate_proj.weight": "model-00052-of-00101.safetensors", + "model.layers.48.mlp.experts.21.up_proj.weight": "model-00052-of-00101.safetensors", + "model.layers.48.mlp.experts.21.down_proj.weight": "model-00052-of-00101.safetensors", + "model.layers.48.mlp.experts.22.gate_proj.weight": "model-00052-of-00101.safetensors", + "model.layers.48.mlp.experts.22.up_proj.weight": "model-00052-of-00101.safetensors", + "model.layers.48.mlp.experts.22.down_proj.weight": "model-00052-of-00101.safetensors", + "model.layers.48.mlp.experts.23.gate_proj.weight": "model-00052-of-00101.safetensors", + "model.layers.48.mlp.experts.23.up_proj.weight": "model-00052-of-00101.safetensors", + "model.layers.48.mlp.experts.23.down_proj.weight": "model-00052-of-00101.safetensors", + "model.layers.48.mlp.experts.24.gate_proj.weight": "model-00052-of-00101.safetensors", + "model.layers.48.mlp.experts.24.up_proj.weight": "model-00052-of-00101.safetensors", + "model.layers.48.mlp.experts.24.down_proj.weight": "model-00052-of-00101.safetensors", + "model.layers.48.mlp.experts.25.gate_proj.weight": "model-00052-of-00101.safetensors", + "model.layers.48.mlp.experts.25.up_proj.weight": "model-00052-of-00101.safetensors", + "model.layers.48.mlp.experts.25.down_proj.weight": "model-00052-of-00101.safetensors", + "model.layers.48.mlp.experts.26.gate_proj.weight": "model-00052-of-00101.safetensors", + "model.layers.48.mlp.experts.26.up_proj.weight": "model-00052-of-00101.safetensors", + "model.layers.48.mlp.experts.26.down_proj.weight": "model-00052-of-00101.safetensors", + "model.layers.48.mlp.experts.27.gate_proj.weight": "model-00052-of-00101.safetensors", + "model.layers.48.mlp.experts.27.up_proj.weight": "model-00052-of-00101.safetensors", + "model.layers.48.mlp.experts.27.down_proj.weight": "model-00052-of-00101.safetensors", + "model.layers.48.mlp.experts.28.gate_proj.weight": "model-00052-of-00101.safetensors", + "model.layers.48.mlp.experts.28.up_proj.weight": "model-00052-of-00101.safetensors", + "model.layers.48.mlp.experts.28.down_proj.weight": "model-00052-of-00101.safetensors", + "model.layers.48.mlp.experts.29.gate_proj.weight": "model-00052-of-00101.safetensors", + "model.layers.48.mlp.experts.29.up_proj.weight": "model-00052-of-00101.safetensors", + "model.layers.48.mlp.experts.29.down_proj.weight": "model-00052-of-00101.safetensors", + "model.layers.48.mlp.experts.30.gate_proj.weight": "model-00052-of-00101.safetensors", + "model.layers.48.mlp.experts.30.up_proj.weight": "model-00052-of-00101.safetensors", + "model.layers.48.mlp.experts.30.down_proj.weight": "model-00052-of-00101.safetensors", + "model.layers.48.mlp.experts.31.gate_proj.weight": "model-00052-of-00101.safetensors", + "model.layers.48.mlp.experts.31.up_proj.weight": "model-00052-of-00101.safetensors", + "model.layers.48.mlp.experts.31.down_proj.weight": "model-00052-of-00101.safetensors", + "model.layers.48.mlp.experts.32.gate_proj.weight": "model-00052-of-00101.safetensors", + "model.layers.48.mlp.experts.32.up_proj.weight": "model-00052-of-00101.safetensors", + "model.layers.48.mlp.experts.32.down_proj.weight": "model-00052-of-00101.safetensors", + "model.layers.48.mlp.experts.33.gate_proj.weight": "model-00052-of-00101.safetensors", + "model.layers.48.mlp.experts.33.up_proj.weight": "model-00052-of-00101.safetensors", + "model.layers.48.mlp.experts.33.down_proj.weight": "model-00052-of-00101.safetensors", + "model.layers.48.mlp.experts.34.gate_proj.weight": "model-00052-of-00101.safetensors", + "model.layers.48.mlp.experts.34.up_proj.weight": "model-00052-of-00101.safetensors", + "model.layers.48.mlp.experts.34.down_proj.weight": "model-00052-of-00101.safetensors", + "model.layers.48.mlp.experts.35.gate_proj.weight": "model-00052-of-00101.safetensors", + "model.layers.48.mlp.experts.35.up_proj.weight": "model-00052-of-00101.safetensors", + "model.layers.48.mlp.experts.35.down_proj.weight": "model-00052-of-00101.safetensors", + "model.layers.48.mlp.experts.36.gate_proj.weight": "model-00052-of-00101.safetensors", + "model.layers.48.mlp.experts.36.up_proj.weight": "model-00052-of-00101.safetensors", + "model.layers.48.mlp.experts.36.down_proj.weight": "model-00052-of-00101.safetensors", + "model.layers.48.mlp.experts.37.gate_proj.weight": "model-00052-of-00101.safetensors", + "model.layers.48.mlp.experts.37.up_proj.weight": "model-00052-of-00101.safetensors", + "model.layers.48.mlp.experts.37.down_proj.weight": "model-00052-of-00101.safetensors", + "model.layers.48.mlp.experts.38.gate_proj.weight": "model-00052-of-00101.safetensors", + "model.layers.48.mlp.experts.38.up_proj.weight": "model-00052-of-00101.safetensors", + "model.layers.48.mlp.experts.38.down_proj.weight": "model-00052-of-00101.safetensors", + "model.layers.48.mlp.experts.39.gate_proj.weight": "model-00052-of-00101.safetensors", + "model.layers.48.mlp.experts.39.up_proj.weight": "model-00052-of-00101.safetensors", + "model.layers.48.mlp.experts.39.down_proj.weight": "model-00052-of-00101.safetensors", + "model.layers.48.mlp.experts.40.gate_proj.weight": "model-00052-of-00101.safetensors", + "model.layers.48.mlp.experts.40.up_proj.weight": "model-00052-of-00101.safetensors", + "model.layers.48.mlp.experts.40.down_proj.weight": "model-00052-of-00101.safetensors", + "model.layers.48.mlp.experts.41.gate_proj.weight": "model-00052-of-00101.safetensors", + "model.layers.48.mlp.experts.41.up_proj.weight": "model-00052-of-00101.safetensors", + "model.layers.48.mlp.experts.41.down_proj.weight": "model-00052-of-00101.safetensors", + "model.layers.48.mlp.experts.42.gate_proj.weight": "model-00052-of-00101.safetensors", + "model.layers.48.mlp.experts.42.up_proj.weight": "model-00052-of-00101.safetensors", + "model.layers.48.mlp.experts.42.down_proj.weight": "model-00052-of-00101.safetensors", + "model.layers.48.mlp.experts.43.gate_proj.weight": "model-00052-of-00101.safetensors", + "model.layers.48.mlp.experts.43.up_proj.weight": "model-00052-of-00101.safetensors", + "model.layers.48.mlp.experts.43.down_proj.weight": "model-00052-of-00101.safetensors", + "model.layers.48.mlp.experts.44.gate_proj.weight": "model-00052-of-00101.safetensors", + "model.layers.48.mlp.experts.44.up_proj.weight": "model-00052-of-00101.safetensors", + "model.layers.48.mlp.experts.44.down_proj.weight": "model-00052-of-00101.safetensors", + "model.layers.48.mlp.experts.45.gate_proj.weight": "model-00052-of-00101.safetensors", + "model.layers.48.mlp.experts.45.up_proj.weight": "model-00052-of-00101.safetensors", + "model.layers.48.mlp.experts.45.down_proj.weight": "model-00052-of-00101.safetensors", + "model.layers.48.mlp.experts.46.gate_proj.weight": "model-00052-of-00101.safetensors", + "model.layers.48.mlp.experts.46.up_proj.weight": "model-00052-of-00101.safetensors", + "model.layers.48.mlp.experts.46.down_proj.weight": "model-00052-of-00101.safetensors", + "model.layers.48.mlp.experts.47.gate_proj.weight": "model-00052-of-00101.safetensors", + "model.layers.48.mlp.experts.47.up_proj.weight": "model-00052-of-00101.safetensors", + "model.layers.48.mlp.experts.47.down_proj.weight": "model-00052-of-00101.safetensors", + "model.layers.48.mlp.experts.48.gate_proj.weight": "model-00052-of-00101.safetensors", + "model.layers.48.mlp.experts.48.up_proj.weight": "model-00052-of-00101.safetensors", + "model.layers.48.mlp.experts.48.down_proj.weight": "model-00052-of-00101.safetensors", + "model.layers.48.mlp.experts.49.gate_proj.weight": "model-00052-of-00101.safetensors", + "model.layers.48.mlp.experts.49.up_proj.weight": "model-00052-of-00101.safetensors", + "model.layers.48.mlp.experts.49.down_proj.weight": "model-00052-of-00101.safetensors", + "model.layers.48.mlp.experts.50.gate_proj.weight": "model-00052-of-00101.safetensors", + "model.layers.48.mlp.experts.50.up_proj.weight": "model-00052-of-00101.safetensors", + "model.layers.48.mlp.experts.50.down_proj.weight": "model-00052-of-00101.safetensors", + "model.layers.48.mlp.experts.51.gate_proj.weight": "model-00052-of-00101.safetensors", + "model.layers.48.mlp.experts.51.up_proj.weight": "model-00052-of-00101.safetensors", + "model.layers.48.mlp.experts.51.down_proj.weight": "model-00052-of-00101.safetensors", + "model.layers.48.mlp.experts.52.gate_proj.weight": "model-00052-of-00101.safetensors", + "model.layers.48.mlp.experts.52.up_proj.weight": "model-00052-of-00101.safetensors", + "model.layers.48.mlp.experts.52.down_proj.weight": "model-00052-of-00101.safetensors", + "model.layers.48.mlp.experts.53.gate_proj.weight": "model-00052-of-00101.safetensors", + "model.layers.48.mlp.experts.53.up_proj.weight": "model-00052-of-00101.safetensors", + "model.layers.48.mlp.experts.53.down_proj.weight": "model-00052-of-00101.safetensors", + "model.layers.48.mlp.experts.54.gate_proj.weight": "model-00052-of-00101.safetensors", + "model.layers.48.mlp.experts.54.up_proj.weight": "model-00052-of-00101.safetensors", + "model.layers.48.mlp.experts.54.down_proj.weight": "model-00052-of-00101.safetensors", + "model.layers.48.mlp.experts.55.gate_proj.weight": "model-00052-of-00101.safetensors", + "model.layers.48.mlp.experts.55.up_proj.weight": "model-00052-of-00101.safetensors", + "model.layers.48.mlp.experts.55.down_proj.weight": "model-00052-of-00101.safetensors", + "model.layers.48.mlp.experts.56.gate_proj.weight": "model-00052-of-00101.safetensors", + "model.layers.48.mlp.experts.56.up_proj.weight": "model-00052-of-00101.safetensors", + "model.layers.48.mlp.experts.56.down_proj.weight": "model-00052-of-00101.safetensors", + "model.layers.48.mlp.experts.57.gate_proj.weight": "model-00052-of-00101.safetensors", + "model.layers.48.mlp.experts.57.up_proj.weight": "model-00052-of-00101.safetensors", + "model.layers.48.mlp.experts.57.down_proj.weight": "model-00052-of-00101.safetensors", + "model.layers.48.mlp.experts.58.gate_proj.weight": "model-00052-of-00101.safetensors", + "model.layers.48.mlp.experts.58.up_proj.weight": "model-00052-of-00101.safetensors", + "model.layers.48.mlp.experts.58.down_proj.weight": "model-00052-of-00101.safetensors", + "model.layers.48.mlp.experts.59.gate_proj.weight": "model-00052-of-00101.safetensors", + "model.layers.48.mlp.experts.59.up_proj.weight": "model-00052-of-00101.safetensors", + "model.layers.48.mlp.experts.59.down_proj.weight": "model-00052-of-00101.safetensors", + "model.layers.48.mlp.experts.60.gate_proj.weight": "model-00052-of-00101.safetensors", + "model.layers.48.mlp.experts.60.up_proj.weight": "model-00052-of-00101.safetensors", + "model.layers.48.mlp.experts.60.down_proj.weight": "model-00052-of-00101.safetensors", + "model.layers.48.mlp.experts.61.gate_proj.weight": "model-00052-of-00101.safetensors", + "model.layers.48.mlp.experts.61.up_proj.weight": "model-00052-of-00101.safetensors", + "model.layers.48.mlp.experts.61.down_proj.weight": "model-00052-of-00101.safetensors", + "model.layers.48.mlp.experts.62.gate_proj.weight": "model-00052-of-00101.safetensors", + "model.layers.48.mlp.experts.62.up_proj.weight": "model-00052-of-00101.safetensors", + "model.layers.48.mlp.experts.62.down_proj.weight": "model-00052-of-00101.safetensors", + "model.layers.48.mlp.experts.63.gate_proj.weight": "model-00052-of-00101.safetensors", + "model.layers.48.mlp.experts.63.up_proj.weight": "model-00052-of-00101.safetensors", + "model.layers.48.mlp.experts.63.down_proj.weight": "model-00052-of-00101.safetensors", + "model.layers.48.mlp.experts.64.gate_proj.weight": "model-00052-of-00101.safetensors", + "model.layers.48.mlp.experts.64.up_proj.weight": "model-00052-of-00101.safetensors", + "model.layers.48.mlp.experts.64.down_proj.weight": "model-00052-of-00101.safetensors", + "model.layers.48.mlp.experts.65.gate_proj.weight": "model-00052-of-00101.safetensors", + "model.layers.48.mlp.experts.65.up_proj.weight": "model-00052-of-00101.safetensors", + "model.layers.48.mlp.experts.65.down_proj.weight": "model-00052-of-00101.safetensors", + "model.layers.48.mlp.experts.66.gate_proj.weight": "model-00052-of-00101.safetensors", + "model.layers.48.mlp.experts.66.up_proj.weight": "model-00052-of-00101.safetensors", + "model.layers.48.mlp.experts.66.down_proj.weight": "model-00052-of-00101.safetensors", + "model.layers.48.mlp.experts.67.gate_proj.weight": "model-00052-of-00101.safetensors", + "model.layers.48.mlp.experts.67.up_proj.weight": "model-00052-of-00101.safetensors", + "model.layers.48.mlp.experts.67.down_proj.weight": "model-00052-of-00101.safetensors", + "model.layers.48.mlp.experts.68.gate_proj.weight": "model-00052-of-00101.safetensors", + "model.layers.48.mlp.experts.68.up_proj.weight": "model-00052-of-00101.safetensors", + "model.layers.48.mlp.experts.68.down_proj.weight": "model-00052-of-00101.safetensors", + "model.layers.48.mlp.experts.69.gate_proj.weight": "model-00052-of-00101.safetensors", + "model.layers.48.mlp.experts.69.up_proj.weight": "model-00052-of-00101.safetensors", + "model.layers.48.mlp.experts.69.down_proj.weight": "model-00052-of-00101.safetensors", + "model.layers.48.mlp.experts.70.gate_proj.weight": "model-00052-of-00101.safetensors", + "model.layers.48.mlp.experts.70.up_proj.weight": "model-00052-of-00101.safetensors", + "model.layers.48.mlp.experts.70.down_proj.weight": "model-00052-of-00101.safetensors", + "model.layers.48.mlp.experts.71.gate_proj.weight": "model-00052-of-00101.safetensors", + "model.layers.48.mlp.experts.71.up_proj.weight": "model-00052-of-00101.safetensors", + "model.layers.48.mlp.experts.71.down_proj.weight": "model-00052-of-00101.safetensors", + "model.layers.48.mlp.experts.72.gate_proj.weight": "model-00052-of-00101.safetensors", + "model.layers.48.mlp.experts.72.up_proj.weight": "model-00052-of-00101.safetensors", + "model.layers.48.mlp.experts.72.down_proj.weight": "model-00052-of-00101.safetensors", + "model.layers.48.mlp.experts.73.gate_proj.weight": "model-00052-of-00101.safetensors", + "model.layers.48.mlp.experts.73.up_proj.weight": "model-00052-of-00101.safetensors", + "model.layers.48.mlp.experts.73.down_proj.weight": "model-00052-of-00101.safetensors", + "model.layers.48.mlp.experts.74.gate_proj.weight": "model-00052-of-00101.safetensors", + "model.layers.48.mlp.experts.74.up_proj.weight": "model-00052-of-00101.safetensors", + "model.layers.48.mlp.experts.74.down_proj.weight": "model-00052-of-00101.safetensors", + "model.layers.48.mlp.experts.75.gate_proj.weight": "model-00052-of-00101.safetensors", + "model.layers.48.mlp.experts.75.up_proj.weight": "model-00052-of-00101.safetensors", + "model.layers.48.mlp.experts.75.down_proj.weight": "model-00052-of-00101.safetensors", + "model.layers.48.mlp.experts.76.gate_proj.weight": "model-00052-of-00101.safetensors", + "model.layers.48.mlp.experts.76.up_proj.weight": "model-00052-of-00101.safetensors", + "model.layers.48.mlp.experts.76.down_proj.weight": "model-00052-of-00101.safetensors", + "model.layers.48.mlp.experts.77.gate_proj.weight": "model-00052-of-00101.safetensors", + "model.layers.48.mlp.experts.77.up_proj.weight": "model-00052-of-00101.safetensors", + "model.layers.48.mlp.experts.77.down_proj.weight": "model-00052-of-00101.safetensors", + "model.layers.48.mlp.experts.78.gate_proj.weight": "model-00052-of-00101.safetensors", + "model.layers.48.mlp.experts.78.up_proj.weight": "model-00052-of-00101.safetensors", + "model.layers.48.mlp.experts.78.down_proj.weight": "model-00052-of-00101.safetensors", + "model.layers.48.mlp.experts.79.gate_proj.weight": "model-00052-of-00101.safetensors", + "model.layers.48.mlp.experts.79.up_proj.weight": "model-00052-of-00101.safetensors", + "model.layers.48.mlp.experts.79.down_proj.weight": "model-00052-of-00101.safetensors", + "model.layers.48.mlp.experts.80.gate_proj.weight": "model-00052-of-00101.safetensors", + "model.layers.48.mlp.experts.80.up_proj.weight": "model-00052-of-00101.safetensors", + "model.layers.48.mlp.experts.80.down_proj.weight": "model-00052-of-00101.safetensors", + "model.layers.48.mlp.experts.81.gate_proj.weight": "model-00052-of-00101.safetensors", + "model.layers.48.mlp.experts.81.up_proj.weight": "model-00052-of-00101.safetensors", + "model.layers.48.mlp.experts.81.down_proj.weight": "model-00052-of-00101.safetensors", + "model.layers.48.mlp.experts.82.gate_proj.weight": "model-00052-of-00101.safetensors", + "model.layers.48.mlp.experts.82.up_proj.weight": "model-00052-of-00101.safetensors", + "model.layers.48.mlp.experts.82.down_proj.weight": "model-00052-of-00101.safetensors", + "model.layers.48.mlp.experts.83.gate_proj.weight": "model-00052-of-00101.safetensors", + "model.layers.48.mlp.experts.83.up_proj.weight": "model-00052-of-00101.safetensors", + "model.layers.48.mlp.experts.83.down_proj.weight": "model-00052-of-00101.safetensors", + "model.layers.48.mlp.experts.84.gate_proj.weight": "model-00052-of-00101.safetensors", + "model.layers.48.mlp.experts.84.up_proj.weight": "model-00052-of-00101.safetensors", + "model.layers.48.mlp.experts.84.down_proj.weight": "model-00052-of-00101.safetensors", + "model.layers.48.mlp.experts.85.gate_proj.weight": "model-00052-of-00101.safetensors", + "model.layers.48.mlp.experts.85.up_proj.weight": "model-00052-of-00101.safetensors", + "model.layers.48.mlp.experts.85.down_proj.weight": "model-00052-of-00101.safetensors", + "model.layers.48.mlp.experts.86.gate_proj.weight": "model-00052-of-00101.safetensors", + "model.layers.48.mlp.experts.86.up_proj.weight": "model-00052-of-00101.safetensors", + "model.layers.48.mlp.experts.86.down_proj.weight": "model-00052-of-00101.safetensors", + "model.layers.48.mlp.experts.87.gate_proj.weight": "model-00052-of-00101.safetensors", + "model.layers.48.mlp.experts.87.up_proj.weight": "model-00052-of-00101.safetensors", + "model.layers.48.mlp.experts.87.down_proj.weight": "model-00052-of-00101.safetensors", + "model.layers.48.mlp.experts.88.gate_proj.weight": "model-00052-of-00101.safetensors", + "model.layers.48.mlp.experts.88.up_proj.weight": "model-00052-of-00101.safetensors", + "model.layers.48.mlp.experts.88.down_proj.weight": "model-00052-of-00101.safetensors", + "model.layers.48.mlp.experts.89.gate_proj.weight": "model-00052-of-00101.safetensors", + "model.layers.48.mlp.experts.89.up_proj.weight": "model-00052-of-00101.safetensors", + "model.layers.48.mlp.experts.89.down_proj.weight": "model-00052-of-00101.safetensors", + "model.layers.48.mlp.experts.90.gate_proj.weight": "model-00052-of-00101.safetensors", + "model.layers.48.mlp.experts.90.up_proj.weight": "model-00052-of-00101.safetensors", + "model.layers.48.mlp.experts.90.down_proj.weight": "model-00052-of-00101.safetensors", + "model.layers.48.mlp.experts.91.gate_proj.weight": "model-00052-of-00101.safetensors", + "model.layers.48.mlp.experts.91.up_proj.weight": "model-00052-of-00101.safetensors", + "model.layers.48.mlp.experts.91.down_proj.weight": "model-00052-of-00101.safetensors", + "model.layers.48.mlp.experts.92.gate_proj.weight": "model-00052-of-00101.safetensors", + "model.layers.48.mlp.experts.92.up_proj.weight": "model-00052-of-00101.safetensors", + "model.layers.48.mlp.experts.92.down_proj.weight": "model-00052-of-00101.safetensors", + "model.layers.48.mlp.experts.93.gate_proj.weight": "model-00052-of-00101.safetensors", + "model.layers.48.mlp.experts.93.up_proj.weight": "model-00052-of-00101.safetensors", + "model.layers.48.mlp.experts.93.down_proj.weight": "model-00052-of-00101.safetensors", + "model.layers.48.mlp.experts.94.gate_proj.weight": "model-00052-of-00101.safetensors", + "model.layers.48.mlp.experts.94.up_proj.weight": "model-00052-of-00101.safetensors", + "model.layers.48.mlp.experts.94.down_proj.weight": "model-00052-of-00101.safetensors", + "model.layers.48.mlp.experts.95.gate_proj.weight": "model-00052-of-00101.safetensors", + "model.layers.48.mlp.experts.95.up_proj.weight": "model-00052-of-00101.safetensors", + "model.layers.48.mlp.experts.95.down_proj.weight": "model-00052-of-00101.safetensors", + "model.layers.48.mlp.experts.96.gate_proj.weight": "model-00052-of-00101.safetensors", + "model.layers.48.mlp.experts.96.up_proj.weight": "model-00052-of-00101.safetensors", + "model.layers.48.mlp.experts.96.down_proj.weight": "model-00052-of-00101.safetensors", + "model.layers.48.mlp.experts.97.gate_proj.weight": "model-00052-of-00101.safetensors", + "model.layers.48.mlp.experts.97.up_proj.weight": "model-00052-of-00101.safetensors", + "model.layers.48.mlp.experts.97.down_proj.weight": "model-00052-of-00101.safetensors", + "model.layers.48.mlp.experts.98.gate_proj.weight": "model-00052-of-00101.safetensors", + "model.layers.48.mlp.experts.98.up_proj.weight": "model-00052-of-00101.safetensors", + "model.layers.48.mlp.experts.98.down_proj.weight": "model-00052-of-00101.safetensors", + "model.layers.48.mlp.experts.99.gate_proj.weight": "model-00052-of-00101.safetensors", + "model.layers.48.mlp.experts.99.up_proj.weight": "model-00052-of-00101.safetensors", + "model.layers.48.mlp.experts.99.down_proj.weight": "model-00052-of-00101.safetensors", + "model.layers.48.mlp.experts.100.gate_proj.weight": "model-00052-of-00101.safetensors", + "model.layers.48.mlp.experts.100.up_proj.weight": "model-00052-of-00101.safetensors", + "model.layers.48.mlp.experts.100.down_proj.weight": "model-00052-of-00101.safetensors", + "model.layers.48.mlp.experts.101.gate_proj.weight": "model-00052-of-00101.safetensors", + "model.layers.48.mlp.experts.101.up_proj.weight": "model-00052-of-00101.safetensors", + "model.layers.48.mlp.experts.101.down_proj.weight": "model-00052-of-00101.safetensors", + "model.layers.48.mlp.experts.102.gate_proj.weight": "model-00052-of-00101.safetensors", + "model.layers.48.mlp.experts.102.up_proj.weight": "model-00052-of-00101.safetensors", + "model.layers.48.mlp.experts.102.down_proj.weight": "model-00052-of-00101.safetensors", + "model.layers.48.mlp.experts.103.gate_proj.weight": "model-00052-of-00101.safetensors", + "model.layers.48.mlp.experts.103.up_proj.weight": "model-00052-of-00101.safetensors", + "model.layers.48.mlp.experts.103.down_proj.weight": "model-00052-of-00101.safetensors", + "model.layers.48.mlp.experts.104.gate_proj.weight": "model-00052-of-00101.safetensors", + "model.layers.48.mlp.experts.104.up_proj.weight": "model-00052-of-00101.safetensors", + "model.layers.48.mlp.experts.104.down_proj.weight": "model-00052-of-00101.safetensors", + "model.layers.48.mlp.experts.105.gate_proj.weight": "model-00052-of-00101.safetensors", + "model.layers.48.mlp.experts.105.up_proj.weight": "model-00052-of-00101.safetensors", + "model.layers.48.mlp.experts.105.down_proj.weight": "model-00052-of-00101.safetensors", + "model.layers.48.mlp.experts.106.gate_proj.weight": "model-00052-of-00101.safetensors", + "model.layers.48.mlp.experts.106.up_proj.weight": "model-00052-of-00101.safetensors", + "model.layers.48.mlp.experts.106.down_proj.weight": "model-00052-of-00101.safetensors", + "model.layers.48.mlp.experts.107.gate_proj.weight": "model-00052-of-00101.safetensors", + "model.layers.48.mlp.experts.107.up_proj.weight": "model-00052-of-00101.safetensors", + "model.layers.48.mlp.experts.107.down_proj.weight": "model-00052-of-00101.safetensors", + "model.layers.48.mlp.experts.108.gate_proj.weight": "model-00052-of-00101.safetensors", + "model.layers.48.mlp.experts.108.up_proj.weight": "model-00052-of-00101.safetensors", + "model.layers.48.mlp.experts.108.down_proj.weight": "model-00052-of-00101.safetensors", + "model.layers.48.mlp.experts.109.gate_proj.weight": "model-00052-of-00101.safetensors", + "model.layers.48.mlp.experts.109.up_proj.weight": "model-00052-of-00101.safetensors", + "model.layers.48.mlp.experts.109.down_proj.weight": "model-00052-of-00101.safetensors", + "model.layers.48.mlp.experts.110.gate_proj.weight": "model-00052-of-00101.safetensors", + "model.layers.48.mlp.experts.110.up_proj.weight": "model-00052-of-00101.safetensors", + "model.layers.48.mlp.experts.110.down_proj.weight": "model-00052-of-00101.safetensors", + "model.layers.48.mlp.experts.111.gate_proj.weight": "model-00052-of-00101.safetensors", + "model.layers.48.mlp.experts.111.up_proj.weight": "model-00052-of-00101.safetensors", + "model.layers.48.mlp.experts.111.down_proj.weight": "model-00052-of-00101.safetensors", + "model.layers.48.mlp.experts.112.gate_proj.weight": "model-00052-of-00101.safetensors", + "model.layers.48.mlp.experts.112.up_proj.weight": "model-00052-of-00101.safetensors", + "model.layers.48.mlp.experts.112.down_proj.weight": "model-00052-of-00101.safetensors", + "model.layers.48.mlp.experts.113.gate_proj.weight": "model-00052-of-00101.safetensors", + "model.layers.48.mlp.experts.113.up_proj.weight": "model-00052-of-00101.safetensors", + "model.layers.48.mlp.experts.113.down_proj.weight": "model-00052-of-00101.safetensors", + "model.layers.48.mlp.experts.114.gate_proj.weight": "model-00052-of-00101.safetensors", + "model.layers.48.mlp.experts.114.up_proj.weight": "model-00052-of-00101.safetensors", + "model.layers.48.mlp.experts.114.down_proj.weight": "model-00053-of-00101.safetensors", + "model.layers.48.mlp.experts.115.gate_proj.weight": "model-00053-of-00101.safetensors", + "model.layers.48.mlp.experts.115.up_proj.weight": "model-00053-of-00101.safetensors", + "model.layers.48.mlp.experts.115.down_proj.weight": "model-00053-of-00101.safetensors", + "model.layers.48.mlp.experts.116.gate_proj.weight": "model-00053-of-00101.safetensors", + "model.layers.48.mlp.experts.116.up_proj.weight": "model-00053-of-00101.safetensors", + "model.layers.48.mlp.experts.116.down_proj.weight": "model-00053-of-00101.safetensors", + "model.layers.48.mlp.experts.117.gate_proj.weight": "model-00053-of-00101.safetensors", + "model.layers.48.mlp.experts.117.up_proj.weight": "model-00053-of-00101.safetensors", + "model.layers.48.mlp.experts.117.down_proj.weight": "model-00053-of-00101.safetensors", + "model.layers.48.mlp.experts.118.gate_proj.weight": "model-00053-of-00101.safetensors", + "model.layers.48.mlp.experts.118.up_proj.weight": "model-00053-of-00101.safetensors", + "model.layers.48.mlp.experts.118.down_proj.weight": "model-00053-of-00101.safetensors", + "model.layers.48.mlp.experts.119.gate_proj.weight": "model-00053-of-00101.safetensors", + "model.layers.48.mlp.experts.119.up_proj.weight": "model-00053-of-00101.safetensors", + "model.layers.48.mlp.experts.119.down_proj.weight": "model-00053-of-00101.safetensors", + "model.layers.48.mlp.gate.weight": "model-00053-of-00101.safetensors", + "model.layers.48.mlp.gate.e_score_correction_bias": "model-00053-of-00101.safetensors", + "model.layers.48.mlp.shared_experts.gate_proj.weight": "model-00053-of-00101.safetensors", + "model.layers.48.mlp.shared_experts.up_proj.weight": "model-00053-of-00101.safetensors", + "model.layers.48.mlp.shared_experts.down_proj.weight": "model-00053-of-00101.safetensors", + "model.layers.48.input_layernorm.weight": "model-00053-of-00101.safetensors", + "model.layers.48.post_attention_layernorm.weight": "model-00053-of-00101.safetensors", + "model.layers.49.self_attn.q_proj.weight": "model-00053-of-00101.safetensors", + "model.layers.49.self_attn.q_proj.bias": "model-00053-of-00101.safetensors", + "model.layers.49.self_attn.k_proj.weight": "model-00053-of-00101.safetensors", + "model.layers.49.self_attn.k_proj.bias": "model-00053-of-00101.safetensors", + "model.layers.49.self_attn.v_proj.weight": "model-00053-of-00101.safetensors", + "model.layers.49.self_attn.v_proj.bias": "model-00053-of-00101.safetensors", + "model.layers.49.self_attn.o_proj.weight": "model-00053-of-00101.safetensors", + "model.layers.49.self_attn.q_norm.weight": "model-00053-of-00101.safetensors", + "model.layers.49.self_attn.k_norm.weight": "model-00053-of-00101.safetensors", + "model.layers.49.mlp.experts.0.gate_proj.weight": "model-00053-of-00101.safetensors", + "model.layers.49.mlp.experts.0.up_proj.weight": "model-00053-of-00101.safetensors", + "model.layers.49.mlp.experts.0.down_proj.weight": "model-00053-of-00101.safetensors", + "model.layers.49.mlp.experts.1.gate_proj.weight": "model-00053-of-00101.safetensors", + "model.layers.49.mlp.experts.1.up_proj.weight": "model-00053-of-00101.safetensors", + "model.layers.49.mlp.experts.1.down_proj.weight": "model-00053-of-00101.safetensors", + "model.layers.49.mlp.experts.2.gate_proj.weight": "model-00053-of-00101.safetensors", + "model.layers.49.mlp.experts.2.up_proj.weight": "model-00053-of-00101.safetensors", + "model.layers.49.mlp.experts.2.down_proj.weight": "model-00053-of-00101.safetensors", + "model.layers.49.mlp.experts.3.gate_proj.weight": "model-00053-of-00101.safetensors", + "model.layers.49.mlp.experts.3.up_proj.weight": "model-00053-of-00101.safetensors", + "model.layers.49.mlp.experts.3.down_proj.weight": "model-00053-of-00101.safetensors", + "model.layers.49.mlp.experts.4.gate_proj.weight": "model-00053-of-00101.safetensors", + "model.layers.49.mlp.experts.4.up_proj.weight": "model-00053-of-00101.safetensors", + "model.layers.49.mlp.experts.4.down_proj.weight": "model-00053-of-00101.safetensors", + "model.layers.49.mlp.experts.5.gate_proj.weight": "model-00053-of-00101.safetensors", + "model.layers.49.mlp.experts.5.up_proj.weight": "model-00053-of-00101.safetensors", + "model.layers.49.mlp.experts.5.down_proj.weight": "model-00053-of-00101.safetensors", + "model.layers.49.mlp.experts.6.gate_proj.weight": "model-00053-of-00101.safetensors", + "model.layers.49.mlp.experts.6.up_proj.weight": "model-00053-of-00101.safetensors", + "model.layers.49.mlp.experts.6.down_proj.weight": "model-00053-of-00101.safetensors", + "model.layers.49.mlp.experts.7.gate_proj.weight": "model-00053-of-00101.safetensors", + "model.layers.49.mlp.experts.7.up_proj.weight": "model-00053-of-00101.safetensors", + "model.layers.49.mlp.experts.7.down_proj.weight": "model-00053-of-00101.safetensors", + "model.layers.49.mlp.experts.8.gate_proj.weight": "model-00053-of-00101.safetensors", + "model.layers.49.mlp.experts.8.up_proj.weight": "model-00053-of-00101.safetensors", + "model.layers.49.mlp.experts.8.down_proj.weight": "model-00053-of-00101.safetensors", + "model.layers.49.mlp.experts.9.gate_proj.weight": "model-00053-of-00101.safetensors", + "model.layers.49.mlp.experts.9.up_proj.weight": "model-00053-of-00101.safetensors", + "model.layers.49.mlp.experts.9.down_proj.weight": "model-00053-of-00101.safetensors", + "model.layers.49.mlp.experts.10.gate_proj.weight": "model-00053-of-00101.safetensors", + "model.layers.49.mlp.experts.10.up_proj.weight": "model-00053-of-00101.safetensors", + "model.layers.49.mlp.experts.10.down_proj.weight": "model-00053-of-00101.safetensors", + "model.layers.49.mlp.experts.11.gate_proj.weight": "model-00053-of-00101.safetensors", + "model.layers.49.mlp.experts.11.up_proj.weight": "model-00053-of-00101.safetensors", + "model.layers.49.mlp.experts.11.down_proj.weight": "model-00053-of-00101.safetensors", + "model.layers.49.mlp.experts.12.gate_proj.weight": "model-00053-of-00101.safetensors", + "model.layers.49.mlp.experts.12.up_proj.weight": "model-00053-of-00101.safetensors", + "model.layers.49.mlp.experts.12.down_proj.weight": "model-00053-of-00101.safetensors", + "model.layers.49.mlp.experts.13.gate_proj.weight": "model-00053-of-00101.safetensors", + "model.layers.49.mlp.experts.13.up_proj.weight": "model-00053-of-00101.safetensors", + "model.layers.49.mlp.experts.13.down_proj.weight": "model-00053-of-00101.safetensors", + "model.layers.49.mlp.experts.14.gate_proj.weight": "model-00053-of-00101.safetensors", + "model.layers.49.mlp.experts.14.up_proj.weight": "model-00053-of-00101.safetensors", + "model.layers.49.mlp.experts.14.down_proj.weight": "model-00053-of-00101.safetensors", + "model.layers.49.mlp.experts.15.gate_proj.weight": "model-00053-of-00101.safetensors", + "model.layers.49.mlp.experts.15.up_proj.weight": "model-00053-of-00101.safetensors", + "model.layers.49.mlp.experts.15.down_proj.weight": "model-00053-of-00101.safetensors", + "model.layers.49.mlp.experts.16.gate_proj.weight": "model-00053-of-00101.safetensors", + "model.layers.49.mlp.experts.16.up_proj.weight": "model-00053-of-00101.safetensors", + "model.layers.49.mlp.experts.16.down_proj.weight": "model-00053-of-00101.safetensors", + "model.layers.49.mlp.experts.17.gate_proj.weight": "model-00053-of-00101.safetensors", + "model.layers.49.mlp.experts.17.up_proj.weight": "model-00053-of-00101.safetensors", + "model.layers.49.mlp.experts.17.down_proj.weight": "model-00053-of-00101.safetensors", + "model.layers.49.mlp.experts.18.gate_proj.weight": "model-00053-of-00101.safetensors", + "model.layers.49.mlp.experts.18.up_proj.weight": "model-00053-of-00101.safetensors", + "model.layers.49.mlp.experts.18.down_proj.weight": "model-00053-of-00101.safetensors", + "model.layers.49.mlp.experts.19.gate_proj.weight": "model-00053-of-00101.safetensors", + "model.layers.49.mlp.experts.19.up_proj.weight": "model-00053-of-00101.safetensors", + "model.layers.49.mlp.experts.19.down_proj.weight": "model-00053-of-00101.safetensors", + "model.layers.49.mlp.experts.20.gate_proj.weight": "model-00053-of-00101.safetensors", + "model.layers.49.mlp.experts.20.up_proj.weight": "model-00053-of-00101.safetensors", + "model.layers.49.mlp.experts.20.down_proj.weight": "model-00053-of-00101.safetensors", + "model.layers.49.mlp.experts.21.gate_proj.weight": "model-00053-of-00101.safetensors", + "model.layers.49.mlp.experts.21.up_proj.weight": "model-00053-of-00101.safetensors", + "model.layers.49.mlp.experts.21.down_proj.weight": "model-00053-of-00101.safetensors", + "model.layers.49.mlp.experts.22.gate_proj.weight": "model-00053-of-00101.safetensors", + "model.layers.49.mlp.experts.22.up_proj.weight": "model-00053-of-00101.safetensors", + "model.layers.49.mlp.experts.22.down_proj.weight": "model-00053-of-00101.safetensors", + "model.layers.49.mlp.experts.23.gate_proj.weight": "model-00053-of-00101.safetensors", + "model.layers.49.mlp.experts.23.up_proj.weight": "model-00053-of-00101.safetensors", + "model.layers.49.mlp.experts.23.down_proj.weight": "model-00053-of-00101.safetensors", + "model.layers.49.mlp.experts.24.gate_proj.weight": "model-00053-of-00101.safetensors", + "model.layers.49.mlp.experts.24.up_proj.weight": "model-00053-of-00101.safetensors", + "model.layers.49.mlp.experts.24.down_proj.weight": "model-00053-of-00101.safetensors", + "model.layers.49.mlp.experts.25.gate_proj.weight": "model-00053-of-00101.safetensors", + "model.layers.49.mlp.experts.25.up_proj.weight": "model-00053-of-00101.safetensors", + "model.layers.49.mlp.experts.25.down_proj.weight": "model-00053-of-00101.safetensors", + "model.layers.49.mlp.experts.26.gate_proj.weight": "model-00053-of-00101.safetensors", + "model.layers.49.mlp.experts.26.up_proj.weight": "model-00053-of-00101.safetensors", + "model.layers.49.mlp.experts.26.down_proj.weight": "model-00053-of-00101.safetensors", + "model.layers.49.mlp.experts.27.gate_proj.weight": "model-00053-of-00101.safetensors", + "model.layers.49.mlp.experts.27.up_proj.weight": "model-00053-of-00101.safetensors", + "model.layers.49.mlp.experts.27.down_proj.weight": "model-00053-of-00101.safetensors", + "model.layers.49.mlp.experts.28.gate_proj.weight": "model-00053-of-00101.safetensors", + "model.layers.49.mlp.experts.28.up_proj.weight": "model-00053-of-00101.safetensors", + "model.layers.49.mlp.experts.28.down_proj.weight": "model-00053-of-00101.safetensors", + "model.layers.49.mlp.experts.29.gate_proj.weight": "model-00053-of-00101.safetensors", + "model.layers.49.mlp.experts.29.up_proj.weight": "model-00053-of-00101.safetensors", + "model.layers.49.mlp.experts.29.down_proj.weight": "model-00053-of-00101.safetensors", + "model.layers.49.mlp.experts.30.gate_proj.weight": "model-00053-of-00101.safetensors", + "model.layers.49.mlp.experts.30.up_proj.weight": "model-00053-of-00101.safetensors", + "model.layers.49.mlp.experts.30.down_proj.weight": "model-00053-of-00101.safetensors", + "model.layers.49.mlp.experts.31.gate_proj.weight": "model-00053-of-00101.safetensors", + "model.layers.49.mlp.experts.31.up_proj.weight": "model-00053-of-00101.safetensors", + "model.layers.49.mlp.experts.31.down_proj.weight": "model-00053-of-00101.safetensors", + "model.layers.49.mlp.experts.32.gate_proj.weight": "model-00053-of-00101.safetensors", + "model.layers.49.mlp.experts.32.up_proj.weight": "model-00053-of-00101.safetensors", + "model.layers.49.mlp.experts.32.down_proj.weight": "model-00053-of-00101.safetensors", + "model.layers.49.mlp.experts.33.gate_proj.weight": "model-00053-of-00101.safetensors", + "model.layers.49.mlp.experts.33.up_proj.weight": "model-00053-of-00101.safetensors", + "model.layers.49.mlp.experts.33.down_proj.weight": "model-00053-of-00101.safetensors", + "model.layers.49.mlp.experts.34.gate_proj.weight": "model-00053-of-00101.safetensors", + "model.layers.49.mlp.experts.34.up_proj.weight": "model-00053-of-00101.safetensors", + "model.layers.49.mlp.experts.34.down_proj.weight": "model-00053-of-00101.safetensors", + "model.layers.49.mlp.experts.35.gate_proj.weight": "model-00053-of-00101.safetensors", + "model.layers.49.mlp.experts.35.up_proj.weight": "model-00053-of-00101.safetensors", + "model.layers.49.mlp.experts.35.down_proj.weight": "model-00053-of-00101.safetensors", + "model.layers.49.mlp.experts.36.gate_proj.weight": "model-00053-of-00101.safetensors", + "model.layers.49.mlp.experts.36.up_proj.weight": "model-00053-of-00101.safetensors", + "model.layers.49.mlp.experts.36.down_proj.weight": "model-00053-of-00101.safetensors", + "model.layers.49.mlp.experts.37.gate_proj.weight": "model-00053-of-00101.safetensors", + "model.layers.49.mlp.experts.37.up_proj.weight": "model-00053-of-00101.safetensors", + "model.layers.49.mlp.experts.37.down_proj.weight": "model-00053-of-00101.safetensors", + "model.layers.49.mlp.experts.38.gate_proj.weight": "model-00053-of-00101.safetensors", + "model.layers.49.mlp.experts.38.up_proj.weight": "model-00053-of-00101.safetensors", + "model.layers.49.mlp.experts.38.down_proj.weight": "model-00053-of-00101.safetensors", + "model.layers.49.mlp.experts.39.gate_proj.weight": "model-00053-of-00101.safetensors", + "model.layers.49.mlp.experts.39.up_proj.weight": "model-00053-of-00101.safetensors", + "model.layers.49.mlp.experts.39.down_proj.weight": "model-00053-of-00101.safetensors", + "model.layers.49.mlp.experts.40.gate_proj.weight": "model-00053-of-00101.safetensors", + "model.layers.49.mlp.experts.40.up_proj.weight": "model-00053-of-00101.safetensors", + "model.layers.49.mlp.experts.40.down_proj.weight": "model-00053-of-00101.safetensors", + "model.layers.49.mlp.experts.41.gate_proj.weight": "model-00053-of-00101.safetensors", + "model.layers.49.mlp.experts.41.up_proj.weight": "model-00053-of-00101.safetensors", + "model.layers.49.mlp.experts.41.down_proj.weight": "model-00053-of-00101.safetensors", + "model.layers.49.mlp.experts.42.gate_proj.weight": "model-00053-of-00101.safetensors", + "model.layers.49.mlp.experts.42.up_proj.weight": "model-00053-of-00101.safetensors", + "model.layers.49.mlp.experts.42.down_proj.weight": "model-00053-of-00101.safetensors", + "model.layers.49.mlp.experts.43.gate_proj.weight": "model-00053-of-00101.safetensors", + "model.layers.49.mlp.experts.43.up_proj.weight": "model-00053-of-00101.safetensors", + "model.layers.49.mlp.experts.43.down_proj.weight": "model-00053-of-00101.safetensors", + "model.layers.49.mlp.experts.44.gate_proj.weight": "model-00053-of-00101.safetensors", + "model.layers.49.mlp.experts.44.up_proj.weight": "model-00053-of-00101.safetensors", + "model.layers.49.mlp.experts.44.down_proj.weight": "model-00053-of-00101.safetensors", + "model.layers.49.mlp.experts.45.gate_proj.weight": "model-00053-of-00101.safetensors", + "model.layers.49.mlp.experts.45.up_proj.weight": "model-00053-of-00101.safetensors", + "model.layers.49.mlp.experts.45.down_proj.weight": "model-00053-of-00101.safetensors", + "model.layers.49.mlp.experts.46.gate_proj.weight": "model-00053-of-00101.safetensors", + "model.layers.49.mlp.experts.46.up_proj.weight": "model-00053-of-00101.safetensors", + "model.layers.49.mlp.experts.46.down_proj.weight": "model-00053-of-00101.safetensors", + "model.layers.49.mlp.experts.47.gate_proj.weight": "model-00053-of-00101.safetensors", + "model.layers.49.mlp.experts.47.up_proj.weight": "model-00053-of-00101.safetensors", + "model.layers.49.mlp.experts.47.down_proj.weight": "model-00053-of-00101.safetensors", + "model.layers.49.mlp.experts.48.gate_proj.weight": "model-00053-of-00101.safetensors", + "model.layers.49.mlp.experts.48.up_proj.weight": "model-00053-of-00101.safetensors", + "model.layers.49.mlp.experts.48.down_proj.weight": "model-00053-of-00101.safetensors", + "model.layers.49.mlp.experts.49.gate_proj.weight": "model-00053-of-00101.safetensors", + "model.layers.49.mlp.experts.49.up_proj.weight": "model-00053-of-00101.safetensors", + "model.layers.49.mlp.experts.49.down_proj.weight": "model-00053-of-00101.safetensors", + "model.layers.49.mlp.experts.50.gate_proj.weight": "model-00053-of-00101.safetensors", + "model.layers.49.mlp.experts.50.up_proj.weight": "model-00053-of-00101.safetensors", + "model.layers.49.mlp.experts.50.down_proj.weight": "model-00053-of-00101.safetensors", + "model.layers.49.mlp.experts.51.gate_proj.weight": "model-00053-of-00101.safetensors", + "model.layers.49.mlp.experts.51.up_proj.weight": "model-00053-of-00101.safetensors", + "model.layers.49.mlp.experts.51.down_proj.weight": "model-00053-of-00101.safetensors", + "model.layers.49.mlp.experts.52.gate_proj.weight": "model-00053-of-00101.safetensors", + "model.layers.49.mlp.experts.52.up_proj.weight": "model-00053-of-00101.safetensors", + "model.layers.49.mlp.experts.52.down_proj.weight": "model-00053-of-00101.safetensors", + "model.layers.49.mlp.experts.53.gate_proj.weight": "model-00053-of-00101.safetensors", + "model.layers.49.mlp.experts.53.up_proj.weight": "model-00053-of-00101.safetensors", + "model.layers.49.mlp.experts.53.down_proj.weight": "model-00053-of-00101.safetensors", + "model.layers.49.mlp.experts.54.gate_proj.weight": "model-00053-of-00101.safetensors", + "model.layers.49.mlp.experts.54.up_proj.weight": "model-00053-of-00101.safetensors", + "model.layers.49.mlp.experts.54.down_proj.weight": "model-00053-of-00101.safetensors", + "model.layers.49.mlp.experts.55.gate_proj.weight": "model-00053-of-00101.safetensors", + "model.layers.49.mlp.experts.55.up_proj.weight": "model-00053-of-00101.safetensors", + "model.layers.49.mlp.experts.55.down_proj.weight": "model-00053-of-00101.safetensors", + "model.layers.49.mlp.experts.56.gate_proj.weight": "model-00053-of-00101.safetensors", + "model.layers.49.mlp.experts.56.up_proj.weight": "model-00053-of-00101.safetensors", + "model.layers.49.mlp.experts.56.down_proj.weight": "model-00053-of-00101.safetensors", + "model.layers.49.mlp.experts.57.gate_proj.weight": "model-00053-of-00101.safetensors", + "model.layers.49.mlp.experts.57.up_proj.weight": "model-00053-of-00101.safetensors", + "model.layers.49.mlp.experts.57.down_proj.weight": "model-00053-of-00101.safetensors", + "model.layers.49.mlp.experts.58.gate_proj.weight": "model-00053-of-00101.safetensors", + "model.layers.49.mlp.experts.58.up_proj.weight": "model-00053-of-00101.safetensors", + "model.layers.49.mlp.experts.58.down_proj.weight": "model-00053-of-00101.safetensors", + "model.layers.49.mlp.experts.59.gate_proj.weight": "model-00053-of-00101.safetensors", + "model.layers.49.mlp.experts.59.up_proj.weight": "model-00053-of-00101.safetensors", + "model.layers.49.mlp.experts.59.down_proj.weight": "model-00053-of-00101.safetensors", + "model.layers.49.mlp.experts.60.gate_proj.weight": "model-00053-of-00101.safetensors", + "model.layers.49.mlp.experts.60.up_proj.weight": "model-00053-of-00101.safetensors", + "model.layers.49.mlp.experts.60.down_proj.weight": "model-00053-of-00101.safetensors", + "model.layers.49.mlp.experts.61.gate_proj.weight": "model-00053-of-00101.safetensors", + "model.layers.49.mlp.experts.61.up_proj.weight": "model-00053-of-00101.safetensors", + "model.layers.49.mlp.experts.61.down_proj.weight": "model-00053-of-00101.safetensors", + "model.layers.49.mlp.experts.62.gate_proj.weight": "model-00053-of-00101.safetensors", + "model.layers.49.mlp.experts.62.up_proj.weight": "model-00053-of-00101.safetensors", + "model.layers.49.mlp.experts.62.down_proj.weight": "model-00053-of-00101.safetensors", + "model.layers.49.mlp.experts.63.gate_proj.weight": "model-00053-of-00101.safetensors", + "model.layers.49.mlp.experts.63.up_proj.weight": "model-00053-of-00101.safetensors", + "model.layers.49.mlp.experts.63.down_proj.weight": "model-00053-of-00101.safetensors", + "model.layers.49.mlp.experts.64.gate_proj.weight": "model-00053-of-00101.safetensors", + "model.layers.49.mlp.experts.64.up_proj.weight": "model-00053-of-00101.safetensors", + "model.layers.49.mlp.experts.64.down_proj.weight": "model-00053-of-00101.safetensors", + "model.layers.49.mlp.experts.65.gate_proj.weight": "model-00053-of-00101.safetensors", + "model.layers.49.mlp.experts.65.up_proj.weight": "model-00053-of-00101.safetensors", + "model.layers.49.mlp.experts.65.down_proj.weight": "model-00053-of-00101.safetensors", + "model.layers.49.mlp.experts.66.gate_proj.weight": "model-00053-of-00101.safetensors", + "model.layers.49.mlp.experts.66.up_proj.weight": "model-00053-of-00101.safetensors", + "model.layers.49.mlp.experts.66.down_proj.weight": "model-00053-of-00101.safetensors", + "model.layers.49.mlp.experts.67.gate_proj.weight": "model-00053-of-00101.safetensors", + "model.layers.49.mlp.experts.67.up_proj.weight": "model-00053-of-00101.safetensors", + "model.layers.49.mlp.experts.67.down_proj.weight": "model-00053-of-00101.safetensors", + "model.layers.49.mlp.experts.68.gate_proj.weight": "model-00053-of-00101.safetensors", + "model.layers.49.mlp.experts.68.up_proj.weight": "model-00053-of-00101.safetensors", + "model.layers.49.mlp.experts.68.down_proj.weight": "model-00053-of-00101.safetensors", + "model.layers.49.mlp.experts.69.gate_proj.weight": "model-00053-of-00101.safetensors", + "model.layers.49.mlp.experts.69.up_proj.weight": "model-00053-of-00101.safetensors", + "model.layers.49.mlp.experts.69.down_proj.weight": "model-00053-of-00101.safetensors", + "model.layers.49.mlp.experts.70.gate_proj.weight": "model-00053-of-00101.safetensors", + "model.layers.49.mlp.experts.70.up_proj.weight": "model-00053-of-00101.safetensors", + "model.layers.49.mlp.experts.70.down_proj.weight": "model-00053-of-00101.safetensors", + "model.layers.49.mlp.experts.71.gate_proj.weight": "model-00053-of-00101.safetensors", + "model.layers.49.mlp.experts.71.up_proj.weight": "model-00053-of-00101.safetensors", + "model.layers.49.mlp.experts.71.down_proj.weight": "model-00053-of-00101.safetensors", + "model.layers.49.mlp.experts.72.gate_proj.weight": "model-00053-of-00101.safetensors", + "model.layers.49.mlp.experts.72.up_proj.weight": "model-00053-of-00101.safetensors", + "model.layers.49.mlp.experts.72.down_proj.weight": "model-00053-of-00101.safetensors", + "model.layers.49.mlp.experts.73.gate_proj.weight": "model-00053-of-00101.safetensors", + "model.layers.49.mlp.experts.73.up_proj.weight": "model-00053-of-00101.safetensors", + "model.layers.49.mlp.experts.73.down_proj.weight": "model-00053-of-00101.safetensors", + "model.layers.49.mlp.experts.74.gate_proj.weight": "model-00053-of-00101.safetensors", + "model.layers.49.mlp.experts.74.up_proj.weight": "model-00053-of-00101.safetensors", + "model.layers.49.mlp.experts.74.down_proj.weight": "model-00053-of-00101.safetensors", + "model.layers.49.mlp.experts.75.gate_proj.weight": "model-00053-of-00101.safetensors", + "model.layers.49.mlp.experts.75.up_proj.weight": "model-00053-of-00101.safetensors", + "model.layers.49.mlp.experts.75.down_proj.weight": "model-00053-of-00101.safetensors", + "model.layers.49.mlp.experts.76.gate_proj.weight": "model-00053-of-00101.safetensors", + "model.layers.49.mlp.experts.76.up_proj.weight": "model-00053-of-00101.safetensors", + "model.layers.49.mlp.experts.76.down_proj.weight": "model-00053-of-00101.safetensors", + "model.layers.49.mlp.experts.77.gate_proj.weight": "model-00053-of-00101.safetensors", + "model.layers.49.mlp.experts.77.up_proj.weight": "model-00053-of-00101.safetensors", + "model.layers.49.mlp.experts.77.down_proj.weight": "model-00053-of-00101.safetensors", + "model.layers.49.mlp.experts.78.gate_proj.weight": "model-00053-of-00101.safetensors", + "model.layers.49.mlp.experts.78.up_proj.weight": "model-00053-of-00101.safetensors", + "model.layers.49.mlp.experts.78.down_proj.weight": "model-00053-of-00101.safetensors", + "model.layers.49.mlp.experts.79.gate_proj.weight": "model-00053-of-00101.safetensors", + "model.layers.49.mlp.experts.79.up_proj.weight": "model-00053-of-00101.safetensors", + "model.layers.49.mlp.experts.79.down_proj.weight": "model-00053-of-00101.safetensors", + "model.layers.49.mlp.experts.80.gate_proj.weight": "model-00053-of-00101.safetensors", + "model.layers.49.mlp.experts.80.up_proj.weight": "model-00053-of-00101.safetensors", + "model.layers.49.mlp.experts.80.down_proj.weight": "model-00053-of-00101.safetensors", + "model.layers.49.mlp.experts.81.gate_proj.weight": "model-00053-of-00101.safetensors", + "model.layers.49.mlp.experts.81.up_proj.weight": "model-00053-of-00101.safetensors", + "model.layers.49.mlp.experts.81.down_proj.weight": "model-00053-of-00101.safetensors", + "model.layers.49.mlp.experts.82.gate_proj.weight": "model-00053-of-00101.safetensors", + "model.layers.49.mlp.experts.82.up_proj.weight": "model-00053-of-00101.safetensors", + "model.layers.49.mlp.experts.82.down_proj.weight": "model-00053-of-00101.safetensors", + "model.layers.49.mlp.experts.83.gate_proj.weight": "model-00053-of-00101.safetensors", + "model.layers.49.mlp.experts.83.up_proj.weight": "model-00053-of-00101.safetensors", + "model.layers.49.mlp.experts.83.down_proj.weight": "model-00053-of-00101.safetensors", + "model.layers.49.mlp.experts.84.gate_proj.weight": "model-00053-of-00101.safetensors", + "model.layers.49.mlp.experts.84.up_proj.weight": "model-00053-of-00101.safetensors", + "model.layers.49.mlp.experts.84.down_proj.weight": "model-00053-of-00101.safetensors", + "model.layers.49.mlp.experts.85.gate_proj.weight": "model-00053-of-00101.safetensors", + "model.layers.49.mlp.experts.85.up_proj.weight": "model-00053-of-00101.safetensors", + "model.layers.49.mlp.experts.85.down_proj.weight": "model-00053-of-00101.safetensors", + "model.layers.49.mlp.experts.86.gate_proj.weight": "model-00053-of-00101.safetensors", + "model.layers.49.mlp.experts.86.up_proj.weight": "model-00053-of-00101.safetensors", + "model.layers.49.mlp.experts.86.down_proj.weight": "model-00053-of-00101.safetensors", + "model.layers.49.mlp.experts.87.gate_proj.weight": "model-00053-of-00101.safetensors", + "model.layers.49.mlp.experts.87.up_proj.weight": "model-00053-of-00101.safetensors", + "model.layers.49.mlp.experts.87.down_proj.weight": "model-00053-of-00101.safetensors", + "model.layers.49.mlp.experts.88.gate_proj.weight": "model-00053-of-00101.safetensors", + "model.layers.49.mlp.experts.88.up_proj.weight": "model-00053-of-00101.safetensors", + "model.layers.49.mlp.experts.88.down_proj.weight": "model-00053-of-00101.safetensors", + "model.layers.49.mlp.experts.89.gate_proj.weight": "model-00053-of-00101.safetensors", + "model.layers.49.mlp.experts.89.up_proj.weight": "model-00053-of-00101.safetensors", + "model.layers.49.mlp.experts.89.down_proj.weight": "model-00053-of-00101.safetensors", + "model.layers.49.mlp.experts.90.gate_proj.weight": "model-00053-of-00101.safetensors", + "model.layers.49.mlp.experts.90.up_proj.weight": "model-00053-of-00101.safetensors", + "model.layers.49.mlp.experts.90.down_proj.weight": "model-00053-of-00101.safetensors", + "model.layers.49.mlp.experts.91.gate_proj.weight": "model-00053-of-00101.safetensors", + "model.layers.49.mlp.experts.91.up_proj.weight": "model-00053-of-00101.safetensors", + "model.layers.49.mlp.experts.91.down_proj.weight": "model-00053-of-00101.safetensors", + "model.layers.49.mlp.experts.92.gate_proj.weight": "model-00053-of-00101.safetensors", + "model.layers.49.mlp.experts.92.up_proj.weight": "model-00053-of-00101.safetensors", + "model.layers.49.mlp.experts.92.down_proj.weight": "model-00053-of-00101.safetensors", + "model.layers.49.mlp.experts.93.gate_proj.weight": "model-00053-of-00101.safetensors", + "model.layers.49.mlp.experts.93.up_proj.weight": "model-00053-of-00101.safetensors", + "model.layers.49.mlp.experts.93.down_proj.weight": "model-00053-of-00101.safetensors", + "model.layers.49.mlp.experts.94.gate_proj.weight": "model-00053-of-00101.safetensors", + "model.layers.49.mlp.experts.94.up_proj.weight": "model-00053-of-00101.safetensors", + "model.layers.49.mlp.experts.94.down_proj.weight": "model-00053-of-00101.safetensors", + "model.layers.49.mlp.experts.95.gate_proj.weight": "model-00053-of-00101.safetensors", + "model.layers.49.mlp.experts.95.up_proj.weight": "model-00053-of-00101.safetensors", + "model.layers.49.mlp.experts.95.down_proj.weight": "model-00053-of-00101.safetensors", + "model.layers.49.mlp.experts.96.gate_proj.weight": "model-00053-of-00101.safetensors", + "model.layers.49.mlp.experts.96.up_proj.weight": "model-00053-of-00101.safetensors", + "model.layers.49.mlp.experts.96.down_proj.weight": "model-00053-of-00101.safetensors", + "model.layers.49.mlp.experts.97.gate_proj.weight": "model-00053-of-00101.safetensors", + "model.layers.49.mlp.experts.97.up_proj.weight": "model-00053-of-00101.safetensors", + "model.layers.49.mlp.experts.97.down_proj.weight": "model-00053-of-00101.safetensors", + "model.layers.49.mlp.experts.98.gate_proj.weight": "model-00053-of-00101.safetensors", + "model.layers.49.mlp.experts.98.up_proj.weight": "model-00053-of-00101.safetensors", + "model.layers.49.mlp.experts.98.down_proj.weight": "model-00053-of-00101.safetensors", + "model.layers.49.mlp.experts.99.gate_proj.weight": "model-00053-of-00101.safetensors", + "model.layers.49.mlp.experts.99.up_proj.weight": "model-00053-of-00101.safetensors", + "model.layers.49.mlp.experts.99.down_proj.weight": "model-00053-of-00101.safetensors", + "model.layers.49.mlp.experts.100.gate_proj.weight": "model-00053-of-00101.safetensors", + "model.layers.49.mlp.experts.100.up_proj.weight": "model-00053-of-00101.safetensors", + "model.layers.49.mlp.experts.100.down_proj.weight": "model-00053-of-00101.safetensors", + "model.layers.49.mlp.experts.101.gate_proj.weight": "model-00053-of-00101.safetensors", + "model.layers.49.mlp.experts.101.up_proj.weight": "model-00054-of-00101.safetensors", + "model.layers.49.mlp.experts.101.down_proj.weight": "model-00054-of-00101.safetensors", + "model.layers.49.mlp.experts.102.gate_proj.weight": "model-00054-of-00101.safetensors", + "model.layers.49.mlp.experts.102.up_proj.weight": "model-00054-of-00101.safetensors", + "model.layers.49.mlp.experts.102.down_proj.weight": "model-00054-of-00101.safetensors", + "model.layers.49.mlp.experts.103.gate_proj.weight": "model-00054-of-00101.safetensors", + "model.layers.49.mlp.experts.103.up_proj.weight": "model-00054-of-00101.safetensors", + "model.layers.49.mlp.experts.103.down_proj.weight": "model-00054-of-00101.safetensors", + "model.layers.49.mlp.experts.104.gate_proj.weight": "model-00054-of-00101.safetensors", + "model.layers.49.mlp.experts.104.up_proj.weight": "model-00054-of-00101.safetensors", + "model.layers.49.mlp.experts.104.down_proj.weight": "model-00054-of-00101.safetensors", + "model.layers.49.mlp.experts.105.gate_proj.weight": "model-00054-of-00101.safetensors", + "model.layers.49.mlp.experts.105.up_proj.weight": "model-00054-of-00101.safetensors", + "model.layers.49.mlp.experts.105.down_proj.weight": "model-00054-of-00101.safetensors", + "model.layers.49.mlp.experts.106.gate_proj.weight": "model-00054-of-00101.safetensors", + "model.layers.49.mlp.experts.106.up_proj.weight": "model-00054-of-00101.safetensors", + "model.layers.49.mlp.experts.106.down_proj.weight": "model-00054-of-00101.safetensors", + "model.layers.49.mlp.experts.107.gate_proj.weight": "model-00054-of-00101.safetensors", + "model.layers.49.mlp.experts.107.up_proj.weight": "model-00054-of-00101.safetensors", + "model.layers.49.mlp.experts.107.down_proj.weight": "model-00054-of-00101.safetensors", + "model.layers.49.mlp.experts.108.gate_proj.weight": "model-00054-of-00101.safetensors", + "model.layers.49.mlp.experts.108.up_proj.weight": "model-00054-of-00101.safetensors", + "model.layers.49.mlp.experts.108.down_proj.weight": "model-00054-of-00101.safetensors", + "model.layers.49.mlp.experts.109.gate_proj.weight": "model-00054-of-00101.safetensors", + "model.layers.49.mlp.experts.109.up_proj.weight": "model-00054-of-00101.safetensors", + "model.layers.49.mlp.experts.109.down_proj.weight": "model-00054-of-00101.safetensors", + "model.layers.49.mlp.experts.110.gate_proj.weight": "model-00054-of-00101.safetensors", + "model.layers.49.mlp.experts.110.up_proj.weight": "model-00054-of-00101.safetensors", + "model.layers.49.mlp.experts.110.down_proj.weight": "model-00054-of-00101.safetensors", + "model.layers.49.mlp.experts.111.gate_proj.weight": "model-00054-of-00101.safetensors", + "model.layers.49.mlp.experts.111.up_proj.weight": "model-00054-of-00101.safetensors", + "model.layers.49.mlp.experts.111.down_proj.weight": "model-00054-of-00101.safetensors", + "model.layers.49.mlp.experts.112.gate_proj.weight": "model-00054-of-00101.safetensors", + "model.layers.49.mlp.experts.112.up_proj.weight": "model-00054-of-00101.safetensors", + "model.layers.49.mlp.experts.112.down_proj.weight": "model-00054-of-00101.safetensors", + "model.layers.49.mlp.experts.113.gate_proj.weight": "model-00054-of-00101.safetensors", + "model.layers.49.mlp.experts.113.up_proj.weight": "model-00054-of-00101.safetensors", + "model.layers.49.mlp.experts.113.down_proj.weight": "model-00054-of-00101.safetensors", + "model.layers.49.mlp.experts.114.gate_proj.weight": "model-00054-of-00101.safetensors", + "model.layers.49.mlp.experts.114.up_proj.weight": "model-00054-of-00101.safetensors", + "model.layers.49.mlp.experts.114.down_proj.weight": "model-00054-of-00101.safetensors", + "model.layers.49.mlp.experts.115.gate_proj.weight": "model-00054-of-00101.safetensors", + "model.layers.49.mlp.experts.115.up_proj.weight": "model-00054-of-00101.safetensors", + "model.layers.49.mlp.experts.115.down_proj.weight": "model-00054-of-00101.safetensors", + "model.layers.49.mlp.experts.116.gate_proj.weight": "model-00054-of-00101.safetensors", + "model.layers.49.mlp.experts.116.up_proj.weight": "model-00054-of-00101.safetensors", + "model.layers.49.mlp.experts.116.down_proj.weight": "model-00054-of-00101.safetensors", + "model.layers.49.mlp.experts.117.gate_proj.weight": "model-00054-of-00101.safetensors", + "model.layers.49.mlp.experts.117.up_proj.weight": "model-00054-of-00101.safetensors", + "model.layers.49.mlp.experts.117.down_proj.weight": "model-00054-of-00101.safetensors", + "model.layers.49.mlp.experts.118.gate_proj.weight": "model-00054-of-00101.safetensors", + "model.layers.49.mlp.experts.118.up_proj.weight": "model-00054-of-00101.safetensors", + "model.layers.49.mlp.experts.118.down_proj.weight": "model-00054-of-00101.safetensors", + "model.layers.49.mlp.experts.119.gate_proj.weight": "model-00054-of-00101.safetensors", + "model.layers.49.mlp.experts.119.up_proj.weight": "model-00054-of-00101.safetensors", + "model.layers.49.mlp.experts.119.down_proj.weight": "model-00054-of-00101.safetensors", + "model.layers.49.mlp.gate.weight": "model-00054-of-00101.safetensors", + "model.layers.49.mlp.gate.e_score_correction_bias": "model-00054-of-00101.safetensors", + "model.layers.49.mlp.shared_experts.gate_proj.weight": "model-00054-of-00101.safetensors", + "model.layers.49.mlp.shared_experts.up_proj.weight": "model-00054-of-00101.safetensors", + "model.layers.49.mlp.shared_experts.down_proj.weight": "model-00054-of-00101.safetensors", + "model.layers.49.input_layernorm.weight": "model-00054-of-00101.safetensors", + "model.layers.49.post_attention_layernorm.weight": "model-00054-of-00101.safetensors", + "model.layers.50.self_attn.q_proj.weight": "model-00054-of-00101.safetensors", + "model.layers.50.self_attn.q_proj.bias": "model-00054-of-00101.safetensors", + "model.layers.50.self_attn.k_proj.weight": "model-00054-of-00101.safetensors", + "model.layers.50.self_attn.k_proj.bias": "model-00054-of-00101.safetensors", + "model.layers.50.self_attn.v_proj.weight": "model-00054-of-00101.safetensors", + "model.layers.50.self_attn.v_proj.bias": "model-00054-of-00101.safetensors", + "model.layers.50.self_attn.o_proj.weight": "model-00054-of-00101.safetensors", + "model.layers.50.self_attn.q_norm.weight": "model-00054-of-00101.safetensors", + "model.layers.50.self_attn.k_norm.weight": "model-00054-of-00101.safetensors", + "model.layers.50.mlp.experts.0.gate_proj.weight": "model-00054-of-00101.safetensors", + "model.layers.50.mlp.experts.0.up_proj.weight": "model-00054-of-00101.safetensors", + "model.layers.50.mlp.experts.0.down_proj.weight": "model-00054-of-00101.safetensors", + "model.layers.50.mlp.experts.1.gate_proj.weight": "model-00054-of-00101.safetensors", + "model.layers.50.mlp.experts.1.up_proj.weight": "model-00054-of-00101.safetensors", + "model.layers.50.mlp.experts.1.down_proj.weight": "model-00054-of-00101.safetensors", + "model.layers.50.mlp.experts.2.gate_proj.weight": "model-00054-of-00101.safetensors", + "model.layers.50.mlp.experts.2.up_proj.weight": "model-00054-of-00101.safetensors", + "model.layers.50.mlp.experts.2.down_proj.weight": "model-00054-of-00101.safetensors", + "model.layers.50.mlp.experts.3.gate_proj.weight": "model-00054-of-00101.safetensors", + "model.layers.50.mlp.experts.3.up_proj.weight": "model-00054-of-00101.safetensors", + "model.layers.50.mlp.experts.3.down_proj.weight": "model-00054-of-00101.safetensors", + "model.layers.50.mlp.experts.4.gate_proj.weight": "model-00054-of-00101.safetensors", + "model.layers.50.mlp.experts.4.up_proj.weight": "model-00054-of-00101.safetensors", + "model.layers.50.mlp.experts.4.down_proj.weight": "model-00054-of-00101.safetensors", + "model.layers.50.mlp.experts.5.gate_proj.weight": "model-00054-of-00101.safetensors", + "model.layers.50.mlp.experts.5.up_proj.weight": "model-00054-of-00101.safetensors", + "model.layers.50.mlp.experts.5.down_proj.weight": "model-00054-of-00101.safetensors", + "model.layers.50.mlp.experts.6.gate_proj.weight": "model-00054-of-00101.safetensors", + "model.layers.50.mlp.experts.6.up_proj.weight": "model-00054-of-00101.safetensors", + "model.layers.50.mlp.experts.6.down_proj.weight": "model-00054-of-00101.safetensors", + "model.layers.50.mlp.experts.7.gate_proj.weight": "model-00054-of-00101.safetensors", + "model.layers.50.mlp.experts.7.up_proj.weight": "model-00054-of-00101.safetensors", + "model.layers.50.mlp.experts.7.down_proj.weight": "model-00054-of-00101.safetensors", + "model.layers.50.mlp.experts.8.gate_proj.weight": "model-00054-of-00101.safetensors", + "model.layers.50.mlp.experts.8.up_proj.weight": "model-00054-of-00101.safetensors", + "model.layers.50.mlp.experts.8.down_proj.weight": "model-00054-of-00101.safetensors", + "model.layers.50.mlp.experts.9.gate_proj.weight": "model-00054-of-00101.safetensors", + "model.layers.50.mlp.experts.9.up_proj.weight": "model-00054-of-00101.safetensors", + "model.layers.50.mlp.experts.9.down_proj.weight": "model-00054-of-00101.safetensors", + "model.layers.50.mlp.experts.10.gate_proj.weight": "model-00054-of-00101.safetensors", + "model.layers.50.mlp.experts.10.up_proj.weight": "model-00054-of-00101.safetensors", + "model.layers.50.mlp.experts.10.down_proj.weight": "model-00054-of-00101.safetensors", + "model.layers.50.mlp.experts.11.gate_proj.weight": "model-00054-of-00101.safetensors", + "model.layers.50.mlp.experts.11.up_proj.weight": "model-00054-of-00101.safetensors", + "model.layers.50.mlp.experts.11.down_proj.weight": "model-00054-of-00101.safetensors", + "model.layers.50.mlp.experts.12.gate_proj.weight": "model-00054-of-00101.safetensors", + "model.layers.50.mlp.experts.12.up_proj.weight": "model-00054-of-00101.safetensors", + "model.layers.50.mlp.experts.12.down_proj.weight": "model-00054-of-00101.safetensors", + "model.layers.50.mlp.experts.13.gate_proj.weight": "model-00054-of-00101.safetensors", + "model.layers.50.mlp.experts.13.up_proj.weight": "model-00054-of-00101.safetensors", + "model.layers.50.mlp.experts.13.down_proj.weight": "model-00054-of-00101.safetensors", + "model.layers.50.mlp.experts.14.gate_proj.weight": "model-00054-of-00101.safetensors", + "model.layers.50.mlp.experts.14.up_proj.weight": "model-00054-of-00101.safetensors", + "model.layers.50.mlp.experts.14.down_proj.weight": "model-00054-of-00101.safetensors", + "model.layers.50.mlp.experts.15.gate_proj.weight": "model-00054-of-00101.safetensors", + "model.layers.50.mlp.experts.15.up_proj.weight": "model-00054-of-00101.safetensors", + "model.layers.50.mlp.experts.15.down_proj.weight": "model-00054-of-00101.safetensors", + "model.layers.50.mlp.experts.16.gate_proj.weight": "model-00054-of-00101.safetensors", + "model.layers.50.mlp.experts.16.up_proj.weight": "model-00054-of-00101.safetensors", + "model.layers.50.mlp.experts.16.down_proj.weight": "model-00054-of-00101.safetensors", + "model.layers.50.mlp.experts.17.gate_proj.weight": "model-00054-of-00101.safetensors", + "model.layers.50.mlp.experts.17.up_proj.weight": "model-00054-of-00101.safetensors", + "model.layers.50.mlp.experts.17.down_proj.weight": "model-00054-of-00101.safetensors", + "model.layers.50.mlp.experts.18.gate_proj.weight": "model-00054-of-00101.safetensors", + "model.layers.50.mlp.experts.18.up_proj.weight": "model-00054-of-00101.safetensors", + "model.layers.50.mlp.experts.18.down_proj.weight": "model-00054-of-00101.safetensors", + "model.layers.50.mlp.experts.19.gate_proj.weight": "model-00054-of-00101.safetensors", + "model.layers.50.mlp.experts.19.up_proj.weight": "model-00054-of-00101.safetensors", + "model.layers.50.mlp.experts.19.down_proj.weight": "model-00054-of-00101.safetensors", + "model.layers.50.mlp.experts.20.gate_proj.weight": "model-00054-of-00101.safetensors", + "model.layers.50.mlp.experts.20.up_proj.weight": "model-00054-of-00101.safetensors", + "model.layers.50.mlp.experts.20.down_proj.weight": "model-00054-of-00101.safetensors", + "model.layers.50.mlp.experts.21.gate_proj.weight": "model-00054-of-00101.safetensors", + "model.layers.50.mlp.experts.21.up_proj.weight": "model-00054-of-00101.safetensors", + "model.layers.50.mlp.experts.21.down_proj.weight": "model-00054-of-00101.safetensors", + "model.layers.50.mlp.experts.22.gate_proj.weight": "model-00054-of-00101.safetensors", + "model.layers.50.mlp.experts.22.up_proj.weight": "model-00054-of-00101.safetensors", + "model.layers.50.mlp.experts.22.down_proj.weight": "model-00054-of-00101.safetensors", + "model.layers.50.mlp.experts.23.gate_proj.weight": "model-00054-of-00101.safetensors", + "model.layers.50.mlp.experts.23.up_proj.weight": "model-00054-of-00101.safetensors", + "model.layers.50.mlp.experts.23.down_proj.weight": "model-00054-of-00101.safetensors", + "model.layers.50.mlp.experts.24.gate_proj.weight": "model-00054-of-00101.safetensors", + "model.layers.50.mlp.experts.24.up_proj.weight": "model-00054-of-00101.safetensors", + "model.layers.50.mlp.experts.24.down_proj.weight": "model-00054-of-00101.safetensors", + "model.layers.50.mlp.experts.25.gate_proj.weight": "model-00054-of-00101.safetensors", + "model.layers.50.mlp.experts.25.up_proj.weight": "model-00054-of-00101.safetensors", + "model.layers.50.mlp.experts.25.down_proj.weight": "model-00054-of-00101.safetensors", + "model.layers.50.mlp.experts.26.gate_proj.weight": "model-00054-of-00101.safetensors", + "model.layers.50.mlp.experts.26.up_proj.weight": "model-00054-of-00101.safetensors", + "model.layers.50.mlp.experts.26.down_proj.weight": "model-00054-of-00101.safetensors", + "model.layers.50.mlp.experts.27.gate_proj.weight": "model-00054-of-00101.safetensors", + "model.layers.50.mlp.experts.27.up_proj.weight": "model-00054-of-00101.safetensors", + "model.layers.50.mlp.experts.27.down_proj.weight": "model-00054-of-00101.safetensors", + "model.layers.50.mlp.experts.28.gate_proj.weight": "model-00054-of-00101.safetensors", + "model.layers.50.mlp.experts.28.up_proj.weight": "model-00054-of-00101.safetensors", + "model.layers.50.mlp.experts.28.down_proj.weight": "model-00054-of-00101.safetensors", + "model.layers.50.mlp.experts.29.gate_proj.weight": "model-00054-of-00101.safetensors", + "model.layers.50.mlp.experts.29.up_proj.weight": "model-00054-of-00101.safetensors", + "model.layers.50.mlp.experts.29.down_proj.weight": "model-00054-of-00101.safetensors", + "model.layers.50.mlp.experts.30.gate_proj.weight": "model-00054-of-00101.safetensors", + "model.layers.50.mlp.experts.30.up_proj.weight": "model-00054-of-00101.safetensors", + "model.layers.50.mlp.experts.30.down_proj.weight": "model-00054-of-00101.safetensors", + "model.layers.50.mlp.experts.31.gate_proj.weight": "model-00054-of-00101.safetensors", + "model.layers.50.mlp.experts.31.up_proj.weight": "model-00054-of-00101.safetensors", + "model.layers.50.mlp.experts.31.down_proj.weight": "model-00054-of-00101.safetensors", + "model.layers.50.mlp.experts.32.gate_proj.weight": "model-00054-of-00101.safetensors", + "model.layers.50.mlp.experts.32.up_proj.weight": "model-00054-of-00101.safetensors", + "model.layers.50.mlp.experts.32.down_proj.weight": "model-00054-of-00101.safetensors", + "model.layers.50.mlp.experts.33.gate_proj.weight": "model-00054-of-00101.safetensors", + "model.layers.50.mlp.experts.33.up_proj.weight": "model-00054-of-00101.safetensors", + "model.layers.50.mlp.experts.33.down_proj.weight": "model-00054-of-00101.safetensors", + "model.layers.50.mlp.experts.34.gate_proj.weight": "model-00054-of-00101.safetensors", + "model.layers.50.mlp.experts.34.up_proj.weight": "model-00054-of-00101.safetensors", + "model.layers.50.mlp.experts.34.down_proj.weight": "model-00054-of-00101.safetensors", + "model.layers.50.mlp.experts.35.gate_proj.weight": "model-00054-of-00101.safetensors", + "model.layers.50.mlp.experts.35.up_proj.weight": "model-00054-of-00101.safetensors", + "model.layers.50.mlp.experts.35.down_proj.weight": "model-00054-of-00101.safetensors", + "model.layers.50.mlp.experts.36.gate_proj.weight": "model-00054-of-00101.safetensors", + "model.layers.50.mlp.experts.36.up_proj.weight": "model-00054-of-00101.safetensors", + "model.layers.50.mlp.experts.36.down_proj.weight": "model-00054-of-00101.safetensors", + "model.layers.50.mlp.experts.37.gate_proj.weight": "model-00054-of-00101.safetensors", + "model.layers.50.mlp.experts.37.up_proj.weight": "model-00054-of-00101.safetensors", + "model.layers.50.mlp.experts.37.down_proj.weight": "model-00054-of-00101.safetensors", + "model.layers.50.mlp.experts.38.gate_proj.weight": "model-00054-of-00101.safetensors", + "model.layers.50.mlp.experts.38.up_proj.weight": "model-00054-of-00101.safetensors", + "model.layers.50.mlp.experts.38.down_proj.weight": "model-00054-of-00101.safetensors", + "model.layers.50.mlp.experts.39.gate_proj.weight": "model-00054-of-00101.safetensors", + "model.layers.50.mlp.experts.39.up_proj.weight": "model-00054-of-00101.safetensors", + "model.layers.50.mlp.experts.39.down_proj.weight": "model-00054-of-00101.safetensors", + "model.layers.50.mlp.experts.40.gate_proj.weight": "model-00054-of-00101.safetensors", + "model.layers.50.mlp.experts.40.up_proj.weight": "model-00054-of-00101.safetensors", + "model.layers.50.mlp.experts.40.down_proj.weight": "model-00054-of-00101.safetensors", + "model.layers.50.mlp.experts.41.gate_proj.weight": "model-00054-of-00101.safetensors", + "model.layers.50.mlp.experts.41.up_proj.weight": "model-00054-of-00101.safetensors", + "model.layers.50.mlp.experts.41.down_proj.weight": "model-00054-of-00101.safetensors", + "model.layers.50.mlp.experts.42.gate_proj.weight": "model-00054-of-00101.safetensors", + "model.layers.50.mlp.experts.42.up_proj.weight": "model-00054-of-00101.safetensors", + "model.layers.50.mlp.experts.42.down_proj.weight": "model-00054-of-00101.safetensors", + "model.layers.50.mlp.experts.43.gate_proj.weight": "model-00054-of-00101.safetensors", + "model.layers.50.mlp.experts.43.up_proj.weight": "model-00054-of-00101.safetensors", + "model.layers.50.mlp.experts.43.down_proj.weight": "model-00054-of-00101.safetensors", + "model.layers.50.mlp.experts.44.gate_proj.weight": "model-00054-of-00101.safetensors", + "model.layers.50.mlp.experts.44.up_proj.weight": "model-00054-of-00101.safetensors", + "model.layers.50.mlp.experts.44.down_proj.weight": "model-00054-of-00101.safetensors", + "model.layers.50.mlp.experts.45.gate_proj.weight": "model-00054-of-00101.safetensors", + "model.layers.50.mlp.experts.45.up_proj.weight": "model-00054-of-00101.safetensors", + "model.layers.50.mlp.experts.45.down_proj.weight": "model-00054-of-00101.safetensors", + "model.layers.50.mlp.experts.46.gate_proj.weight": "model-00054-of-00101.safetensors", + "model.layers.50.mlp.experts.46.up_proj.weight": "model-00054-of-00101.safetensors", + "model.layers.50.mlp.experts.46.down_proj.weight": "model-00054-of-00101.safetensors", + "model.layers.50.mlp.experts.47.gate_proj.weight": "model-00054-of-00101.safetensors", + "model.layers.50.mlp.experts.47.up_proj.weight": "model-00054-of-00101.safetensors", + "model.layers.50.mlp.experts.47.down_proj.weight": "model-00054-of-00101.safetensors", + "model.layers.50.mlp.experts.48.gate_proj.weight": "model-00054-of-00101.safetensors", + "model.layers.50.mlp.experts.48.up_proj.weight": "model-00054-of-00101.safetensors", + "model.layers.50.mlp.experts.48.down_proj.weight": "model-00054-of-00101.safetensors", + "model.layers.50.mlp.experts.49.gate_proj.weight": "model-00054-of-00101.safetensors", + "model.layers.50.mlp.experts.49.up_proj.weight": "model-00054-of-00101.safetensors", + "model.layers.50.mlp.experts.49.down_proj.weight": "model-00054-of-00101.safetensors", + "model.layers.50.mlp.experts.50.gate_proj.weight": "model-00054-of-00101.safetensors", + "model.layers.50.mlp.experts.50.up_proj.weight": "model-00054-of-00101.safetensors", + "model.layers.50.mlp.experts.50.down_proj.weight": "model-00054-of-00101.safetensors", + "model.layers.50.mlp.experts.51.gate_proj.weight": "model-00054-of-00101.safetensors", + "model.layers.50.mlp.experts.51.up_proj.weight": "model-00054-of-00101.safetensors", + "model.layers.50.mlp.experts.51.down_proj.weight": "model-00054-of-00101.safetensors", + "model.layers.50.mlp.experts.52.gate_proj.weight": "model-00054-of-00101.safetensors", + "model.layers.50.mlp.experts.52.up_proj.weight": "model-00054-of-00101.safetensors", + "model.layers.50.mlp.experts.52.down_proj.weight": "model-00054-of-00101.safetensors", + "model.layers.50.mlp.experts.53.gate_proj.weight": "model-00054-of-00101.safetensors", + "model.layers.50.mlp.experts.53.up_proj.weight": "model-00054-of-00101.safetensors", + "model.layers.50.mlp.experts.53.down_proj.weight": "model-00054-of-00101.safetensors", + "model.layers.50.mlp.experts.54.gate_proj.weight": "model-00054-of-00101.safetensors", + "model.layers.50.mlp.experts.54.up_proj.weight": "model-00054-of-00101.safetensors", + "model.layers.50.mlp.experts.54.down_proj.weight": "model-00054-of-00101.safetensors", + "model.layers.50.mlp.experts.55.gate_proj.weight": "model-00054-of-00101.safetensors", + "model.layers.50.mlp.experts.55.up_proj.weight": "model-00054-of-00101.safetensors", + "model.layers.50.mlp.experts.55.down_proj.weight": "model-00054-of-00101.safetensors", + "model.layers.50.mlp.experts.56.gate_proj.weight": "model-00054-of-00101.safetensors", + "model.layers.50.mlp.experts.56.up_proj.weight": "model-00054-of-00101.safetensors", + "model.layers.50.mlp.experts.56.down_proj.weight": "model-00054-of-00101.safetensors", + "model.layers.50.mlp.experts.57.gate_proj.weight": "model-00054-of-00101.safetensors", + "model.layers.50.mlp.experts.57.up_proj.weight": "model-00054-of-00101.safetensors", + "model.layers.50.mlp.experts.57.down_proj.weight": "model-00054-of-00101.safetensors", + "model.layers.50.mlp.experts.58.gate_proj.weight": "model-00054-of-00101.safetensors", + "model.layers.50.mlp.experts.58.up_proj.weight": "model-00054-of-00101.safetensors", + "model.layers.50.mlp.experts.58.down_proj.weight": "model-00054-of-00101.safetensors", + "model.layers.50.mlp.experts.59.gate_proj.weight": "model-00054-of-00101.safetensors", + "model.layers.50.mlp.experts.59.up_proj.weight": "model-00054-of-00101.safetensors", + "model.layers.50.mlp.experts.59.down_proj.weight": "model-00054-of-00101.safetensors", + "model.layers.50.mlp.experts.60.gate_proj.weight": "model-00054-of-00101.safetensors", + "model.layers.50.mlp.experts.60.up_proj.weight": "model-00054-of-00101.safetensors", + "model.layers.50.mlp.experts.60.down_proj.weight": "model-00054-of-00101.safetensors", + "model.layers.50.mlp.experts.61.gate_proj.weight": "model-00054-of-00101.safetensors", + "model.layers.50.mlp.experts.61.up_proj.weight": "model-00054-of-00101.safetensors", + "model.layers.50.mlp.experts.61.down_proj.weight": "model-00054-of-00101.safetensors", + "model.layers.50.mlp.experts.62.gate_proj.weight": "model-00054-of-00101.safetensors", + "model.layers.50.mlp.experts.62.up_proj.weight": "model-00054-of-00101.safetensors", + "model.layers.50.mlp.experts.62.down_proj.weight": "model-00054-of-00101.safetensors", + "model.layers.50.mlp.experts.63.gate_proj.weight": "model-00054-of-00101.safetensors", + "model.layers.50.mlp.experts.63.up_proj.weight": "model-00054-of-00101.safetensors", + "model.layers.50.mlp.experts.63.down_proj.weight": "model-00054-of-00101.safetensors", + "model.layers.50.mlp.experts.64.gate_proj.weight": "model-00054-of-00101.safetensors", + "model.layers.50.mlp.experts.64.up_proj.weight": "model-00054-of-00101.safetensors", + "model.layers.50.mlp.experts.64.down_proj.weight": "model-00054-of-00101.safetensors", + "model.layers.50.mlp.experts.65.gate_proj.weight": "model-00054-of-00101.safetensors", + "model.layers.50.mlp.experts.65.up_proj.weight": "model-00054-of-00101.safetensors", + "model.layers.50.mlp.experts.65.down_proj.weight": "model-00054-of-00101.safetensors", + "model.layers.50.mlp.experts.66.gate_proj.weight": "model-00054-of-00101.safetensors", + "model.layers.50.mlp.experts.66.up_proj.weight": "model-00054-of-00101.safetensors", + "model.layers.50.mlp.experts.66.down_proj.weight": "model-00054-of-00101.safetensors", + "model.layers.50.mlp.experts.67.gate_proj.weight": "model-00054-of-00101.safetensors", + "model.layers.50.mlp.experts.67.up_proj.weight": "model-00054-of-00101.safetensors", + "model.layers.50.mlp.experts.67.down_proj.weight": "model-00054-of-00101.safetensors", + "model.layers.50.mlp.experts.68.gate_proj.weight": "model-00054-of-00101.safetensors", + "model.layers.50.mlp.experts.68.up_proj.weight": "model-00054-of-00101.safetensors", + "model.layers.50.mlp.experts.68.down_proj.weight": "model-00054-of-00101.safetensors", + "model.layers.50.mlp.experts.69.gate_proj.weight": "model-00054-of-00101.safetensors", + "model.layers.50.mlp.experts.69.up_proj.weight": "model-00054-of-00101.safetensors", + "model.layers.50.mlp.experts.69.down_proj.weight": "model-00054-of-00101.safetensors", + "model.layers.50.mlp.experts.70.gate_proj.weight": "model-00054-of-00101.safetensors", + "model.layers.50.mlp.experts.70.up_proj.weight": "model-00054-of-00101.safetensors", + "model.layers.50.mlp.experts.70.down_proj.weight": "model-00054-of-00101.safetensors", + "model.layers.50.mlp.experts.71.gate_proj.weight": "model-00054-of-00101.safetensors", + "model.layers.50.mlp.experts.71.up_proj.weight": "model-00054-of-00101.safetensors", + "model.layers.50.mlp.experts.71.down_proj.weight": "model-00054-of-00101.safetensors", + "model.layers.50.mlp.experts.72.gate_proj.weight": "model-00054-of-00101.safetensors", + "model.layers.50.mlp.experts.72.up_proj.weight": "model-00054-of-00101.safetensors", + "model.layers.50.mlp.experts.72.down_proj.weight": "model-00054-of-00101.safetensors", + "model.layers.50.mlp.experts.73.gate_proj.weight": "model-00054-of-00101.safetensors", + "model.layers.50.mlp.experts.73.up_proj.weight": "model-00054-of-00101.safetensors", + "model.layers.50.mlp.experts.73.down_proj.weight": "model-00054-of-00101.safetensors", + "model.layers.50.mlp.experts.74.gate_proj.weight": "model-00054-of-00101.safetensors", + "model.layers.50.mlp.experts.74.up_proj.weight": "model-00054-of-00101.safetensors", + "model.layers.50.mlp.experts.74.down_proj.weight": "model-00054-of-00101.safetensors", + "model.layers.50.mlp.experts.75.gate_proj.weight": "model-00054-of-00101.safetensors", + "model.layers.50.mlp.experts.75.up_proj.weight": "model-00054-of-00101.safetensors", + "model.layers.50.mlp.experts.75.down_proj.weight": "model-00054-of-00101.safetensors", + "model.layers.50.mlp.experts.76.gate_proj.weight": "model-00054-of-00101.safetensors", + "model.layers.50.mlp.experts.76.up_proj.weight": "model-00054-of-00101.safetensors", + "model.layers.50.mlp.experts.76.down_proj.weight": "model-00054-of-00101.safetensors", + "model.layers.50.mlp.experts.77.gate_proj.weight": "model-00054-of-00101.safetensors", + "model.layers.50.mlp.experts.77.up_proj.weight": "model-00054-of-00101.safetensors", + "model.layers.50.mlp.experts.77.down_proj.weight": "model-00054-of-00101.safetensors", + "model.layers.50.mlp.experts.78.gate_proj.weight": "model-00054-of-00101.safetensors", + "model.layers.50.mlp.experts.78.up_proj.weight": "model-00054-of-00101.safetensors", + "model.layers.50.mlp.experts.78.down_proj.weight": "model-00054-of-00101.safetensors", + "model.layers.50.mlp.experts.79.gate_proj.weight": "model-00054-of-00101.safetensors", + "model.layers.50.mlp.experts.79.up_proj.weight": "model-00054-of-00101.safetensors", + "model.layers.50.mlp.experts.79.down_proj.weight": "model-00054-of-00101.safetensors", + "model.layers.50.mlp.experts.80.gate_proj.weight": "model-00054-of-00101.safetensors", + "model.layers.50.mlp.experts.80.up_proj.weight": "model-00054-of-00101.safetensors", + "model.layers.50.mlp.experts.80.down_proj.weight": "model-00054-of-00101.safetensors", + "model.layers.50.mlp.experts.81.gate_proj.weight": "model-00054-of-00101.safetensors", + "model.layers.50.mlp.experts.81.up_proj.weight": "model-00054-of-00101.safetensors", + "model.layers.50.mlp.experts.81.down_proj.weight": "model-00054-of-00101.safetensors", + "model.layers.50.mlp.experts.82.gate_proj.weight": "model-00054-of-00101.safetensors", + "model.layers.50.mlp.experts.82.up_proj.weight": "model-00054-of-00101.safetensors", + "model.layers.50.mlp.experts.82.down_proj.weight": "model-00054-of-00101.safetensors", + "model.layers.50.mlp.experts.83.gate_proj.weight": "model-00054-of-00101.safetensors", + "model.layers.50.mlp.experts.83.up_proj.weight": "model-00054-of-00101.safetensors", + "model.layers.50.mlp.experts.83.down_proj.weight": "model-00054-of-00101.safetensors", + "model.layers.50.mlp.experts.84.gate_proj.weight": "model-00054-of-00101.safetensors", + "model.layers.50.mlp.experts.84.up_proj.weight": "model-00054-of-00101.safetensors", + "model.layers.50.mlp.experts.84.down_proj.weight": "model-00054-of-00101.safetensors", + "model.layers.50.mlp.experts.85.gate_proj.weight": "model-00054-of-00101.safetensors", + "model.layers.50.mlp.experts.85.up_proj.weight": "model-00054-of-00101.safetensors", + "model.layers.50.mlp.experts.85.down_proj.weight": "model-00054-of-00101.safetensors", + "model.layers.50.mlp.experts.86.gate_proj.weight": "model-00054-of-00101.safetensors", + "model.layers.50.mlp.experts.86.up_proj.weight": "model-00054-of-00101.safetensors", + "model.layers.50.mlp.experts.86.down_proj.weight": "model-00054-of-00101.safetensors", + "model.layers.50.mlp.experts.87.gate_proj.weight": "model-00054-of-00101.safetensors", + "model.layers.50.mlp.experts.87.up_proj.weight": "model-00054-of-00101.safetensors", + "model.layers.50.mlp.experts.87.down_proj.weight": "model-00054-of-00101.safetensors", + "model.layers.50.mlp.experts.88.gate_proj.weight": "model-00055-of-00101.safetensors", + "model.layers.50.mlp.experts.88.up_proj.weight": "model-00055-of-00101.safetensors", + "model.layers.50.mlp.experts.88.down_proj.weight": "model-00055-of-00101.safetensors", + "model.layers.50.mlp.experts.89.gate_proj.weight": "model-00055-of-00101.safetensors", + "model.layers.50.mlp.experts.89.up_proj.weight": "model-00055-of-00101.safetensors", + "model.layers.50.mlp.experts.89.down_proj.weight": "model-00055-of-00101.safetensors", + "model.layers.50.mlp.experts.90.gate_proj.weight": "model-00055-of-00101.safetensors", + "model.layers.50.mlp.experts.90.up_proj.weight": "model-00055-of-00101.safetensors", + "model.layers.50.mlp.experts.90.down_proj.weight": "model-00055-of-00101.safetensors", + "model.layers.50.mlp.experts.91.gate_proj.weight": "model-00055-of-00101.safetensors", + "model.layers.50.mlp.experts.91.up_proj.weight": "model-00055-of-00101.safetensors", + "model.layers.50.mlp.experts.91.down_proj.weight": "model-00055-of-00101.safetensors", + "model.layers.50.mlp.experts.92.gate_proj.weight": "model-00055-of-00101.safetensors", + "model.layers.50.mlp.experts.92.up_proj.weight": "model-00055-of-00101.safetensors", + "model.layers.50.mlp.experts.92.down_proj.weight": "model-00055-of-00101.safetensors", + "model.layers.50.mlp.experts.93.gate_proj.weight": "model-00055-of-00101.safetensors", + "model.layers.50.mlp.experts.93.up_proj.weight": "model-00055-of-00101.safetensors", + "model.layers.50.mlp.experts.93.down_proj.weight": "model-00055-of-00101.safetensors", + "model.layers.50.mlp.experts.94.gate_proj.weight": "model-00055-of-00101.safetensors", + "model.layers.50.mlp.experts.94.up_proj.weight": "model-00055-of-00101.safetensors", + "model.layers.50.mlp.experts.94.down_proj.weight": "model-00055-of-00101.safetensors", + "model.layers.50.mlp.experts.95.gate_proj.weight": "model-00055-of-00101.safetensors", + "model.layers.50.mlp.experts.95.up_proj.weight": "model-00055-of-00101.safetensors", + "model.layers.50.mlp.experts.95.down_proj.weight": "model-00055-of-00101.safetensors", + "model.layers.50.mlp.experts.96.gate_proj.weight": "model-00055-of-00101.safetensors", + "model.layers.50.mlp.experts.96.up_proj.weight": "model-00055-of-00101.safetensors", + "model.layers.50.mlp.experts.96.down_proj.weight": "model-00055-of-00101.safetensors", + "model.layers.50.mlp.experts.97.gate_proj.weight": "model-00055-of-00101.safetensors", + "model.layers.50.mlp.experts.97.up_proj.weight": "model-00055-of-00101.safetensors", + "model.layers.50.mlp.experts.97.down_proj.weight": "model-00055-of-00101.safetensors", + "model.layers.50.mlp.experts.98.gate_proj.weight": "model-00055-of-00101.safetensors", + "model.layers.50.mlp.experts.98.up_proj.weight": "model-00055-of-00101.safetensors", + "model.layers.50.mlp.experts.98.down_proj.weight": "model-00055-of-00101.safetensors", + "model.layers.50.mlp.experts.99.gate_proj.weight": "model-00055-of-00101.safetensors", + "model.layers.50.mlp.experts.99.up_proj.weight": "model-00055-of-00101.safetensors", + "model.layers.50.mlp.experts.99.down_proj.weight": "model-00055-of-00101.safetensors", + "model.layers.50.mlp.experts.100.gate_proj.weight": "model-00055-of-00101.safetensors", + "model.layers.50.mlp.experts.100.up_proj.weight": "model-00055-of-00101.safetensors", + "model.layers.50.mlp.experts.100.down_proj.weight": "model-00055-of-00101.safetensors", + "model.layers.50.mlp.experts.101.gate_proj.weight": "model-00055-of-00101.safetensors", + "model.layers.50.mlp.experts.101.up_proj.weight": "model-00055-of-00101.safetensors", + "model.layers.50.mlp.experts.101.down_proj.weight": "model-00055-of-00101.safetensors", + "model.layers.50.mlp.experts.102.gate_proj.weight": "model-00055-of-00101.safetensors", + "model.layers.50.mlp.experts.102.up_proj.weight": "model-00055-of-00101.safetensors", + "model.layers.50.mlp.experts.102.down_proj.weight": "model-00055-of-00101.safetensors", + "model.layers.50.mlp.experts.103.gate_proj.weight": "model-00055-of-00101.safetensors", + "model.layers.50.mlp.experts.103.up_proj.weight": "model-00055-of-00101.safetensors", + "model.layers.50.mlp.experts.103.down_proj.weight": "model-00055-of-00101.safetensors", + "model.layers.50.mlp.experts.104.gate_proj.weight": "model-00055-of-00101.safetensors", + "model.layers.50.mlp.experts.104.up_proj.weight": "model-00055-of-00101.safetensors", + "model.layers.50.mlp.experts.104.down_proj.weight": "model-00055-of-00101.safetensors", + "model.layers.50.mlp.experts.105.gate_proj.weight": "model-00055-of-00101.safetensors", + "model.layers.50.mlp.experts.105.up_proj.weight": "model-00055-of-00101.safetensors", + "model.layers.50.mlp.experts.105.down_proj.weight": "model-00055-of-00101.safetensors", + "model.layers.50.mlp.experts.106.gate_proj.weight": "model-00055-of-00101.safetensors", + "model.layers.50.mlp.experts.106.up_proj.weight": "model-00055-of-00101.safetensors", + "model.layers.50.mlp.experts.106.down_proj.weight": "model-00055-of-00101.safetensors", + "model.layers.50.mlp.experts.107.gate_proj.weight": "model-00055-of-00101.safetensors", + "model.layers.50.mlp.experts.107.up_proj.weight": "model-00055-of-00101.safetensors", + "model.layers.50.mlp.experts.107.down_proj.weight": "model-00055-of-00101.safetensors", + "model.layers.50.mlp.experts.108.gate_proj.weight": "model-00055-of-00101.safetensors", + "model.layers.50.mlp.experts.108.up_proj.weight": "model-00055-of-00101.safetensors", + "model.layers.50.mlp.experts.108.down_proj.weight": "model-00055-of-00101.safetensors", + "model.layers.50.mlp.experts.109.gate_proj.weight": "model-00055-of-00101.safetensors", + "model.layers.50.mlp.experts.109.up_proj.weight": "model-00055-of-00101.safetensors", + "model.layers.50.mlp.experts.109.down_proj.weight": "model-00055-of-00101.safetensors", + "model.layers.50.mlp.experts.110.gate_proj.weight": "model-00055-of-00101.safetensors", + "model.layers.50.mlp.experts.110.up_proj.weight": "model-00055-of-00101.safetensors", + "model.layers.50.mlp.experts.110.down_proj.weight": "model-00055-of-00101.safetensors", + "model.layers.50.mlp.experts.111.gate_proj.weight": "model-00055-of-00101.safetensors", + "model.layers.50.mlp.experts.111.up_proj.weight": "model-00055-of-00101.safetensors", + "model.layers.50.mlp.experts.111.down_proj.weight": "model-00055-of-00101.safetensors", + "model.layers.50.mlp.experts.112.gate_proj.weight": "model-00055-of-00101.safetensors", + "model.layers.50.mlp.experts.112.up_proj.weight": "model-00055-of-00101.safetensors", + "model.layers.50.mlp.experts.112.down_proj.weight": "model-00055-of-00101.safetensors", + "model.layers.50.mlp.experts.113.gate_proj.weight": "model-00055-of-00101.safetensors", + "model.layers.50.mlp.experts.113.up_proj.weight": "model-00055-of-00101.safetensors", + "model.layers.50.mlp.experts.113.down_proj.weight": "model-00055-of-00101.safetensors", + "model.layers.50.mlp.experts.114.gate_proj.weight": "model-00055-of-00101.safetensors", + "model.layers.50.mlp.experts.114.up_proj.weight": "model-00055-of-00101.safetensors", + "model.layers.50.mlp.experts.114.down_proj.weight": "model-00055-of-00101.safetensors", + "model.layers.50.mlp.experts.115.gate_proj.weight": "model-00055-of-00101.safetensors", + "model.layers.50.mlp.experts.115.up_proj.weight": "model-00055-of-00101.safetensors", + "model.layers.50.mlp.experts.115.down_proj.weight": "model-00055-of-00101.safetensors", + "model.layers.50.mlp.experts.116.gate_proj.weight": "model-00055-of-00101.safetensors", + "model.layers.50.mlp.experts.116.up_proj.weight": "model-00055-of-00101.safetensors", + "model.layers.50.mlp.experts.116.down_proj.weight": "model-00055-of-00101.safetensors", + "model.layers.50.mlp.experts.117.gate_proj.weight": "model-00055-of-00101.safetensors", + "model.layers.50.mlp.experts.117.up_proj.weight": "model-00055-of-00101.safetensors", + "model.layers.50.mlp.experts.117.down_proj.weight": "model-00055-of-00101.safetensors", + "model.layers.50.mlp.experts.118.gate_proj.weight": "model-00055-of-00101.safetensors", + "model.layers.50.mlp.experts.118.up_proj.weight": "model-00055-of-00101.safetensors", + "model.layers.50.mlp.experts.118.down_proj.weight": "model-00055-of-00101.safetensors", + "model.layers.50.mlp.experts.119.gate_proj.weight": "model-00055-of-00101.safetensors", + "model.layers.50.mlp.experts.119.up_proj.weight": "model-00055-of-00101.safetensors", + "model.layers.50.mlp.experts.119.down_proj.weight": "model-00055-of-00101.safetensors", + "model.layers.50.mlp.gate.weight": "model-00055-of-00101.safetensors", + "model.layers.50.mlp.gate.e_score_correction_bias": "model-00055-of-00101.safetensors", + "model.layers.50.mlp.shared_experts.gate_proj.weight": "model-00055-of-00101.safetensors", + "model.layers.50.mlp.shared_experts.up_proj.weight": "model-00055-of-00101.safetensors", + "model.layers.50.mlp.shared_experts.down_proj.weight": "model-00055-of-00101.safetensors", + "model.layers.50.input_layernorm.weight": "model-00055-of-00101.safetensors", + "model.layers.50.post_attention_layernorm.weight": "model-00055-of-00101.safetensors", + "model.layers.51.self_attn.q_proj.weight": "model-00055-of-00101.safetensors", + "model.layers.51.self_attn.q_proj.bias": "model-00055-of-00101.safetensors", + "model.layers.51.self_attn.k_proj.weight": "model-00055-of-00101.safetensors", + "model.layers.51.self_attn.k_proj.bias": "model-00055-of-00101.safetensors", + "model.layers.51.self_attn.v_proj.weight": "model-00055-of-00101.safetensors", + "model.layers.51.self_attn.v_proj.bias": "model-00055-of-00101.safetensors", + "model.layers.51.self_attn.o_proj.weight": "model-00055-of-00101.safetensors", + "model.layers.51.self_attn.q_norm.weight": "model-00055-of-00101.safetensors", + "model.layers.51.self_attn.k_norm.weight": "model-00055-of-00101.safetensors", + "model.layers.51.mlp.experts.0.gate_proj.weight": "model-00055-of-00101.safetensors", + "model.layers.51.mlp.experts.0.up_proj.weight": "model-00055-of-00101.safetensors", + "model.layers.51.mlp.experts.0.down_proj.weight": "model-00055-of-00101.safetensors", + "model.layers.51.mlp.experts.1.gate_proj.weight": "model-00055-of-00101.safetensors", + "model.layers.51.mlp.experts.1.up_proj.weight": "model-00055-of-00101.safetensors", + "model.layers.51.mlp.experts.1.down_proj.weight": "model-00055-of-00101.safetensors", + "model.layers.51.mlp.experts.2.gate_proj.weight": "model-00055-of-00101.safetensors", + "model.layers.51.mlp.experts.2.up_proj.weight": "model-00055-of-00101.safetensors", + "model.layers.51.mlp.experts.2.down_proj.weight": "model-00055-of-00101.safetensors", + "model.layers.51.mlp.experts.3.gate_proj.weight": "model-00055-of-00101.safetensors", + "model.layers.51.mlp.experts.3.up_proj.weight": "model-00055-of-00101.safetensors", + "model.layers.51.mlp.experts.3.down_proj.weight": "model-00055-of-00101.safetensors", + "model.layers.51.mlp.experts.4.gate_proj.weight": "model-00055-of-00101.safetensors", + "model.layers.51.mlp.experts.4.up_proj.weight": "model-00055-of-00101.safetensors", + "model.layers.51.mlp.experts.4.down_proj.weight": "model-00055-of-00101.safetensors", + "model.layers.51.mlp.experts.5.gate_proj.weight": "model-00055-of-00101.safetensors", + "model.layers.51.mlp.experts.5.up_proj.weight": "model-00055-of-00101.safetensors", + "model.layers.51.mlp.experts.5.down_proj.weight": "model-00055-of-00101.safetensors", + "model.layers.51.mlp.experts.6.gate_proj.weight": "model-00055-of-00101.safetensors", + "model.layers.51.mlp.experts.6.up_proj.weight": "model-00055-of-00101.safetensors", + "model.layers.51.mlp.experts.6.down_proj.weight": "model-00055-of-00101.safetensors", + "model.layers.51.mlp.experts.7.gate_proj.weight": "model-00055-of-00101.safetensors", + "model.layers.51.mlp.experts.7.up_proj.weight": "model-00055-of-00101.safetensors", + "model.layers.51.mlp.experts.7.down_proj.weight": "model-00055-of-00101.safetensors", + "model.layers.51.mlp.experts.8.gate_proj.weight": "model-00055-of-00101.safetensors", + "model.layers.51.mlp.experts.8.up_proj.weight": "model-00055-of-00101.safetensors", + "model.layers.51.mlp.experts.8.down_proj.weight": "model-00055-of-00101.safetensors", + "model.layers.51.mlp.experts.9.gate_proj.weight": "model-00055-of-00101.safetensors", + "model.layers.51.mlp.experts.9.up_proj.weight": "model-00055-of-00101.safetensors", + "model.layers.51.mlp.experts.9.down_proj.weight": "model-00055-of-00101.safetensors", + "model.layers.51.mlp.experts.10.gate_proj.weight": "model-00055-of-00101.safetensors", + "model.layers.51.mlp.experts.10.up_proj.weight": "model-00055-of-00101.safetensors", + "model.layers.51.mlp.experts.10.down_proj.weight": "model-00055-of-00101.safetensors", + "model.layers.51.mlp.experts.11.gate_proj.weight": "model-00055-of-00101.safetensors", + "model.layers.51.mlp.experts.11.up_proj.weight": "model-00055-of-00101.safetensors", + "model.layers.51.mlp.experts.11.down_proj.weight": "model-00055-of-00101.safetensors", + "model.layers.51.mlp.experts.12.gate_proj.weight": "model-00055-of-00101.safetensors", + "model.layers.51.mlp.experts.12.up_proj.weight": "model-00055-of-00101.safetensors", + "model.layers.51.mlp.experts.12.down_proj.weight": "model-00055-of-00101.safetensors", + "model.layers.51.mlp.experts.13.gate_proj.weight": "model-00055-of-00101.safetensors", + "model.layers.51.mlp.experts.13.up_proj.weight": "model-00055-of-00101.safetensors", + "model.layers.51.mlp.experts.13.down_proj.weight": "model-00055-of-00101.safetensors", + "model.layers.51.mlp.experts.14.gate_proj.weight": "model-00055-of-00101.safetensors", + "model.layers.51.mlp.experts.14.up_proj.weight": "model-00055-of-00101.safetensors", + "model.layers.51.mlp.experts.14.down_proj.weight": "model-00055-of-00101.safetensors", + "model.layers.51.mlp.experts.15.gate_proj.weight": "model-00055-of-00101.safetensors", + "model.layers.51.mlp.experts.15.up_proj.weight": "model-00055-of-00101.safetensors", + "model.layers.51.mlp.experts.15.down_proj.weight": "model-00055-of-00101.safetensors", + "model.layers.51.mlp.experts.16.gate_proj.weight": "model-00055-of-00101.safetensors", + "model.layers.51.mlp.experts.16.up_proj.weight": "model-00055-of-00101.safetensors", + "model.layers.51.mlp.experts.16.down_proj.weight": "model-00055-of-00101.safetensors", + "model.layers.51.mlp.experts.17.gate_proj.weight": "model-00055-of-00101.safetensors", + "model.layers.51.mlp.experts.17.up_proj.weight": "model-00055-of-00101.safetensors", + "model.layers.51.mlp.experts.17.down_proj.weight": "model-00055-of-00101.safetensors", + "model.layers.51.mlp.experts.18.gate_proj.weight": "model-00055-of-00101.safetensors", + "model.layers.51.mlp.experts.18.up_proj.weight": "model-00055-of-00101.safetensors", + "model.layers.51.mlp.experts.18.down_proj.weight": "model-00055-of-00101.safetensors", + "model.layers.51.mlp.experts.19.gate_proj.weight": "model-00055-of-00101.safetensors", + "model.layers.51.mlp.experts.19.up_proj.weight": "model-00055-of-00101.safetensors", + "model.layers.51.mlp.experts.19.down_proj.weight": "model-00055-of-00101.safetensors", + "model.layers.51.mlp.experts.20.gate_proj.weight": "model-00055-of-00101.safetensors", + "model.layers.51.mlp.experts.20.up_proj.weight": "model-00055-of-00101.safetensors", + "model.layers.51.mlp.experts.20.down_proj.weight": "model-00055-of-00101.safetensors", + "model.layers.51.mlp.experts.21.gate_proj.weight": "model-00055-of-00101.safetensors", + "model.layers.51.mlp.experts.21.up_proj.weight": "model-00055-of-00101.safetensors", + "model.layers.51.mlp.experts.21.down_proj.weight": "model-00055-of-00101.safetensors", + "model.layers.51.mlp.experts.22.gate_proj.weight": "model-00055-of-00101.safetensors", + "model.layers.51.mlp.experts.22.up_proj.weight": "model-00055-of-00101.safetensors", + "model.layers.51.mlp.experts.22.down_proj.weight": "model-00055-of-00101.safetensors", + "model.layers.51.mlp.experts.23.gate_proj.weight": "model-00055-of-00101.safetensors", + "model.layers.51.mlp.experts.23.up_proj.weight": "model-00055-of-00101.safetensors", + "model.layers.51.mlp.experts.23.down_proj.weight": "model-00055-of-00101.safetensors", + "model.layers.51.mlp.experts.24.gate_proj.weight": "model-00055-of-00101.safetensors", + "model.layers.51.mlp.experts.24.up_proj.weight": "model-00055-of-00101.safetensors", + "model.layers.51.mlp.experts.24.down_proj.weight": "model-00055-of-00101.safetensors", + "model.layers.51.mlp.experts.25.gate_proj.weight": "model-00055-of-00101.safetensors", + "model.layers.51.mlp.experts.25.up_proj.weight": "model-00055-of-00101.safetensors", + "model.layers.51.mlp.experts.25.down_proj.weight": "model-00055-of-00101.safetensors", + "model.layers.51.mlp.experts.26.gate_proj.weight": "model-00055-of-00101.safetensors", + "model.layers.51.mlp.experts.26.up_proj.weight": "model-00055-of-00101.safetensors", + "model.layers.51.mlp.experts.26.down_proj.weight": "model-00055-of-00101.safetensors", + "model.layers.51.mlp.experts.27.gate_proj.weight": "model-00055-of-00101.safetensors", + "model.layers.51.mlp.experts.27.up_proj.weight": "model-00055-of-00101.safetensors", + "model.layers.51.mlp.experts.27.down_proj.weight": "model-00055-of-00101.safetensors", + "model.layers.51.mlp.experts.28.gate_proj.weight": "model-00055-of-00101.safetensors", + "model.layers.51.mlp.experts.28.up_proj.weight": "model-00055-of-00101.safetensors", + "model.layers.51.mlp.experts.28.down_proj.weight": "model-00055-of-00101.safetensors", + "model.layers.51.mlp.experts.29.gate_proj.weight": "model-00055-of-00101.safetensors", + "model.layers.51.mlp.experts.29.up_proj.weight": "model-00055-of-00101.safetensors", + "model.layers.51.mlp.experts.29.down_proj.weight": "model-00055-of-00101.safetensors", + "model.layers.51.mlp.experts.30.gate_proj.weight": "model-00055-of-00101.safetensors", + "model.layers.51.mlp.experts.30.up_proj.weight": "model-00055-of-00101.safetensors", + "model.layers.51.mlp.experts.30.down_proj.weight": "model-00055-of-00101.safetensors", + "model.layers.51.mlp.experts.31.gate_proj.weight": "model-00055-of-00101.safetensors", + "model.layers.51.mlp.experts.31.up_proj.weight": "model-00055-of-00101.safetensors", + "model.layers.51.mlp.experts.31.down_proj.weight": "model-00055-of-00101.safetensors", + "model.layers.51.mlp.experts.32.gate_proj.weight": "model-00055-of-00101.safetensors", + "model.layers.51.mlp.experts.32.up_proj.weight": "model-00055-of-00101.safetensors", + "model.layers.51.mlp.experts.32.down_proj.weight": "model-00055-of-00101.safetensors", + "model.layers.51.mlp.experts.33.gate_proj.weight": "model-00055-of-00101.safetensors", + "model.layers.51.mlp.experts.33.up_proj.weight": "model-00055-of-00101.safetensors", + "model.layers.51.mlp.experts.33.down_proj.weight": "model-00055-of-00101.safetensors", + "model.layers.51.mlp.experts.34.gate_proj.weight": "model-00055-of-00101.safetensors", + "model.layers.51.mlp.experts.34.up_proj.weight": "model-00055-of-00101.safetensors", + "model.layers.51.mlp.experts.34.down_proj.weight": "model-00055-of-00101.safetensors", + "model.layers.51.mlp.experts.35.gate_proj.weight": "model-00055-of-00101.safetensors", + "model.layers.51.mlp.experts.35.up_proj.weight": "model-00055-of-00101.safetensors", + "model.layers.51.mlp.experts.35.down_proj.weight": "model-00055-of-00101.safetensors", + "model.layers.51.mlp.experts.36.gate_proj.weight": "model-00055-of-00101.safetensors", + "model.layers.51.mlp.experts.36.up_proj.weight": "model-00055-of-00101.safetensors", + "model.layers.51.mlp.experts.36.down_proj.weight": "model-00055-of-00101.safetensors", + "model.layers.51.mlp.experts.37.gate_proj.weight": "model-00055-of-00101.safetensors", + "model.layers.51.mlp.experts.37.up_proj.weight": "model-00055-of-00101.safetensors", + "model.layers.51.mlp.experts.37.down_proj.weight": "model-00055-of-00101.safetensors", + "model.layers.51.mlp.experts.38.gate_proj.weight": "model-00055-of-00101.safetensors", + "model.layers.51.mlp.experts.38.up_proj.weight": "model-00055-of-00101.safetensors", + "model.layers.51.mlp.experts.38.down_proj.weight": "model-00055-of-00101.safetensors", + "model.layers.51.mlp.experts.39.gate_proj.weight": "model-00055-of-00101.safetensors", + "model.layers.51.mlp.experts.39.up_proj.weight": "model-00055-of-00101.safetensors", + "model.layers.51.mlp.experts.39.down_proj.weight": "model-00055-of-00101.safetensors", + "model.layers.51.mlp.experts.40.gate_proj.weight": "model-00055-of-00101.safetensors", + "model.layers.51.mlp.experts.40.up_proj.weight": "model-00055-of-00101.safetensors", + "model.layers.51.mlp.experts.40.down_proj.weight": "model-00055-of-00101.safetensors", + "model.layers.51.mlp.experts.41.gate_proj.weight": "model-00055-of-00101.safetensors", + "model.layers.51.mlp.experts.41.up_proj.weight": "model-00055-of-00101.safetensors", + "model.layers.51.mlp.experts.41.down_proj.weight": "model-00055-of-00101.safetensors", + "model.layers.51.mlp.experts.42.gate_proj.weight": "model-00055-of-00101.safetensors", + "model.layers.51.mlp.experts.42.up_proj.weight": "model-00055-of-00101.safetensors", + "model.layers.51.mlp.experts.42.down_proj.weight": "model-00055-of-00101.safetensors", + "model.layers.51.mlp.experts.43.gate_proj.weight": "model-00055-of-00101.safetensors", + "model.layers.51.mlp.experts.43.up_proj.weight": "model-00055-of-00101.safetensors", + "model.layers.51.mlp.experts.43.down_proj.weight": "model-00055-of-00101.safetensors", + "model.layers.51.mlp.experts.44.gate_proj.weight": "model-00055-of-00101.safetensors", + "model.layers.51.mlp.experts.44.up_proj.weight": "model-00055-of-00101.safetensors", + "model.layers.51.mlp.experts.44.down_proj.weight": "model-00055-of-00101.safetensors", + "model.layers.51.mlp.experts.45.gate_proj.weight": "model-00055-of-00101.safetensors", + "model.layers.51.mlp.experts.45.up_proj.weight": "model-00055-of-00101.safetensors", + "model.layers.51.mlp.experts.45.down_proj.weight": "model-00055-of-00101.safetensors", + "model.layers.51.mlp.experts.46.gate_proj.weight": "model-00055-of-00101.safetensors", + "model.layers.51.mlp.experts.46.up_proj.weight": "model-00055-of-00101.safetensors", + "model.layers.51.mlp.experts.46.down_proj.weight": "model-00055-of-00101.safetensors", + "model.layers.51.mlp.experts.47.gate_proj.weight": "model-00055-of-00101.safetensors", + "model.layers.51.mlp.experts.47.up_proj.weight": "model-00055-of-00101.safetensors", + "model.layers.51.mlp.experts.47.down_proj.weight": "model-00055-of-00101.safetensors", + "model.layers.51.mlp.experts.48.gate_proj.weight": "model-00055-of-00101.safetensors", + "model.layers.51.mlp.experts.48.up_proj.weight": "model-00055-of-00101.safetensors", + "model.layers.51.mlp.experts.48.down_proj.weight": "model-00055-of-00101.safetensors", + "model.layers.51.mlp.experts.49.gate_proj.weight": "model-00055-of-00101.safetensors", + "model.layers.51.mlp.experts.49.up_proj.weight": "model-00055-of-00101.safetensors", + "model.layers.51.mlp.experts.49.down_proj.weight": "model-00055-of-00101.safetensors", + "model.layers.51.mlp.experts.50.gate_proj.weight": "model-00055-of-00101.safetensors", + "model.layers.51.mlp.experts.50.up_proj.weight": "model-00055-of-00101.safetensors", + "model.layers.51.mlp.experts.50.down_proj.weight": "model-00055-of-00101.safetensors", + "model.layers.51.mlp.experts.51.gate_proj.weight": "model-00055-of-00101.safetensors", + "model.layers.51.mlp.experts.51.up_proj.weight": "model-00055-of-00101.safetensors", + "model.layers.51.mlp.experts.51.down_proj.weight": "model-00055-of-00101.safetensors", + "model.layers.51.mlp.experts.52.gate_proj.weight": "model-00055-of-00101.safetensors", + "model.layers.51.mlp.experts.52.up_proj.weight": "model-00055-of-00101.safetensors", + "model.layers.51.mlp.experts.52.down_proj.weight": "model-00055-of-00101.safetensors", + "model.layers.51.mlp.experts.53.gate_proj.weight": "model-00055-of-00101.safetensors", + "model.layers.51.mlp.experts.53.up_proj.weight": "model-00055-of-00101.safetensors", + "model.layers.51.mlp.experts.53.down_proj.weight": "model-00055-of-00101.safetensors", + "model.layers.51.mlp.experts.54.gate_proj.weight": "model-00055-of-00101.safetensors", + "model.layers.51.mlp.experts.54.up_proj.weight": "model-00055-of-00101.safetensors", + "model.layers.51.mlp.experts.54.down_proj.weight": "model-00055-of-00101.safetensors", + "model.layers.51.mlp.experts.55.gate_proj.weight": "model-00055-of-00101.safetensors", + "model.layers.51.mlp.experts.55.up_proj.weight": "model-00055-of-00101.safetensors", + "model.layers.51.mlp.experts.55.down_proj.weight": "model-00055-of-00101.safetensors", + "model.layers.51.mlp.experts.56.gate_proj.weight": "model-00055-of-00101.safetensors", + "model.layers.51.mlp.experts.56.up_proj.weight": "model-00055-of-00101.safetensors", + "model.layers.51.mlp.experts.56.down_proj.weight": "model-00055-of-00101.safetensors", + "model.layers.51.mlp.experts.57.gate_proj.weight": "model-00055-of-00101.safetensors", + "model.layers.51.mlp.experts.57.up_proj.weight": "model-00055-of-00101.safetensors", + "model.layers.51.mlp.experts.57.down_proj.weight": "model-00055-of-00101.safetensors", + "model.layers.51.mlp.experts.58.gate_proj.weight": "model-00055-of-00101.safetensors", + "model.layers.51.mlp.experts.58.up_proj.weight": "model-00055-of-00101.safetensors", + "model.layers.51.mlp.experts.58.down_proj.weight": "model-00055-of-00101.safetensors", + "model.layers.51.mlp.experts.59.gate_proj.weight": "model-00055-of-00101.safetensors", + "model.layers.51.mlp.experts.59.up_proj.weight": "model-00055-of-00101.safetensors", + "model.layers.51.mlp.experts.59.down_proj.weight": "model-00055-of-00101.safetensors", + "model.layers.51.mlp.experts.60.gate_proj.weight": "model-00055-of-00101.safetensors", + "model.layers.51.mlp.experts.60.up_proj.weight": "model-00055-of-00101.safetensors", + "model.layers.51.mlp.experts.60.down_proj.weight": "model-00055-of-00101.safetensors", + "model.layers.51.mlp.experts.61.gate_proj.weight": "model-00055-of-00101.safetensors", + "model.layers.51.mlp.experts.61.up_proj.weight": "model-00055-of-00101.safetensors", + "model.layers.51.mlp.experts.61.down_proj.weight": "model-00055-of-00101.safetensors", + "model.layers.51.mlp.experts.62.gate_proj.weight": "model-00055-of-00101.safetensors", + "model.layers.51.mlp.experts.62.up_proj.weight": "model-00055-of-00101.safetensors", + "model.layers.51.mlp.experts.62.down_proj.weight": "model-00055-of-00101.safetensors", + "model.layers.51.mlp.experts.63.gate_proj.weight": "model-00055-of-00101.safetensors", + "model.layers.51.mlp.experts.63.up_proj.weight": "model-00055-of-00101.safetensors", + "model.layers.51.mlp.experts.63.down_proj.weight": "model-00055-of-00101.safetensors", + "model.layers.51.mlp.experts.64.gate_proj.weight": "model-00055-of-00101.safetensors", + "model.layers.51.mlp.experts.64.up_proj.weight": "model-00055-of-00101.safetensors", + "model.layers.51.mlp.experts.64.down_proj.weight": "model-00055-of-00101.safetensors", + "model.layers.51.mlp.experts.65.gate_proj.weight": "model-00055-of-00101.safetensors", + "model.layers.51.mlp.experts.65.up_proj.weight": "model-00055-of-00101.safetensors", + "model.layers.51.mlp.experts.65.down_proj.weight": "model-00055-of-00101.safetensors", + "model.layers.51.mlp.experts.66.gate_proj.weight": "model-00055-of-00101.safetensors", + "model.layers.51.mlp.experts.66.up_proj.weight": "model-00055-of-00101.safetensors", + "model.layers.51.mlp.experts.66.down_proj.weight": "model-00055-of-00101.safetensors", + "model.layers.51.mlp.experts.67.gate_proj.weight": "model-00055-of-00101.safetensors", + "model.layers.51.mlp.experts.67.up_proj.weight": "model-00055-of-00101.safetensors", + "model.layers.51.mlp.experts.67.down_proj.weight": "model-00055-of-00101.safetensors", + "model.layers.51.mlp.experts.68.gate_proj.weight": "model-00055-of-00101.safetensors", + "model.layers.51.mlp.experts.68.up_proj.weight": "model-00055-of-00101.safetensors", + "model.layers.51.mlp.experts.68.down_proj.weight": "model-00055-of-00101.safetensors", + "model.layers.51.mlp.experts.69.gate_proj.weight": "model-00055-of-00101.safetensors", + "model.layers.51.mlp.experts.69.up_proj.weight": "model-00055-of-00101.safetensors", + "model.layers.51.mlp.experts.69.down_proj.weight": "model-00055-of-00101.safetensors", + "model.layers.51.mlp.experts.70.gate_proj.weight": "model-00055-of-00101.safetensors", + "model.layers.51.mlp.experts.70.up_proj.weight": "model-00055-of-00101.safetensors", + "model.layers.51.mlp.experts.70.down_proj.weight": "model-00055-of-00101.safetensors", + "model.layers.51.mlp.experts.71.gate_proj.weight": "model-00055-of-00101.safetensors", + "model.layers.51.mlp.experts.71.up_proj.weight": "model-00055-of-00101.safetensors", + "model.layers.51.mlp.experts.71.down_proj.weight": "model-00055-of-00101.safetensors", + "model.layers.51.mlp.experts.72.gate_proj.weight": "model-00055-of-00101.safetensors", + "model.layers.51.mlp.experts.72.up_proj.weight": "model-00055-of-00101.safetensors", + "model.layers.51.mlp.experts.72.down_proj.weight": "model-00055-of-00101.safetensors", + "model.layers.51.mlp.experts.73.gate_proj.weight": "model-00055-of-00101.safetensors", + "model.layers.51.mlp.experts.73.up_proj.weight": "model-00055-of-00101.safetensors", + "model.layers.51.mlp.experts.73.down_proj.weight": "model-00055-of-00101.safetensors", + "model.layers.51.mlp.experts.74.gate_proj.weight": "model-00055-of-00101.safetensors", + "model.layers.51.mlp.experts.74.up_proj.weight": "model-00055-of-00101.safetensors", + "model.layers.51.mlp.experts.74.down_proj.weight": "model-00056-of-00101.safetensors", + "model.layers.51.mlp.experts.75.gate_proj.weight": "model-00056-of-00101.safetensors", + "model.layers.51.mlp.experts.75.up_proj.weight": "model-00056-of-00101.safetensors", + "model.layers.51.mlp.experts.75.down_proj.weight": "model-00056-of-00101.safetensors", + "model.layers.51.mlp.experts.76.gate_proj.weight": "model-00056-of-00101.safetensors", + "model.layers.51.mlp.experts.76.up_proj.weight": "model-00056-of-00101.safetensors", + "model.layers.51.mlp.experts.76.down_proj.weight": "model-00056-of-00101.safetensors", + "model.layers.51.mlp.experts.77.gate_proj.weight": "model-00056-of-00101.safetensors", + "model.layers.51.mlp.experts.77.up_proj.weight": "model-00056-of-00101.safetensors", + "model.layers.51.mlp.experts.77.down_proj.weight": "model-00056-of-00101.safetensors", + "model.layers.51.mlp.experts.78.gate_proj.weight": "model-00056-of-00101.safetensors", + "model.layers.51.mlp.experts.78.up_proj.weight": "model-00056-of-00101.safetensors", + "model.layers.51.mlp.experts.78.down_proj.weight": "model-00056-of-00101.safetensors", + "model.layers.51.mlp.experts.79.gate_proj.weight": "model-00056-of-00101.safetensors", + "model.layers.51.mlp.experts.79.up_proj.weight": "model-00056-of-00101.safetensors", + "model.layers.51.mlp.experts.79.down_proj.weight": "model-00056-of-00101.safetensors", + "model.layers.51.mlp.experts.80.gate_proj.weight": "model-00056-of-00101.safetensors", + "model.layers.51.mlp.experts.80.up_proj.weight": "model-00056-of-00101.safetensors", + "model.layers.51.mlp.experts.80.down_proj.weight": "model-00056-of-00101.safetensors", + "model.layers.51.mlp.experts.81.gate_proj.weight": "model-00056-of-00101.safetensors", + "model.layers.51.mlp.experts.81.up_proj.weight": "model-00056-of-00101.safetensors", + "model.layers.51.mlp.experts.81.down_proj.weight": "model-00056-of-00101.safetensors", + "model.layers.51.mlp.experts.82.gate_proj.weight": "model-00056-of-00101.safetensors", + "model.layers.51.mlp.experts.82.up_proj.weight": "model-00056-of-00101.safetensors", + "model.layers.51.mlp.experts.82.down_proj.weight": "model-00056-of-00101.safetensors", + "model.layers.51.mlp.experts.83.gate_proj.weight": "model-00056-of-00101.safetensors", + "model.layers.51.mlp.experts.83.up_proj.weight": "model-00056-of-00101.safetensors", + "model.layers.51.mlp.experts.83.down_proj.weight": "model-00056-of-00101.safetensors", + "model.layers.51.mlp.experts.84.gate_proj.weight": "model-00056-of-00101.safetensors", + "model.layers.51.mlp.experts.84.up_proj.weight": "model-00056-of-00101.safetensors", + "model.layers.51.mlp.experts.84.down_proj.weight": "model-00056-of-00101.safetensors", + "model.layers.51.mlp.experts.85.gate_proj.weight": "model-00056-of-00101.safetensors", + "model.layers.51.mlp.experts.85.up_proj.weight": "model-00056-of-00101.safetensors", + "model.layers.51.mlp.experts.85.down_proj.weight": "model-00056-of-00101.safetensors", + "model.layers.51.mlp.experts.86.gate_proj.weight": "model-00056-of-00101.safetensors", + "model.layers.51.mlp.experts.86.up_proj.weight": "model-00056-of-00101.safetensors", + "model.layers.51.mlp.experts.86.down_proj.weight": "model-00056-of-00101.safetensors", + "model.layers.51.mlp.experts.87.gate_proj.weight": "model-00056-of-00101.safetensors", + "model.layers.51.mlp.experts.87.up_proj.weight": "model-00056-of-00101.safetensors", + "model.layers.51.mlp.experts.87.down_proj.weight": "model-00056-of-00101.safetensors", + "model.layers.51.mlp.experts.88.gate_proj.weight": "model-00056-of-00101.safetensors", + "model.layers.51.mlp.experts.88.up_proj.weight": "model-00056-of-00101.safetensors", + "model.layers.51.mlp.experts.88.down_proj.weight": "model-00056-of-00101.safetensors", + "model.layers.51.mlp.experts.89.gate_proj.weight": "model-00056-of-00101.safetensors", + "model.layers.51.mlp.experts.89.up_proj.weight": "model-00056-of-00101.safetensors", + "model.layers.51.mlp.experts.89.down_proj.weight": "model-00056-of-00101.safetensors", + "model.layers.51.mlp.experts.90.gate_proj.weight": "model-00056-of-00101.safetensors", + "model.layers.51.mlp.experts.90.up_proj.weight": "model-00056-of-00101.safetensors", + "model.layers.51.mlp.experts.90.down_proj.weight": "model-00056-of-00101.safetensors", + "model.layers.51.mlp.experts.91.gate_proj.weight": "model-00056-of-00101.safetensors", + "model.layers.51.mlp.experts.91.up_proj.weight": "model-00056-of-00101.safetensors", + "model.layers.51.mlp.experts.91.down_proj.weight": "model-00056-of-00101.safetensors", + "model.layers.51.mlp.experts.92.gate_proj.weight": "model-00056-of-00101.safetensors", + "model.layers.51.mlp.experts.92.up_proj.weight": "model-00056-of-00101.safetensors", + "model.layers.51.mlp.experts.92.down_proj.weight": "model-00056-of-00101.safetensors", + "model.layers.51.mlp.experts.93.gate_proj.weight": "model-00056-of-00101.safetensors", + "model.layers.51.mlp.experts.93.up_proj.weight": "model-00056-of-00101.safetensors", + "model.layers.51.mlp.experts.93.down_proj.weight": "model-00056-of-00101.safetensors", + "model.layers.51.mlp.experts.94.gate_proj.weight": "model-00056-of-00101.safetensors", + "model.layers.51.mlp.experts.94.up_proj.weight": "model-00056-of-00101.safetensors", + "model.layers.51.mlp.experts.94.down_proj.weight": "model-00056-of-00101.safetensors", + "model.layers.51.mlp.experts.95.gate_proj.weight": "model-00056-of-00101.safetensors", + "model.layers.51.mlp.experts.95.up_proj.weight": "model-00056-of-00101.safetensors", + "model.layers.51.mlp.experts.95.down_proj.weight": "model-00056-of-00101.safetensors", + "model.layers.51.mlp.experts.96.gate_proj.weight": "model-00056-of-00101.safetensors", + "model.layers.51.mlp.experts.96.up_proj.weight": "model-00056-of-00101.safetensors", + "model.layers.51.mlp.experts.96.down_proj.weight": "model-00056-of-00101.safetensors", + "model.layers.51.mlp.experts.97.gate_proj.weight": "model-00056-of-00101.safetensors", + "model.layers.51.mlp.experts.97.up_proj.weight": "model-00056-of-00101.safetensors", + "model.layers.51.mlp.experts.97.down_proj.weight": "model-00056-of-00101.safetensors", + "model.layers.51.mlp.experts.98.gate_proj.weight": "model-00056-of-00101.safetensors", + "model.layers.51.mlp.experts.98.up_proj.weight": "model-00056-of-00101.safetensors", + "model.layers.51.mlp.experts.98.down_proj.weight": "model-00056-of-00101.safetensors", + "model.layers.51.mlp.experts.99.gate_proj.weight": "model-00056-of-00101.safetensors", + "model.layers.51.mlp.experts.99.up_proj.weight": "model-00056-of-00101.safetensors", + "model.layers.51.mlp.experts.99.down_proj.weight": "model-00056-of-00101.safetensors", + "model.layers.51.mlp.experts.100.gate_proj.weight": "model-00056-of-00101.safetensors", + "model.layers.51.mlp.experts.100.up_proj.weight": "model-00056-of-00101.safetensors", + "model.layers.51.mlp.experts.100.down_proj.weight": "model-00056-of-00101.safetensors", + "model.layers.51.mlp.experts.101.gate_proj.weight": "model-00056-of-00101.safetensors", + "model.layers.51.mlp.experts.101.up_proj.weight": "model-00056-of-00101.safetensors", + "model.layers.51.mlp.experts.101.down_proj.weight": "model-00056-of-00101.safetensors", + "model.layers.51.mlp.experts.102.gate_proj.weight": "model-00056-of-00101.safetensors", + "model.layers.51.mlp.experts.102.up_proj.weight": "model-00056-of-00101.safetensors", + "model.layers.51.mlp.experts.102.down_proj.weight": "model-00056-of-00101.safetensors", + "model.layers.51.mlp.experts.103.gate_proj.weight": "model-00056-of-00101.safetensors", + "model.layers.51.mlp.experts.103.up_proj.weight": "model-00056-of-00101.safetensors", + "model.layers.51.mlp.experts.103.down_proj.weight": "model-00056-of-00101.safetensors", + "model.layers.51.mlp.experts.104.gate_proj.weight": "model-00056-of-00101.safetensors", + "model.layers.51.mlp.experts.104.up_proj.weight": "model-00056-of-00101.safetensors", + "model.layers.51.mlp.experts.104.down_proj.weight": "model-00056-of-00101.safetensors", + "model.layers.51.mlp.experts.105.gate_proj.weight": "model-00056-of-00101.safetensors", + "model.layers.51.mlp.experts.105.up_proj.weight": "model-00056-of-00101.safetensors", + "model.layers.51.mlp.experts.105.down_proj.weight": "model-00056-of-00101.safetensors", + "model.layers.51.mlp.experts.106.gate_proj.weight": "model-00056-of-00101.safetensors", + "model.layers.51.mlp.experts.106.up_proj.weight": "model-00056-of-00101.safetensors", + "model.layers.51.mlp.experts.106.down_proj.weight": "model-00056-of-00101.safetensors", + "model.layers.51.mlp.experts.107.gate_proj.weight": "model-00056-of-00101.safetensors", + "model.layers.51.mlp.experts.107.up_proj.weight": "model-00056-of-00101.safetensors", + "model.layers.51.mlp.experts.107.down_proj.weight": "model-00056-of-00101.safetensors", + "model.layers.51.mlp.experts.108.gate_proj.weight": "model-00056-of-00101.safetensors", + "model.layers.51.mlp.experts.108.up_proj.weight": "model-00056-of-00101.safetensors", + "model.layers.51.mlp.experts.108.down_proj.weight": "model-00056-of-00101.safetensors", + "model.layers.51.mlp.experts.109.gate_proj.weight": "model-00056-of-00101.safetensors", + "model.layers.51.mlp.experts.109.up_proj.weight": "model-00056-of-00101.safetensors", + "model.layers.51.mlp.experts.109.down_proj.weight": "model-00056-of-00101.safetensors", + "model.layers.51.mlp.experts.110.gate_proj.weight": "model-00056-of-00101.safetensors", + "model.layers.51.mlp.experts.110.up_proj.weight": "model-00056-of-00101.safetensors", + "model.layers.51.mlp.experts.110.down_proj.weight": "model-00056-of-00101.safetensors", + "model.layers.51.mlp.experts.111.gate_proj.weight": "model-00056-of-00101.safetensors", + "model.layers.51.mlp.experts.111.up_proj.weight": "model-00056-of-00101.safetensors", + "model.layers.51.mlp.experts.111.down_proj.weight": "model-00056-of-00101.safetensors", + "model.layers.51.mlp.experts.112.gate_proj.weight": "model-00056-of-00101.safetensors", + "model.layers.51.mlp.experts.112.up_proj.weight": "model-00056-of-00101.safetensors", + "model.layers.51.mlp.experts.112.down_proj.weight": "model-00056-of-00101.safetensors", + "model.layers.51.mlp.experts.113.gate_proj.weight": "model-00056-of-00101.safetensors", + "model.layers.51.mlp.experts.113.up_proj.weight": "model-00056-of-00101.safetensors", + "model.layers.51.mlp.experts.113.down_proj.weight": "model-00056-of-00101.safetensors", + "model.layers.51.mlp.experts.114.gate_proj.weight": "model-00056-of-00101.safetensors", + "model.layers.51.mlp.experts.114.up_proj.weight": "model-00056-of-00101.safetensors", + "model.layers.51.mlp.experts.114.down_proj.weight": "model-00056-of-00101.safetensors", + "model.layers.51.mlp.experts.115.gate_proj.weight": "model-00056-of-00101.safetensors", + "model.layers.51.mlp.experts.115.up_proj.weight": "model-00056-of-00101.safetensors", + "model.layers.51.mlp.experts.115.down_proj.weight": "model-00056-of-00101.safetensors", + "model.layers.51.mlp.experts.116.gate_proj.weight": "model-00056-of-00101.safetensors", + "model.layers.51.mlp.experts.116.up_proj.weight": "model-00056-of-00101.safetensors", + "model.layers.51.mlp.experts.116.down_proj.weight": "model-00056-of-00101.safetensors", + "model.layers.51.mlp.experts.117.gate_proj.weight": "model-00056-of-00101.safetensors", + "model.layers.51.mlp.experts.117.up_proj.weight": "model-00056-of-00101.safetensors", + "model.layers.51.mlp.experts.117.down_proj.weight": "model-00056-of-00101.safetensors", + "model.layers.51.mlp.experts.118.gate_proj.weight": "model-00056-of-00101.safetensors", + "model.layers.51.mlp.experts.118.up_proj.weight": "model-00056-of-00101.safetensors", + "model.layers.51.mlp.experts.118.down_proj.weight": "model-00056-of-00101.safetensors", + "model.layers.51.mlp.experts.119.gate_proj.weight": "model-00056-of-00101.safetensors", + "model.layers.51.mlp.experts.119.up_proj.weight": "model-00056-of-00101.safetensors", + "model.layers.51.mlp.experts.119.down_proj.weight": "model-00056-of-00101.safetensors", + "model.layers.51.mlp.gate.weight": "model-00056-of-00101.safetensors", + "model.layers.51.mlp.gate.e_score_correction_bias": "model-00056-of-00101.safetensors", + "model.layers.51.mlp.shared_experts.gate_proj.weight": "model-00056-of-00101.safetensors", + "model.layers.51.mlp.shared_experts.up_proj.weight": "model-00056-of-00101.safetensors", + "model.layers.51.mlp.shared_experts.down_proj.weight": "model-00056-of-00101.safetensors", + "model.layers.51.input_layernorm.weight": "model-00056-of-00101.safetensors", + "model.layers.51.post_attention_layernorm.weight": "model-00056-of-00101.safetensors", + "model.layers.52.self_attn.q_proj.weight": "model-00056-of-00101.safetensors", + "model.layers.52.self_attn.q_proj.bias": "model-00056-of-00101.safetensors", + "model.layers.52.self_attn.k_proj.weight": "model-00056-of-00101.safetensors", + "model.layers.52.self_attn.k_proj.bias": "model-00056-of-00101.safetensors", + "model.layers.52.self_attn.v_proj.weight": "model-00056-of-00101.safetensors", + "model.layers.52.self_attn.v_proj.bias": "model-00056-of-00101.safetensors", + "model.layers.52.self_attn.o_proj.weight": "model-00056-of-00101.safetensors", + "model.layers.52.self_attn.q_norm.weight": "model-00056-of-00101.safetensors", + "model.layers.52.self_attn.k_norm.weight": "model-00056-of-00101.safetensors", + "model.layers.52.mlp.experts.0.gate_proj.weight": "model-00056-of-00101.safetensors", + "model.layers.52.mlp.experts.0.up_proj.weight": "model-00056-of-00101.safetensors", + "model.layers.52.mlp.experts.0.down_proj.weight": "model-00056-of-00101.safetensors", + "model.layers.52.mlp.experts.1.gate_proj.weight": "model-00056-of-00101.safetensors", + "model.layers.52.mlp.experts.1.up_proj.weight": "model-00056-of-00101.safetensors", + "model.layers.52.mlp.experts.1.down_proj.weight": "model-00056-of-00101.safetensors", + "model.layers.52.mlp.experts.2.gate_proj.weight": "model-00056-of-00101.safetensors", + "model.layers.52.mlp.experts.2.up_proj.weight": "model-00056-of-00101.safetensors", + "model.layers.52.mlp.experts.2.down_proj.weight": "model-00056-of-00101.safetensors", + "model.layers.52.mlp.experts.3.gate_proj.weight": "model-00056-of-00101.safetensors", + "model.layers.52.mlp.experts.3.up_proj.weight": "model-00056-of-00101.safetensors", + "model.layers.52.mlp.experts.3.down_proj.weight": "model-00056-of-00101.safetensors", + "model.layers.52.mlp.experts.4.gate_proj.weight": "model-00056-of-00101.safetensors", + "model.layers.52.mlp.experts.4.up_proj.weight": "model-00056-of-00101.safetensors", + "model.layers.52.mlp.experts.4.down_proj.weight": "model-00056-of-00101.safetensors", + "model.layers.52.mlp.experts.5.gate_proj.weight": "model-00056-of-00101.safetensors", + "model.layers.52.mlp.experts.5.up_proj.weight": "model-00056-of-00101.safetensors", + "model.layers.52.mlp.experts.5.down_proj.weight": "model-00056-of-00101.safetensors", + "model.layers.52.mlp.experts.6.gate_proj.weight": "model-00056-of-00101.safetensors", + "model.layers.52.mlp.experts.6.up_proj.weight": "model-00056-of-00101.safetensors", + "model.layers.52.mlp.experts.6.down_proj.weight": "model-00056-of-00101.safetensors", + "model.layers.52.mlp.experts.7.gate_proj.weight": "model-00056-of-00101.safetensors", + "model.layers.52.mlp.experts.7.up_proj.weight": "model-00056-of-00101.safetensors", + "model.layers.52.mlp.experts.7.down_proj.weight": "model-00056-of-00101.safetensors", + "model.layers.52.mlp.experts.8.gate_proj.weight": "model-00056-of-00101.safetensors", + "model.layers.52.mlp.experts.8.up_proj.weight": "model-00056-of-00101.safetensors", + "model.layers.52.mlp.experts.8.down_proj.weight": "model-00056-of-00101.safetensors", + "model.layers.52.mlp.experts.9.gate_proj.weight": "model-00056-of-00101.safetensors", + "model.layers.52.mlp.experts.9.up_proj.weight": "model-00056-of-00101.safetensors", + "model.layers.52.mlp.experts.9.down_proj.weight": "model-00056-of-00101.safetensors", + "model.layers.52.mlp.experts.10.gate_proj.weight": "model-00056-of-00101.safetensors", + "model.layers.52.mlp.experts.10.up_proj.weight": "model-00056-of-00101.safetensors", + "model.layers.52.mlp.experts.10.down_proj.weight": "model-00056-of-00101.safetensors", + "model.layers.52.mlp.experts.11.gate_proj.weight": "model-00056-of-00101.safetensors", + "model.layers.52.mlp.experts.11.up_proj.weight": "model-00056-of-00101.safetensors", + "model.layers.52.mlp.experts.11.down_proj.weight": "model-00056-of-00101.safetensors", + "model.layers.52.mlp.experts.12.gate_proj.weight": "model-00056-of-00101.safetensors", + "model.layers.52.mlp.experts.12.up_proj.weight": "model-00056-of-00101.safetensors", + "model.layers.52.mlp.experts.12.down_proj.weight": "model-00056-of-00101.safetensors", + "model.layers.52.mlp.experts.13.gate_proj.weight": "model-00056-of-00101.safetensors", + "model.layers.52.mlp.experts.13.up_proj.weight": "model-00056-of-00101.safetensors", + "model.layers.52.mlp.experts.13.down_proj.weight": "model-00056-of-00101.safetensors", + "model.layers.52.mlp.experts.14.gate_proj.weight": "model-00056-of-00101.safetensors", + "model.layers.52.mlp.experts.14.up_proj.weight": "model-00056-of-00101.safetensors", + "model.layers.52.mlp.experts.14.down_proj.weight": "model-00056-of-00101.safetensors", + "model.layers.52.mlp.experts.15.gate_proj.weight": "model-00056-of-00101.safetensors", + "model.layers.52.mlp.experts.15.up_proj.weight": "model-00056-of-00101.safetensors", + "model.layers.52.mlp.experts.15.down_proj.weight": "model-00056-of-00101.safetensors", + "model.layers.52.mlp.experts.16.gate_proj.weight": "model-00056-of-00101.safetensors", + "model.layers.52.mlp.experts.16.up_proj.weight": "model-00056-of-00101.safetensors", + "model.layers.52.mlp.experts.16.down_proj.weight": "model-00056-of-00101.safetensors", + "model.layers.52.mlp.experts.17.gate_proj.weight": "model-00056-of-00101.safetensors", + "model.layers.52.mlp.experts.17.up_proj.weight": "model-00056-of-00101.safetensors", + "model.layers.52.mlp.experts.17.down_proj.weight": "model-00056-of-00101.safetensors", + "model.layers.52.mlp.experts.18.gate_proj.weight": "model-00056-of-00101.safetensors", + "model.layers.52.mlp.experts.18.up_proj.weight": "model-00056-of-00101.safetensors", + "model.layers.52.mlp.experts.18.down_proj.weight": "model-00056-of-00101.safetensors", + "model.layers.52.mlp.experts.19.gate_proj.weight": "model-00056-of-00101.safetensors", + "model.layers.52.mlp.experts.19.up_proj.weight": "model-00056-of-00101.safetensors", + "model.layers.52.mlp.experts.19.down_proj.weight": "model-00056-of-00101.safetensors", + "model.layers.52.mlp.experts.20.gate_proj.weight": "model-00056-of-00101.safetensors", + "model.layers.52.mlp.experts.20.up_proj.weight": "model-00056-of-00101.safetensors", + "model.layers.52.mlp.experts.20.down_proj.weight": "model-00056-of-00101.safetensors", + "model.layers.52.mlp.experts.21.gate_proj.weight": "model-00056-of-00101.safetensors", + "model.layers.52.mlp.experts.21.up_proj.weight": "model-00056-of-00101.safetensors", + "model.layers.52.mlp.experts.21.down_proj.weight": "model-00056-of-00101.safetensors", + "model.layers.52.mlp.experts.22.gate_proj.weight": "model-00056-of-00101.safetensors", + "model.layers.52.mlp.experts.22.up_proj.weight": "model-00056-of-00101.safetensors", + "model.layers.52.mlp.experts.22.down_proj.weight": "model-00056-of-00101.safetensors", + "model.layers.52.mlp.experts.23.gate_proj.weight": "model-00056-of-00101.safetensors", + "model.layers.52.mlp.experts.23.up_proj.weight": "model-00056-of-00101.safetensors", + "model.layers.52.mlp.experts.23.down_proj.weight": "model-00056-of-00101.safetensors", + "model.layers.52.mlp.experts.24.gate_proj.weight": "model-00056-of-00101.safetensors", + "model.layers.52.mlp.experts.24.up_proj.weight": "model-00056-of-00101.safetensors", + "model.layers.52.mlp.experts.24.down_proj.weight": "model-00056-of-00101.safetensors", + "model.layers.52.mlp.experts.25.gate_proj.weight": "model-00056-of-00101.safetensors", + "model.layers.52.mlp.experts.25.up_proj.weight": "model-00056-of-00101.safetensors", + "model.layers.52.mlp.experts.25.down_proj.weight": "model-00056-of-00101.safetensors", + "model.layers.52.mlp.experts.26.gate_proj.weight": "model-00056-of-00101.safetensors", + "model.layers.52.mlp.experts.26.up_proj.weight": "model-00056-of-00101.safetensors", + "model.layers.52.mlp.experts.26.down_proj.weight": "model-00056-of-00101.safetensors", + "model.layers.52.mlp.experts.27.gate_proj.weight": "model-00056-of-00101.safetensors", + "model.layers.52.mlp.experts.27.up_proj.weight": "model-00056-of-00101.safetensors", + "model.layers.52.mlp.experts.27.down_proj.weight": "model-00056-of-00101.safetensors", + "model.layers.52.mlp.experts.28.gate_proj.weight": "model-00056-of-00101.safetensors", + "model.layers.52.mlp.experts.28.up_proj.weight": "model-00056-of-00101.safetensors", + "model.layers.52.mlp.experts.28.down_proj.weight": "model-00056-of-00101.safetensors", + "model.layers.52.mlp.experts.29.gate_proj.weight": "model-00056-of-00101.safetensors", + "model.layers.52.mlp.experts.29.up_proj.weight": "model-00056-of-00101.safetensors", + "model.layers.52.mlp.experts.29.down_proj.weight": "model-00056-of-00101.safetensors", + "model.layers.52.mlp.experts.30.gate_proj.weight": "model-00056-of-00101.safetensors", + "model.layers.52.mlp.experts.30.up_proj.weight": "model-00056-of-00101.safetensors", + "model.layers.52.mlp.experts.30.down_proj.weight": "model-00056-of-00101.safetensors", + "model.layers.52.mlp.experts.31.gate_proj.weight": "model-00056-of-00101.safetensors", + "model.layers.52.mlp.experts.31.up_proj.weight": "model-00056-of-00101.safetensors", + "model.layers.52.mlp.experts.31.down_proj.weight": "model-00056-of-00101.safetensors", + "model.layers.52.mlp.experts.32.gate_proj.weight": "model-00056-of-00101.safetensors", + "model.layers.52.mlp.experts.32.up_proj.weight": "model-00056-of-00101.safetensors", + "model.layers.52.mlp.experts.32.down_proj.weight": "model-00056-of-00101.safetensors", + "model.layers.52.mlp.experts.33.gate_proj.weight": "model-00056-of-00101.safetensors", + "model.layers.52.mlp.experts.33.up_proj.weight": "model-00056-of-00101.safetensors", + "model.layers.52.mlp.experts.33.down_proj.weight": "model-00056-of-00101.safetensors", + "model.layers.52.mlp.experts.34.gate_proj.weight": "model-00056-of-00101.safetensors", + "model.layers.52.mlp.experts.34.up_proj.weight": "model-00056-of-00101.safetensors", + "model.layers.52.mlp.experts.34.down_proj.weight": "model-00056-of-00101.safetensors", + "model.layers.52.mlp.experts.35.gate_proj.weight": "model-00056-of-00101.safetensors", + "model.layers.52.mlp.experts.35.up_proj.weight": "model-00056-of-00101.safetensors", + "model.layers.52.mlp.experts.35.down_proj.weight": "model-00056-of-00101.safetensors", + "model.layers.52.mlp.experts.36.gate_proj.weight": "model-00056-of-00101.safetensors", + "model.layers.52.mlp.experts.36.up_proj.weight": "model-00056-of-00101.safetensors", + "model.layers.52.mlp.experts.36.down_proj.weight": "model-00056-of-00101.safetensors", + "model.layers.52.mlp.experts.37.gate_proj.weight": "model-00056-of-00101.safetensors", + "model.layers.52.mlp.experts.37.up_proj.weight": "model-00056-of-00101.safetensors", + "model.layers.52.mlp.experts.37.down_proj.weight": "model-00056-of-00101.safetensors", + "model.layers.52.mlp.experts.38.gate_proj.weight": "model-00056-of-00101.safetensors", + "model.layers.52.mlp.experts.38.up_proj.weight": "model-00056-of-00101.safetensors", + "model.layers.52.mlp.experts.38.down_proj.weight": "model-00056-of-00101.safetensors", + "model.layers.52.mlp.experts.39.gate_proj.weight": "model-00056-of-00101.safetensors", + "model.layers.52.mlp.experts.39.up_proj.weight": "model-00056-of-00101.safetensors", + "model.layers.52.mlp.experts.39.down_proj.weight": "model-00056-of-00101.safetensors", + "model.layers.52.mlp.experts.40.gate_proj.weight": "model-00056-of-00101.safetensors", + "model.layers.52.mlp.experts.40.up_proj.weight": "model-00056-of-00101.safetensors", + "model.layers.52.mlp.experts.40.down_proj.weight": "model-00056-of-00101.safetensors", + "model.layers.52.mlp.experts.41.gate_proj.weight": "model-00056-of-00101.safetensors", + "model.layers.52.mlp.experts.41.up_proj.weight": "model-00056-of-00101.safetensors", + "model.layers.52.mlp.experts.41.down_proj.weight": "model-00056-of-00101.safetensors", + "model.layers.52.mlp.experts.42.gate_proj.weight": "model-00056-of-00101.safetensors", + "model.layers.52.mlp.experts.42.up_proj.weight": "model-00056-of-00101.safetensors", + "model.layers.52.mlp.experts.42.down_proj.weight": "model-00056-of-00101.safetensors", + "model.layers.52.mlp.experts.43.gate_proj.weight": "model-00056-of-00101.safetensors", + "model.layers.52.mlp.experts.43.up_proj.weight": "model-00056-of-00101.safetensors", + "model.layers.52.mlp.experts.43.down_proj.weight": "model-00056-of-00101.safetensors", + "model.layers.52.mlp.experts.44.gate_proj.weight": "model-00056-of-00101.safetensors", + "model.layers.52.mlp.experts.44.up_proj.weight": "model-00056-of-00101.safetensors", + "model.layers.52.mlp.experts.44.down_proj.weight": "model-00056-of-00101.safetensors", + "model.layers.52.mlp.experts.45.gate_proj.weight": "model-00056-of-00101.safetensors", + "model.layers.52.mlp.experts.45.up_proj.weight": "model-00056-of-00101.safetensors", + "model.layers.52.mlp.experts.45.down_proj.weight": "model-00056-of-00101.safetensors", + "model.layers.52.mlp.experts.46.gate_proj.weight": "model-00056-of-00101.safetensors", + "model.layers.52.mlp.experts.46.up_proj.weight": "model-00056-of-00101.safetensors", + "model.layers.52.mlp.experts.46.down_proj.weight": "model-00056-of-00101.safetensors", + "model.layers.52.mlp.experts.47.gate_proj.weight": "model-00056-of-00101.safetensors", + "model.layers.52.mlp.experts.47.up_proj.weight": "model-00056-of-00101.safetensors", + "model.layers.52.mlp.experts.47.down_proj.weight": "model-00056-of-00101.safetensors", + "model.layers.52.mlp.experts.48.gate_proj.weight": "model-00056-of-00101.safetensors", + "model.layers.52.mlp.experts.48.up_proj.weight": "model-00056-of-00101.safetensors", + "model.layers.52.mlp.experts.48.down_proj.weight": "model-00056-of-00101.safetensors", + "model.layers.52.mlp.experts.49.gate_proj.weight": "model-00056-of-00101.safetensors", + "model.layers.52.mlp.experts.49.up_proj.weight": "model-00056-of-00101.safetensors", + "model.layers.52.mlp.experts.49.down_proj.weight": "model-00056-of-00101.safetensors", + "model.layers.52.mlp.experts.50.gate_proj.weight": "model-00056-of-00101.safetensors", + "model.layers.52.mlp.experts.50.up_proj.weight": "model-00056-of-00101.safetensors", + "model.layers.52.mlp.experts.50.down_proj.weight": "model-00056-of-00101.safetensors", + "model.layers.52.mlp.experts.51.gate_proj.weight": "model-00056-of-00101.safetensors", + "model.layers.52.mlp.experts.51.up_proj.weight": "model-00056-of-00101.safetensors", + "model.layers.52.mlp.experts.51.down_proj.weight": "model-00056-of-00101.safetensors", + "model.layers.52.mlp.experts.52.gate_proj.weight": "model-00056-of-00101.safetensors", + "model.layers.52.mlp.experts.52.up_proj.weight": "model-00056-of-00101.safetensors", + "model.layers.52.mlp.experts.52.down_proj.weight": "model-00056-of-00101.safetensors", + "model.layers.52.mlp.experts.53.gate_proj.weight": "model-00056-of-00101.safetensors", + "model.layers.52.mlp.experts.53.up_proj.weight": "model-00056-of-00101.safetensors", + "model.layers.52.mlp.experts.53.down_proj.weight": "model-00056-of-00101.safetensors", + "model.layers.52.mlp.experts.54.gate_proj.weight": "model-00056-of-00101.safetensors", + "model.layers.52.mlp.experts.54.up_proj.weight": "model-00056-of-00101.safetensors", + "model.layers.52.mlp.experts.54.down_proj.weight": "model-00056-of-00101.safetensors", + "model.layers.52.mlp.experts.55.gate_proj.weight": "model-00056-of-00101.safetensors", + "model.layers.52.mlp.experts.55.up_proj.weight": "model-00056-of-00101.safetensors", + "model.layers.52.mlp.experts.55.down_proj.weight": "model-00056-of-00101.safetensors", + "model.layers.52.mlp.experts.56.gate_proj.weight": "model-00056-of-00101.safetensors", + "model.layers.52.mlp.experts.56.up_proj.weight": "model-00056-of-00101.safetensors", + "model.layers.52.mlp.experts.56.down_proj.weight": "model-00056-of-00101.safetensors", + "model.layers.52.mlp.experts.57.gate_proj.weight": "model-00056-of-00101.safetensors", + "model.layers.52.mlp.experts.57.up_proj.weight": "model-00056-of-00101.safetensors", + "model.layers.52.mlp.experts.57.down_proj.weight": "model-00056-of-00101.safetensors", + "model.layers.52.mlp.experts.58.gate_proj.weight": "model-00056-of-00101.safetensors", + "model.layers.52.mlp.experts.58.up_proj.weight": "model-00056-of-00101.safetensors", + "model.layers.52.mlp.experts.58.down_proj.weight": "model-00056-of-00101.safetensors", + "model.layers.52.mlp.experts.59.gate_proj.weight": "model-00056-of-00101.safetensors", + "model.layers.52.mlp.experts.59.up_proj.weight": "model-00056-of-00101.safetensors", + "model.layers.52.mlp.experts.59.down_proj.weight": "model-00056-of-00101.safetensors", + "model.layers.52.mlp.experts.60.gate_proj.weight": "model-00056-of-00101.safetensors", + "model.layers.52.mlp.experts.60.up_proj.weight": "model-00056-of-00101.safetensors", + "model.layers.52.mlp.experts.60.down_proj.weight": "model-00056-of-00101.safetensors", + "model.layers.52.mlp.experts.61.gate_proj.weight": "model-00056-of-00101.safetensors", + "model.layers.52.mlp.experts.61.up_proj.weight": "model-00057-of-00101.safetensors", + "model.layers.52.mlp.experts.61.down_proj.weight": "model-00057-of-00101.safetensors", + "model.layers.52.mlp.experts.62.gate_proj.weight": "model-00057-of-00101.safetensors", + "model.layers.52.mlp.experts.62.up_proj.weight": "model-00057-of-00101.safetensors", + "model.layers.52.mlp.experts.62.down_proj.weight": "model-00057-of-00101.safetensors", + "model.layers.52.mlp.experts.63.gate_proj.weight": "model-00057-of-00101.safetensors", + "model.layers.52.mlp.experts.63.up_proj.weight": "model-00057-of-00101.safetensors", + "model.layers.52.mlp.experts.63.down_proj.weight": "model-00057-of-00101.safetensors", + "model.layers.52.mlp.experts.64.gate_proj.weight": "model-00057-of-00101.safetensors", + "model.layers.52.mlp.experts.64.up_proj.weight": "model-00057-of-00101.safetensors", + "model.layers.52.mlp.experts.64.down_proj.weight": "model-00057-of-00101.safetensors", + "model.layers.52.mlp.experts.65.gate_proj.weight": "model-00057-of-00101.safetensors", + "model.layers.52.mlp.experts.65.up_proj.weight": "model-00057-of-00101.safetensors", + "model.layers.52.mlp.experts.65.down_proj.weight": "model-00057-of-00101.safetensors", + "model.layers.52.mlp.experts.66.gate_proj.weight": "model-00057-of-00101.safetensors", + "model.layers.52.mlp.experts.66.up_proj.weight": "model-00057-of-00101.safetensors", + "model.layers.52.mlp.experts.66.down_proj.weight": "model-00057-of-00101.safetensors", + "model.layers.52.mlp.experts.67.gate_proj.weight": "model-00057-of-00101.safetensors", + "model.layers.52.mlp.experts.67.up_proj.weight": "model-00057-of-00101.safetensors", + "model.layers.52.mlp.experts.67.down_proj.weight": "model-00057-of-00101.safetensors", + "model.layers.52.mlp.experts.68.gate_proj.weight": "model-00057-of-00101.safetensors", + "model.layers.52.mlp.experts.68.up_proj.weight": "model-00057-of-00101.safetensors", + "model.layers.52.mlp.experts.68.down_proj.weight": "model-00057-of-00101.safetensors", + "model.layers.52.mlp.experts.69.gate_proj.weight": "model-00057-of-00101.safetensors", + "model.layers.52.mlp.experts.69.up_proj.weight": "model-00057-of-00101.safetensors", + "model.layers.52.mlp.experts.69.down_proj.weight": "model-00057-of-00101.safetensors", + "model.layers.52.mlp.experts.70.gate_proj.weight": "model-00057-of-00101.safetensors", + "model.layers.52.mlp.experts.70.up_proj.weight": "model-00057-of-00101.safetensors", + "model.layers.52.mlp.experts.70.down_proj.weight": "model-00057-of-00101.safetensors", + "model.layers.52.mlp.experts.71.gate_proj.weight": "model-00057-of-00101.safetensors", + "model.layers.52.mlp.experts.71.up_proj.weight": "model-00057-of-00101.safetensors", + "model.layers.52.mlp.experts.71.down_proj.weight": "model-00057-of-00101.safetensors", + "model.layers.52.mlp.experts.72.gate_proj.weight": "model-00057-of-00101.safetensors", + "model.layers.52.mlp.experts.72.up_proj.weight": "model-00057-of-00101.safetensors", + "model.layers.52.mlp.experts.72.down_proj.weight": "model-00057-of-00101.safetensors", + "model.layers.52.mlp.experts.73.gate_proj.weight": "model-00057-of-00101.safetensors", + "model.layers.52.mlp.experts.73.up_proj.weight": "model-00057-of-00101.safetensors", + "model.layers.52.mlp.experts.73.down_proj.weight": "model-00057-of-00101.safetensors", + "model.layers.52.mlp.experts.74.gate_proj.weight": "model-00057-of-00101.safetensors", + "model.layers.52.mlp.experts.74.up_proj.weight": "model-00057-of-00101.safetensors", + "model.layers.52.mlp.experts.74.down_proj.weight": "model-00057-of-00101.safetensors", + "model.layers.52.mlp.experts.75.gate_proj.weight": "model-00057-of-00101.safetensors", + "model.layers.52.mlp.experts.75.up_proj.weight": "model-00057-of-00101.safetensors", + "model.layers.52.mlp.experts.75.down_proj.weight": "model-00057-of-00101.safetensors", + "model.layers.52.mlp.experts.76.gate_proj.weight": "model-00057-of-00101.safetensors", + "model.layers.52.mlp.experts.76.up_proj.weight": "model-00057-of-00101.safetensors", + "model.layers.52.mlp.experts.76.down_proj.weight": "model-00057-of-00101.safetensors", + "model.layers.52.mlp.experts.77.gate_proj.weight": "model-00057-of-00101.safetensors", + "model.layers.52.mlp.experts.77.up_proj.weight": "model-00057-of-00101.safetensors", + "model.layers.52.mlp.experts.77.down_proj.weight": "model-00057-of-00101.safetensors", + "model.layers.52.mlp.experts.78.gate_proj.weight": "model-00057-of-00101.safetensors", + "model.layers.52.mlp.experts.78.up_proj.weight": "model-00057-of-00101.safetensors", + "model.layers.52.mlp.experts.78.down_proj.weight": "model-00057-of-00101.safetensors", + "model.layers.52.mlp.experts.79.gate_proj.weight": "model-00057-of-00101.safetensors", + "model.layers.52.mlp.experts.79.up_proj.weight": "model-00057-of-00101.safetensors", + "model.layers.52.mlp.experts.79.down_proj.weight": "model-00057-of-00101.safetensors", + "model.layers.52.mlp.experts.80.gate_proj.weight": "model-00057-of-00101.safetensors", + "model.layers.52.mlp.experts.80.up_proj.weight": "model-00057-of-00101.safetensors", + "model.layers.52.mlp.experts.80.down_proj.weight": "model-00057-of-00101.safetensors", + "model.layers.52.mlp.experts.81.gate_proj.weight": "model-00057-of-00101.safetensors", + "model.layers.52.mlp.experts.81.up_proj.weight": "model-00057-of-00101.safetensors", + "model.layers.52.mlp.experts.81.down_proj.weight": "model-00057-of-00101.safetensors", + "model.layers.52.mlp.experts.82.gate_proj.weight": "model-00057-of-00101.safetensors", + "model.layers.52.mlp.experts.82.up_proj.weight": "model-00057-of-00101.safetensors", + "model.layers.52.mlp.experts.82.down_proj.weight": "model-00057-of-00101.safetensors", + "model.layers.52.mlp.experts.83.gate_proj.weight": "model-00057-of-00101.safetensors", + "model.layers.52.mlp.experts.83.up_proj.weight": "model-00057-of-00101.safetensors", + "model.layers.52.mlp.experts.83.down_proj.weight": "model-00057-of-00101.safetensors", + "model.layers.52.mlp.experts.84.gate_proj.weight": "model-00057-of-00101.safetensors", + "model.layers.52.mlp.experts.84.up_proj.weight": "model-00057-of-00101.safetensors", + "model.layers.52.mlp.experts.84.down_proj.weight": "model-00057-of-00101.safetensors", + "model.layers.52.mlp.experts.85.gate_proj.weight": "model-00057-of-00101.safetensors", + "model.layers.52.mlp.experts.85.up_proj.weight": "model-00057-of-00101.safetensors", + "model.layers.52.mlp.experts.85.down_proj.weight": "model-00057-of-00101.safetensors", + "model.layers.52.mlp.experts.86.gate_proj.weight": "model-00057-of-00101.safetensors", + "model.layers.52.mlp.experts.86.up_proj.weight": "model-00057-of-00101.safetensors", + "model.layers.52.mlp.experts.86.down_proj.weight": "model-00057-of-00101.safetensors", + "model.layers.52.mlp.experts.87.gate_proj.weight": "model-00057-of-00101.safetensors", + "model.layers.52.mlp.experts.87.up_proj.weight": "model-00057-of-00101.safetensors", + "model.layers.52.mlp.experts.87.down_proj.weight": "model-00057-of-00101.safetensors", + "model.layers.52.mlp.experts.88.gate_proj.weight": "model-00057-of-00101.safetensors", + "model.layers.52.mlp.experts.88.up_proj.weight": "model-00057-of-00101.safetensors", + "model.layers.52.mlp.experts.88.down_proj.weight": "model-00057-of-00101.safetensors", + "model.layers.52.mlp.experts.89.gate_proj.weight": "model-00057-of-00101.safetensors", + "model.layers.52.mlp.experts.89.up_proj.weight": "model-00057-of-00101.safetensors", + "model.layers.52.mlp.experts.89.down_proj.weight": "model-00057-of-00101.safetensors", + "model.layers.52.mlp.experts.90.gate_proj.weight": "model-00057-of-00101.safetensors", + "model.layers.52.mlp.experts.90.up_proj.weight": "model-00057-of-00101.safetensors", + "model.layers.52.mlp.experts.90.down_proj.weight": "model-00057-of-00101.safetensors", + "model.layers.52.mlp.experts.91.gate_proj.weight": "model-00057-of-00101.safetensors", + "model.layers.52.mlp.experts.91.up_proj.weight": "model-00057-of-00101.safetensors", + "model.layers.52.mlp.experts.91.down_proj.weight": "model-00057-of-00101.safetensors", + "model.layers.52.mlp.experts.92.gate_proj.weight": "model-00057-of-00101.safetensors", + "model.layers.52.mlp.experts.92.up_proj.weight": "model-00057-of-00101.safetensors", + "model.layers.52.mlp.experts.92.down_proj.weight": "model-00057-of-00101.safetensors", + "model.layers.52.mlp.experts.93.gate_proj.weight": "model-00057-of-00101.safetensors", + "model.layers.52.mlp.experts.93.up_proj.weight": "model-00057-of-00101.safetensors", + "model.layers.52.mlp.experts.93.down_proj.weight": "model-00057-of-00101.safetensors", + "model.layers.52.mlp.experts.94.gate_proj.weight": "model-00057-of-00101.safetensors", + "model.layers.52.mlp.experts.94.up_proj.weight": "model-00057-of-00101.safetensors", + "model.layers.52.mlp.experts.94.down_proj.weight": "model-00057-of-00101.safetensors", + "model.layers.52.mlp.experts.95.gate_proj.weight": "model-00057-of-00101.safetensors", + "model.layers.52.mlp.experts.95.up_proj.weight": "model-00057-of-00101.safetensors", + "model.layers.52.mlp.experts.95.down_proj.weight": "model-00057-of-00101.safetensors", + "model.layers.52.mlp.experts.96.gate_proj.weight": "model-00057-of-00101.safetensors", + "model.layers.52.mlp.experts.96.up_proj.weight": "model-00057-of-00101.safetensors", + "model.layers.52.mlp.experts.96.down_proj.weight": "model-00057-of-00101.safetensors", + "model.layers.52.mlp.experts.97.gate_proj.weight": "model-00057-of-00101.safetensors", + "model.layers.52.mlp.experts.97.up_proj.weight": "model-00057-of-00101.safetensors", + "model.layers.52.mlp.experts.97.down_proj.weight": "model-00057-of-00101.safetensors", + "model.layers.52.mlp.experts.98.gate_proj.weight": "model-00057-of-00101.safetensors", + "model.layers.52.mlp.experts.98.up_proj.weight": "model-00057-of-00101.safetensors", + "model.layers.52.mlp.experts.98.down_proj.weight": "model-00057-of-00101.safetensors", + "model.layers.52.mlp.experts.99.gate_proj.weight": "model-00057-of-00101.safetensors", + "model.layers.52.mlp.experts.99.up_proj.weight": "model-00057-of-00101.safetensors", + "model.layers.52.mlp.experts.99.down_proj.weight": "model-00057-of-00101.safetensors", + "model.layers.52.mlp.experts.100.gate_proj.weight": "model-00057-of-00101.safetensors", + "model.layers.52.mlp.experts.100.up_proj.weight": "model-00057-of-00101.safetensors", + "model.layers.52.mlp.experts.100.down_proj.weight": "model-00057-of-00101.safetensors", + "model.layers.52.mlp.experts.101.gate_proj.weight": "model-00057-of-00101.safetensors", + "model.layers.52.mlp.experts.101.up_proj.weight": "model-00057-of-00101.safetensors", + "model.layers.52.mlp.experts.101.down_proj.weight": "model-00057-of-00101.safetensors", + "model.layers.52.mlp.experts.102.gate_proj.weight": "model-00057-of-00101.safetensors", + "model.layers.52.mlp.experts.102.up_proj.weight": "model-00057-of-00101.safetensors", + "model.layers.52.mlp.experts.102.down_proj.weight": "model-00057-of-00101.safetensors", + "model.layers.52.mlp.experts.103.gate_proj.weight": "model-00057-of-00101.safetensors", + "model.layers.52.mlp.experts.103.up_proj.weight": "model-00057-of-00101.safetensors", + "model.layers.52.mlp.experts.103.down_proj.weight": "model-00057-of-00101.safetensors", + "model.layers.52.mlp.experts.104.gate_proj.weight": "model-00057-of-00101.safetensors", + "model.layers.52.mlp.experts.104.up_proj.weight": "model-00057-of-00101.safetensors", + "model.layers.52.mlp.experts.104.down_proj.weight": "model-00057-of-00101.safetensors", + "model.layers.52.mlp.experts.105.gate_proj.weight": "model-00057-of-00101.safetensors", + "model.layers.52.mlp.experts.105.up_proj.weight": "model-00057-of-00101.safetensors", + "model.layers.52.mlp.experts.105.down_proj.weight": "model-00057-of-00101.safetensors", + "model.layers.52.mlp.experts.106.gate_proj.weight": "model-00057-of-00101.safetensors", + "model.layers.52.mlp.experts.106.up_proj.weight": "model-00057-of-00101.safetensors", + "model.layers.52.mlp.experts.106.down_proj.weight": "model-00057-of-00101.safetensors", + "model.layers.52.mlp.experts.107.gate_proj.weight": "model-00057-of-00101.safetensors", + "model.layers.52.mlp.experts.107.up_proj.weight": "model-00057-of-00101.safetensors", + "model.layers.52.mlp.experts.107.down_proj.weight": "model-00057-of-00101.safetensors", + "model.layers.52.mlp.experts.108.gate_proj.weight": "model-00057-of-00101.safetensors", + "model.layers.52.mlp.experts.108.up_proj.weight": "model-00057-of-00101.safetensors", + "model.layers.52.mlp.experts.108.down_proj.weight": "model-00057-of-00101.safetensors", + "model.layers.52.mlp.experts.109.gate_proj.weight": "model-00057-of-00101.safetensors", + "model.layers.52.mlp.experts.109.up_proj.weight": "model-00057-of-00101.safetensors", + "model.layers.52.mlp.experts.109.down_proj.weight": "model-00057-of-00101.safetensors", + "model.layers.52.mlp.experts.110.gate_proj.weight": "model-00057-of-00101.safetensors", + "model.layers.52.mlp.experts.110.up_proj.weight": "model-00057-of-00101.safetensors", + "model.layers.52.mlp.experts.110.down_proj.weight": "model-00057-of-00101.safetensors", + "model.layers.52.mlp.experts.111.gate_proj.weight": "model-00057-of-00101.safetensors", + "model.layers.52.mlp.experts.111.up_proj.weight": "model-00057-of-00101.safetensors", + "model.layers.52.mlp.experts.111.down_proj.weight": "model-00057-of-00101.safetensors", + "model.layers.52.mlp.experts.112.gate_proj.weight": "model-00057-of-00101.safetensors", + "model.layers.52.mlp.experts.112.up_proj.weight": "model-00057-of-00101.safetensors", + "model.layers.52.mlp.experts.112.down_proj.weight": "model-00057-of-00101.safetensors", + "model.layers.52.mlp.experts.113.gate_proj.weight": "model-00057-of-00101.safetensors", + "model.layers.52.mlp.experts.113.up_proj.weight": "model-00057-of-00101.safetensors", + "model.layers.52.mlp.experts.113.down_proj.weight": "model-00057-of-00101.safetensors", + "model.layers.52.mlp.experts.114.gate_proj.weight": "model-00057-of-00101.safetensors", + "model.layers.52.mlp.experts.114.up_proj.weight": "model-00057-of-00101.safetensors", + "model.layers.52.mlp.experts.114.down_proj.weight": "model-00057-of-00101.safetensors", + "model.layers.52.mlp.experts.115.gate_proj.weight": "model-00057-of-00101.safetensors", + "model.layers.52.mlp.experts.115.up_proj.weight": "model-00057-of-00101.safetensors", + "model.layers.52.mlp.experts.115.down_proj.weight": "model-00057-of-00101.safetensors", + "model.layers.52.mlp.experts.116.gate_proj.weight": "model-00057-of-00101.safetensors", + "model.layers.52.mlp.experts.116.up_proj.weight": "model-00057-of-00101.safetensors", + "model.layers.52.mlp.experts.116.down_proj.weight": "model-00057-of-00101.safetensors", + "model.layers.52.mlp.experts.117.gate_proj.weight": "model-00057-of-00101.safetensors", + "model.layers.52.mlp.experts.117.up_proj.weight": "model-00057-of-00101.safetensors", + "model.layers.52.mlp.experts.117.down_proj.weight": "model-00057-of-00101.safetensors", + "model.layers.52.mlp.experts.118.gate_proj.weight": "model-00057-of-00101.safetensors", + "model.layers.52.mlp.experts.118.up_proj.weight": "model-00057-of-00101.safetensors", + "model.layers.52.mlp.experts.118.down_proj.weight": "model-00057-of-00101.safetensors", + "model.layers.52.mlp.experts.119.gate_proj.weight": "model-00057-of-00101.safetensors", + "model.layers.52.mlp.experts.119.up_proj.weight": "model-00057-of-00101.safetensors", + "model.layers.52.mlp.experts.119.down_proj.weight": "model-00057-of-00101.safetensors", + "model.layers.52.mlp.gate.weight": "model-00057-of-00101.safetensors", + "model.layers.52.mlp.gate.e_score_correction_bias": "model-00057-of-00101.safetensors", + "model.layers.52.mlp.shared_experts.gate_proj.weight": "model-00057-of-00101.safetensors", + "model.layers.52.mlp.shared_experts.up_proj.weight": "model-00057-of-00101.safetensors", + "model.layers.52.mlp.shared_experts.down_proj.weight": "model-00057-of-00101.safetensors", + "model.layers.52.input_layernorm.weight": "model-00057-of-00101.safetensors", + "model.layers.52.post_attention_layernorm.weight": "model-00057-of-00101.safetensors", + "model.layers.53.self_attn.q_proj.weight": "model-00057-of-00101.safetensors", + "model.layers.53.self_attn.q_proj.bias": "model-00057-of-00101.safetensors", + "model.layers.53.self_attn.k_proj.weight": "model-00057-of-00101.safetensors", + "model.layers.53.self_attn.k_proj.bias": "model-00057-of-00101.safetensors", + "model.layers.53.self_attn.v_proj.weight": "model-00057-of-00101.safetensors", + "model.layers.53.self_attn.v_proj.bias": "model-00057-of-00101.safetensors", + "model.layers.53.self_attn.o_proj.weight": "model-00057-of-00101.safetensors", + "model.layers.53.self_attn.q_norm.weight": "model-00057-of-00101.safetensors", + "model.layers.53.self_attn.k_norm.weight": "model-00057-of-00101.safetensors", + "model.layers.53.mlp.experts.0.gate_proj.weight": "model-00057-of-00101.safetensors", + "model.layers.53.mlp.experts.0.up_proj.weight": "model-00057-of-00101.safetensors", + "model.layers.53.mlp.experts.0.down_proj.weight": "model-00057-of-00101.safetensors", + "model.layers.53.mlp.experts.1.gate_proj.weight": "model-00057-of-00101.safetensors", + "model.layers.53.mlp.experts.1.up_proj.weight": "model-00057-of-00101.safetensors", + "model.layers.53.mlp.experts.1.down_proj.weight": "model-00057-of-00101.safetensors", + "model.layers.53.mlp.experts.2.gate_proj.weight": "model-00057-of-00101.safetensors", + "model.layers.53.mlp.experts.2.up_proj.weight": "model-00057-of-00101.safetensors", + "model.layers.53.mlp.experts.2.down_proj.weight": "model-00057-of-00101.safetensors", + "model.layers.53.mlp.experts.3.gate_proj.weight": "model-00057-of-00101.safetensors", + "model.layers.53.mlp.experts.3.up_proj.weight": "model-00057-of-00101.safetensors", + "model.layers.53.mlp.experts.3.down_proj.weight": "model-00057-of-00101.safetensors", + "model.layers.53.mlp.experts.4.gate_proj.weight": "model-00057-of-00101.safetensors", + "model.layers.53.mlp.experts.4.up_proj.weight": "model-00057-of-00101.safetensors", + "model.layers.53.mlp.experts.4.down_proj.weight": "model-00057-of-00101.safetensors", + "model.layers.53.mlp.experts.5.gate_proj.weight": "model-00057-of-00101.safetensors", + "model.layers.53.mlp.experts.5.up_proj.weight": "model-00057-of-00101.safetensors", + "model.layers.53.mlp.experts.5.down_proj.weight": "model-00057-of-00101.safetensors", + "model.layers.53.mlp.experts.6.gate_proj.weight": "model-00057-of-00101.safetensors", + "model.layers.53.mlp.experts.6.up_proj.weight": "model-00057-of-00101.safetensors", + "model.layers.53.mlp.experts.6.down_proj.weight": "model-00057-of-00101.safetensors", + "model.layers.53.mlp.experts.7.gate_proj.weight": "model-00057-of-00101.safetensors", + "model.layers.53.mlp.experts.7.up_proj.weight": "model-00057-of-00101.safetensors", + "model.layers.53.mlp.experts.7.down_proj.weight": "model-00057-of-00101.safetensors", + "model.layers.53.mlp.experts.8.gate_proj.weight": "model-00057-of-00101.safetensors", + "model.layers.53.mlp.experts.8.up_proj.weight": "model-00057-of-00101.safetensors", + "model.layers.53.mlp.experts.8.down_proj.weight": "model-00057-of-00101.safetensors", + "model.layers.53.mlp.experts.9.gate_proj.weight": "model-00057-of-00101.safetensors", + "model.layers.53.mlp.experts.9.up_proj.weight": "model-00057-of-00101.safetensors", + "model.layers.53.mlp.experts.9.down_proj.weight": "model-00057-of-00101.safetensors", + "model.layers.53.mlp.experts.10.gate_proj.weight": "model-00057-of-00101.safetensors", + "model.layers.53.mlp.experts.10.up_proj.weight": "model-00057-of-00101.safetensors", + "model.layers.53.mlp.experts.10.down_proj.weight": "model-00057-of-00101.safetensors", + "model.layers.53.mlp.experts.11.gate_proj.weight": "model-00057-of-00101.safetensors", + "model.layers.53.mlp.experts.11.up_proj.weight": "model-00057-of-00101.safetensors", + "model.layers.53.mlp.experts.11.down_proj.weight": "model-00057-of-00101.safetensors", + "model.layers.53.mlp.experts.12.gate_proj.weight": "model-00057-of-00101.safetensors", + "model.layers.53.mlp.experts.12.up_proj.weight": "model-00057-of-00101.safetensors", + "model.layers.53.mlp.experts.12.down_proj.weight": "model-00057-of-00101.safetensors", + "model.layers.53.mlp.experts.13.gate_proj.weight": "model-00057-of-00101.safetensors", + "model.layers.53.mlp.experts.13.up_proj.weight": "model-00057-of-00101.safetensors", + "model.layers.53.mlp.experts.13.down_proj.weight": "model-00057-of-00101.safetensors", + "model.layers.53.mlp.experts.14.gate_proj.weight": "model-00057-of-00101.safetensors", + "model.layers.53.mlp.experts.14.up_proj.weight": "model-00057-of-00101.safetensors", + "model.layers.53.mlp.experts.14.down_proj.weight": "model-00057-of-00101.safetensors", + "model.layers.53.mlp.experts.15.gate_proj.weight": "model-00057-of-00101.safetensors", + "model.layers.53.mlp.experts.15.up_proj.weight": "model-00057-of-00101.safetensors", + "model.layers.53.mlp.experts.15.down_proj.weight": "model-00057-of-00101.safetensors", + "model.layers.53.mlp.experts.16.gate_proj.weight": "model-00057-of-00101.safetensors", + "model.layers.53.mlp.experts.16.up_proj.weight": "model-00057-of-00101.safetensors", + "model.layers.53.mlp.experts.16.down_proj.weight": "model-00057-of-00101.safetensors", + "model.layers.53.mlp.experts.17.gate_proj.weight": "model-00057-of-00101.safetensors", + "model.layers.53.mlp.experts.17.up_proj.weight": "model-00057-of-00101.safetensors", + "model.layers.53.mlp.experts.17.down_proj.weight": "model-00057-of-00101.safetensors", + "model.layers.53.mlp.experts.18.gate_proj.weight": "model-00057-of-00101.safetensors", + "model.layers.53.mlp.experts.18.up_proj.weight": "model-00057-of-00101.safetensors", + "model.layers.53.mlp.experts.18.down_proj.weight": "model-00057-of-00101.safetensors", + "model.layers.53.mlp.experts.19.gate_proj.weight": "model-00057-of-00101.safetensors", + "model.layers.53.mlp.experts.19.up_proj.weight": "model-00057-of-00101.safetensors", + "model.layers.53.mlp.experts.19.down_proj.weight": "model-00057-of-00101.safetensors", + "model.layers.53.mlp.experts.20.gate_proj.weight": "model-00057-of-00101.safetensors", + "model.layers.53.mlp.experts.20.up_proj.weight": "model-00057-of-00101.safetensors", + "model.layers.53.mlp.experts.20.down_proj.weight": "model-00057-of-00101.safetensors", + "model.layers.53.mlp.experts.21.gate_proj.weight": "model-00057-of-00101.safetensors", + "model.layers.53.mlp.experts.21.up_proj.weight": "model-00057-of-00101.safetensors", + "model.layers.53.mlp.experts.21.down_proj.weight": "model-00057-of-00101.safetensors", + "model.layers.53.mlp.experts.22.gate_proj.weight": "model-00057-of-00101.safetensors", + "model.layers.53.mlp.experts.22.up_proj.weight": "model-00057-of-00101.safetensors", + "model.layers.53.mlp.experts.22.down_proj.weight": "model-00057-of-00101.safetensors", + "model.layers.53.mlp.experts.23.gate_proj.weight": "model-00057-of-00101.safetensors", + "model.layers.53.mlp.experts.23.up_proj.weight": "model-00057-of-00101.safetensors", + "model.layers.53.mlp.experts.23.down_proj.weight": "model-00057-of-00101.safetensors", + "model.layers.53.mlp.experts.24.gate_proj.weight": "model-00057-of-00101.safetensors", + "model.layers.53.mlp.experts.24.up_proj.weight": "model-00057-of-00101.safetensors", + "model.layers.53.mlp.experts.24.down_proj.weight": "model-00057-of-00101.safetensors", + "model.layers.53.mlp.experts.25.gate_proj.weight": "model-00057-of-00101.safetensors", + "model.layers.53.mlp.experts.25.up_proj.weight": "model-00057-of-00101.safetensors", + "model.layers.53.mlp.experts.25.down_proj.weight": "model-00057-of-00101.safetensors", + "model.layers.53.mlp.experts.26.gate_proj.weight": "model-00057-of-00101.safetensors", + "model.layers.53.mlp.experts.26.up_proj.weight": "model-00057-of-00101.safetensors", + "model.layers.53.mlp.experts.26.down_proj.weight": "model-00057-of-00101.safetensors", + "model.layers.53.mlp.experts.27.gate_proj.weight": "model-00057-of-00101.safetensors", + "model.layers.53.mlp.experts.27.up_proj.weight": "model-00057-of-00101.safetensors", + "model.layers.53.mlp.experts.27.down_proj.weight": "model-00057-of-00101.safetensors", + "model.layers.53.mlp.experts.28.gate_proj.weight": "model-00057-of-00101.safetensors", + "model.layers.53.mlp.experts.28.up_proj.weight": "model-00057-of-00101.safetensors", + "model.layers.53.mlp.experts.28.down_proj.weight": "model-00057-of-00101.safetensors", + "model.layers.53.mlp.experts.29.gate_proj.weight": "model-00057-of-00101.safetensors", + "model.layers.53.mlp.experts.29.up_proj.weight": "model-00057-of-00101.safetensors", + "model.layers.53.mlp.experts.29.down_proj.weight": "model-00057-of-00101.safetensors", + "model.layers.53.mlp.experts.30.gate_proj.weight": "model-00057-of-00101.safetensors", + "model.layers.53.mlp.experts.30.up_proj.weight": "model-00057-of-00101.safetensors", + "model.layers.53.mlp.experts.30.down_proj.weight": "model-00057-of-00101.safetensors", + "model.layers.53.mlp.experts.31.gate_proj.weight": "model-00057-of-00101.safetensors", + "model.layers.53.mlp.experts.31.up_proj.weight": "model-00057-of-00101.safetensors", + "model.layers.53.mlp.experts.31.down_proj.weight": "model-00057-of-00101.safetensors", + "model.layers.53.mlp.experts.32.gate_proj.weight": "model-00057-of-00101.safetensors", + "model.layers.53.mlp.experts.32.up_proj.weight": "model-00057-of-00101.safetensors", + "model.layers.53.mlp.experts.32.down_proj.weight": "model-00057-of-00101.safetensors", + "model.layers.53.mlp.experts.33.gate_proj.weight": "model-00057-of-00101.safetensors", + "model.layers.53.mlp.experts.33.up_proj.weight": "model-00057-of-00101.safetensors", + "model.layers.53.mlp.experts.33.down_proj.weight": "model-00057-of-00101.safetensors", + "model.layers.53.mlp.experts.34.gate_proj.weight": "model-00057-of-00101.safetensors", + "model.layers.53.mlp.experts.34.up_proj.weight": "model-00057-of-00101.safetensors", + "model.layers.53.mlp.experts.34.down_proj.weight": "model-00057-of-00101.safetensors", + "model.layers.53.mlp.experts.35.gate_proj.weight": "model-00057-of-00101.safetensors", + "model.layers.53.mlp.experts.35.up_proj.weight": "model-00057-of-00101.safetensors", + "model.layers.53.mlp.experts.35.down_proj.weight": "model-00057-of-00101.safetensors", + "model.layers.53.mlp.experts.36.gate_proj.weight": "model-00057-of-00101.safetensors", + "model.layers.53.mlp.experts.36.up_proj.weight": "model-00057-of-00101.safetensors", + "model.layers.53.mlp.experts.36.down_proj.weight": "model-00057-of-00101.safetensors", + "model.layers.53.mlp.experts.37.gate_proj.weight": "model-00057-of-00101.safetensors", + "model.layers.53.mlp.experts.37.up_proj.weight": "model-00057-of-00101.safetensors", + "model.layers.53.mlp.experts.37.down_proj.weight": "model-00057-of-00101.safetensors", + "model.layers.53.mlp.experts.38.gate_proj.weight": "model-00057-of-00101.safetensors", + "model.layers.53.mlp.experts.38.up_proj.weight": "model-00057-of-00101.safetensors", + "model.layers.53.mlp.experts.38.down_proj.weight": "model-00057-of-00101.safetensors", + "model.layers.53.mlp.experts.39.gate_proj.weight": "model-00057-of-00101.safetensors", + "model.layers.53.mlp.experts.39.up_proj.weight": "model-00057-of-00101.safetensors", + "model.layers.53.mlp.experts.39.down_proj.weight": "model-00057-of-00101.safetensors", + "model.layers.53.mlp.experts.40.gate_proj.weight": "model-00057-of-00101.safetensors", + "model.layers.53.mlp.experts.40.up_proj.weight": "model-00057-of-00101.safetensors", + "model.layers.53.mlp.experts.40.down_proj.weight": "model-00057-of-00101.safetensors", + "model.layers.53.mlp.experts.41.gate_proj.weight": "model-00057-of-00101.safetensors", + "model.layers.53.mlp.experts.41.up_proj.weight": "model-00057-of-00101.safetensors", + "model.layers.53.mlp.experts.41.down_proj.weight": "model-00057-of-00101.safetensors", + "model.layers.53.mlp.experts.42.gate_proj.weight": "model-00057-of-00101.safetensors", + "model.layers.53.mlp.experts.42.up_proj.weight": "model-00057-of-00101.safetensors", + "model.layers.53.mlp.experts.42.down_proj.weight": "model-00057-of-00101.safetensors", + "model.layers.53.mlp.experts.43.gate_proj.weight": "model-00057-of-00101.safetensors", + "model.layers.53.mlp.experts.43.up_proj.weight": "model-00057-of-00101.safetensors", + "model.layers.53.mlp.experts.43.down_proj.weight": "model-00057-of-00101.safetensors", + "model.layers.53.mlp.experts.44.gate_proj.weight": "model-00057-of-00101.safetensors", + "model.layers.53.mlp.experts.44.up_proj.weight": "model-00057-of-00101.safetensors", + "model.layers.53.mlp.experts.44.down_proj.weight": "model-00057-of-00101.safetensors", + "model.layers.53.mlp.experts.45.gate_proj.weight": "model-00057-of-00101.safetensors", + "model.layers.53.mlp.experts.45.up_proj.weight": "model-00057-of-00101.safetensors", + "model.layers.53.mlp.experts.45.down_proj.weight": "model-00057-of-00101.safetensors", + "model.layers.53.mlp.experts.46.gate_proj.weight": "model-00057-of-00101.safetensors", + "model.layers.53.mlp.experts.46.up_proj.weight": "model-00057-of-00101.safetensors", + "model.layers.53.mlp.experts.46.down_proj.weight": "model-00057-of-00101.safetensors", + "model.layers.53.mlp.experts.47.gate_proj.weight": "model-00057-of-00101.safetensors", + "model.layers.53.mlp.experts.47.up_proj.weight": "model-00057-of-00101.safetensors", + "model.layers.53.mlp.experts.47.down_proj.weight": "model-00057-of-00101.safetensors", + "model.layers.53.mlp.experts.48.gate_proj.weight": "model-00058-of-00101.safetensors", + "model.layers.53.mlp.experts.48.up_proj.weight": "model-00058-of-00101.safetensors", + "model.layers.53.mlp.experts.48.down_proj.weight": "model-00058-of-00101.safetensors", + "model.layers.53.mlp.experts.49.gate_proj.weight": "model-00058-of-00101.safetensors", + "model.layers.53.mlp.experts.49.up_proj.weight": "model-00058-of-00101.safetensors", + "model.layers.53.mlp.experts.49.down_proj.weight": "model-00058-of-00101.safetensors", + "model.layers.53.mlp.experts.50.gate_proj.weight": "model-00058-of-00101.safetensors", + "model.layers.53.mlp.experts.50.up_proj.weight": "model-00058-of-00101.safetensors", + "model.layers.53.mlp.experts.50.down_proj.weight": "model-00058-of-00101.safetensors", + "model.layers.53.mlp.experts.51.gate_proj.weight": "model-00058-of-00101.safetensors", + "model.layers.53.mlp.experts.51.up_proj.weight": "model-00058-of-00101.safetensors", + "model.layers.53.mlp.experts.51.down_proj.weight": "model-00058-of-00101.safetensors", + "model.layers.53.mlp.experts.52.gate_proj.weight": "model-00058-of-00101.safetensors", + "model.layers.53.mlp.experts.52.up_proj.weight": "model-00058-of-00101.safetensors", + "model.layers.53.mlp.experts.52.down_proj.weight": "model-00058-of-00101.safetensors", + "model.layers.53.mlp.experts.53.gate_proj.weight": "model-00058-of-00101.safetensors", + "model.layers.53.mlp.experts.53.up_proj.weight": "model-00058-of-00101.safetensors", + "model.layers.53.mlp.experts.53.down_proj.weight": "model-00058-of-00101.safetensors", + "model.layers.53.mlp.experts.54.gate_proj.weight": "model-00058-of-00101.safetensors", + "model.layers.53.mlp.experts.54.up_proj.weight": "model-00058-of-00101.safetensors", + "model.layers.53.mlp.experts.54.down_proj.weight": "model-00058-of-00101.safetensors", + "model.layers.53.mlp.experts.55.gate_proj.weight": "model-00058-of-00101.safetensors", + "model.layers.53.mlp.experts.55.up_proj.weight": "model-00058-of-00101.safetensors", + "model.layers.53.mlp.experts.55.down_proj.weight": "model-00058-of-00101.safetensors", + "model.layers.53.mlp.experts.56.gate_proj.weight": "model-00058-of-00101.safetensors", + "model.layers.53.mlp.experts.56.up_proj.weight": "model-00058-of-00101.safetensors", + "model.layers.53.mlp.experts.56.down_proj.weight": "model-00058-of-00101.safetensors", + "model.layers.53.mlp.experts.57.gate_proj.weight": "model-00058-of-00101.safetensors", + "model.layers.53.mlp.experts.57.up_proj.weight": "model-00058-of-00101.safetensors", + "model.layers.53.mlp.experts.57.down_proj.weight": "model-00058-of-00101.safetensors", + "model.layers.53.mlp.experts.58.gate_proj.weight": "model-00058-of-00101.safetensors", + "model.layers.53.mlp.experts.58.up_proj.weight": "model-00058-of-00101.safetensors", + "model.layers.53.mlp.experts.58.down_proj.weight": "model-00058-of-00101.safetensors", + "model.layers.53.mlp.experts.59.gate_proj.weight": "model-00058-of-00101.safetensors", + "model.layers.53.mlp.experts.59.up_proj.weight": "model-00058-of-00101.safetensors", + "model.layers.53.mlp.experts.59.down_proj.weight": "model-00058-of-00101.safetensors", + "model.layers.53.mlp.experts.60.gate_proj.weight": "model-00058-of-00101.safetensors", + "model.layers.53.mlp.experts.60.up_proj.weight": "model-00058-of-00101.safetensors", + "model.layers.53.mlp.experts.60.down_proj.weight": "model-00058-of-00101.safetensors", + "model.layers.53.mlp.experts.61.gate_proj.weight": "model-00058-of-00101.safetensors", + "model.layers.53.mlp.experts.61.up_proj.weight": "model-00058-of-00101.safetensors", + "model.layers.53.mlp.experts.61.down_proj.weight": "model-00058-of-00101.safetensors", + "model.layers.53.mlp.experts.62.gate_proj.weight": "model-00058-of-00101.safetensors", + "model.layers.53.mlp.experts.62.up_proj.weight": "model-00058-of-00101.safetensors", + "model.layers.53.mlp.experts.62.down_proj.weight": "model-00058-of-00101.safetensors", + "model.layers.53.mlp.experts.63.gate_proj.weight": "model-00058-of-00101.safetensors", + "model.layers.53.mlp.experts.63.up_proj.weight": "model-00058-of-00101.safetensors", + "model.layers.53.mlp.experts.63.down_proj.weight": "model-00058-of-00101.safetensors", + "model.layers.53.mlp.experts.64.gate_proj.weight": "model-00058-of-00101.safetensors", + "model.layers.53.mlp.experts.64.up_proj.weight": "model-00058-of-00101.safetensors", + "model.layers.53.mlp.experts.64.down_proj.weight": "model-00058-of-00101.safetensors", + "model.layers.53.mlp.experts.65.gate_proj.weight": "model-00058-of-00101.safetensors", + "model.layers.53.mlp.experts.65.up_proj.weight": "model-00058-of-00101.safetensors", + "model.layers.53.mlp.experts.65.down_proj.weight": "model-00058-of-00101.safetensors", + "model.layers.53.mlp.experts.66.gate_proj.weight": "model-00058-of-00101.safetensors", + "model.layers.53.mlp.experts.66.up_proj.weight": "model-00058-of-00101.safetensors", + "model.layers.53.mlp.experts.66.down_proj.weight": "model-00058-of-00101.safetensors", + "model.layers.53.mlp.experts.67.gate_proj.weight": "model-00058-of-00101.safetensors", + "model.layers.53.mlp.experts.67.up_proj.weight": "model-00058-of-00101.safetensors", + "model.layers.53.mlp.experts.67.down_proj.weight": "model-00058-of-00101.safetensors", + "model.layers.53.mlp.experts.68.gate_proj.weight": "model-00058-of-00101.safetensors", + "model.layers.53.mlp.experts.68.up_proj.weight": "model-00058-of-00101.safetensors", + "model.layers.53.mlp.experts.68.down_proj.weight": "model-00058-of-00101.safetensors", + "model.layers.53.mlp.experts.69.gate_proj.weight": "model-00058-of-00101.safetensors", + "model.layers.53.mlp.experts.69.up_proj.weight": "model-00058-of-00101.safetensors", + "model.layers.53.mlp.experts.69.down_proj.weight": "model-00058-of-00101.safetensors", + "model.layers.53.mlp.experts.70.gate_proj.weight": "model-00058-of-00101.safetensors", + "model.layers.53.mlp.experts.70.up_proj.weight": "model-00058-of-00101.safetensors", + "model.layers.53.mlp.experts.70.down_proj.weight": "model-00058-of-00101.safetensors", + "model.layers.53.mlp.experts.71.gate_proj.weight": "model-00058-of-00101.safetensors", + "model.layers.53.mlp.experts.71.up_proj.weight": "model-00058-of-00101.safetensors", + "model.layers.53.mlp.experts.71.down_proj.weight": "model-00058-of-00101.safetensors", + "model.layers.53.mlp.experts.72.gate_proj.weight": "model-00058-of-00101.safetensors", + "model.layers.53.mlp.experts.72.up_proj.weight": "model-00058-of-00101.safetensors", + "model.layers.53.mlp.experts.72.down_proj.weight": "model-00058-of-00101.safetensors", + "model.layers.53.mlp.experts.73.gate_proj.weight": "model-00058-of-00101.safetensors", + "model.layers.53.mlp.experts.73.up_proj.weight": "model-00058-of-00101.safetensors", + "model.layers.53.mlp.experts.73.down_proj.weight": "model-00058-of-00101.safetensors", + "model.layers.53.mlp.experts.74.gate_proj.weight": "model-00058-of-00101.safetensors", + "model.layers.53.mlp.experts.74.up_proj.weight": "model-00058-of-00101.safetensors", + "model.layers.53.mlp.experts.74.down_proj.weight": "model-00058-of-00101.safetensors", + "model.layers.53.mlp.experts.75.gate_proj.weight": "model-00058-of-00101.safetensors", + "model.layers.53.mlp.experts.75.up_proj.weight": "model-00058-of-00101.safetensors", + "model.layers.53.mlp.experts.75.down_proj.weight": "model-00058-of-00101.safetensors", + "model.layers.53.mlp.experts.76.gate_proj.weight": "model-00058-of-00101.safetensors", + "model.layers.53.mlp.experts.76.up_proj.weight": "model-00058-of-00101.safetensors", + "model.layers.53.mlp.experts.76.down_proj.weight": "model-00058-of-00101.safetensors", + "model.layers.53.mlp.experts.77.gate_proj.weight": "model-00058-of-00101.safetensors", + "model.layers.53.mlp.experts.77.up_proj.weight": "model-00058-of-00101.safetensors", + "model.layers.53.mlp.experts.77.down_proj.weight": "model-00058-of-00101.safetensors", + "model.layers.53.mlp.experts.78.gate_proj.weight": "model-00058-of-00101.safetensors", + "model.layers.53.mlp.experts.78.up_proj.weight": "model-00058-of-00101.safetensors", + "model.layers.53.mlp.experts.78.down_proj.weight": "model-00058-of-00101.safetensors", + "model.layers.53.mlp.experts.79.gate_proj.weight": "model-00058-of-00101.safetensors", + "model.layers.53.mlp.experts.79.up_proj.weight": "model-00058-of-00101.safetensors", + "model.layers.53.mlp.experts.79.down_proj.weight": "model-00058-of-00101.safetensors", + "model.layers.53.mlp.experts.80.gate_proj.weight": "model-00058-of-00101.safetensors", + "model.layers.53.mlp.experts.80.up_proj.weight": "model-00058-of-00101.safetensors", + "model.layers.53.mlp.experts.80.down_proj.weight": "model-00058-of-00101.safetensors", + "model.layers.53.mlp.experts.81.gate_proj.weight": "model-00058-of-00101.safetensors", + "model.layers.53.mlp.experts.81.up_proj.weight": "model-00058-of-00101.safetensors", + "model.layers.53.mlp.experts.81.down_proj.weight": "model-00058-of-00101.safetensors", + "model.layers.53.mlp.experts.82.gate_proj.weight": "model-00058-of-00101.safetensors", + "model.layers.53.mlp.experts.82.up_proj.weight": "model-00058-of-00101.safetensors", + "model.layers.53.mlp.experts.82.down_proj.weight": "model-00058-of-00101.safetensors", + "model.layers.53.mlp.experts.83.gate_proj.weight": "model-00058-of-00101.safetensors", + "model.layers.53.mlp.experts.83.up_proj.weight": "model-00058-of-00101.safetensors", + "model.layers.53.mlp.experts.83.down_proj.weight": "model-00058-of-00101.safetensors", + "model.layers.53.mlp.experts.84.gate_proj.weight": "model-00058-of-00101.safetensors", + "model.layers.53.mlp.experts.84.up_proj.weight": "model-00058-of-00101.safetensors", + "model.layers.53.mlp.experts.84.down_proj.weight": "model-00058-of-00101.safetensors", + "model.layers.53.mlp.experts.85.gate_proj.weight": "model-00058-of-00101.safetensors", + "model.layers.53.mlp.experts.85.up_proj.weight": "model-00058-of-00101.safetensors", + "model.layers.53.mlp.experts.85.down_proj.weight": "model-00058-of-00101.safetensors", + "model.layers.53.mlp.experts.86.gate_proj.weight": "model-00058-of-00101.safetensors", + "model.layers.53.mlp.experts.86.up_proj.weight": "model-00058-of-00101.safetensors", + "model.layers.53.mlp.experts.86.down_proj.weight": "model-00058-of-00101.safetensors", + "model.layers.53.mlp.experts.87.gate_proj.weight": "model-00058-of-00101.safetensors", + "model.layers.53.mlp.experts.87.up_proj.weight": "model-00058-of-00101.safetensors", + "model.layers.53.mlp.experts.87.down_proj.weight": "model-00058-of-00101.safetensors", + "model.layers.53.mlp.experts.88.gate_proj.weight": "model-00058-of-00101.safetensors", + "model.layers.53.mlp.experts.88.up_proj.weight": "model-00058-of-00101.safetensors", + "model.layers.53.mlp.experts.88.down_proj.weight": "model-00058-of-00101.safetensors", + "model.layers.53.mlp.experts.89.gate_proj.weight": "model-00058-of-00101.safetensors", + "model.layers.53.mlp.experts.89.up_proj.weight": "model-00058-of-00101.safetensors", + "model.layers.53.mlp.experts.89.down_proj.weight": "model-00058-of-00101.safetensors", + "model.layers.53.mlp.experts.90.gate_proj.weight": "model-00058-of-00101.safetensors", + "model.layers.53.mlp.experts.90.up_proj.weight": "model-00058-of-00101.safetensors", + "model.layers.53.mlp.experts.90.down_proj.weight": "model-00058-of-00101.safetensors", + "model.layers.53.mlp.experts.91.gate_proj.weight": "model-00058-of-00101.safetensors", + "model.layers.53.mlp.experts.91.up_proj.weight": "model-00058-of-00101.safetensors", + "model.layers.53.mlp.experts.91.down_proj.weight": "model-00058-of-00101.safetensors", + "model.layers.53.mlp.experts.92.gate_proj.weight": "model-00058-of-00101.safetensors", + "model.layers.53.mlp.experts.92.up_proj.weight": "model-00058-of-00101.safetensors", + "model.layers.53.mlp.experts.92.down_proj.weight": "model-00058-of-00101.safetensors", + "model.layers.53.mlp.experts.93.gate_proj.weight": "model-00058-of-00101.safetensors", + "model.layers.53.mlp.experts.93.up_proj.weight": "model-00058-of-00101.safetensors", + "model.layers.53.mlp.experts.93.down_proj.weight": "model-00058-of-00101.safetensors", + "model.layers.53.mlp.experts.94.gate_proj.weight": "model-00058-of-00101.safetensors", + "model.layers.53.mlp.experts.94.up_proj.weight": "model-00058-of-00101.safetensors", + "model.layers.53.mlp.experts.94.down_proj.weight": "model-00058-of-00101.safetensors", + "model.layers.53.mlp.experts.95.gate_proj.weight": "model-00058-of-00101.safetensors", + "model.layers.53.mlp.experts.95.up_proj.weight": "model-00058-of-00101.safetensors", + "model.layers.53.mlp.experts.95.down_proj.weight": "model-00058-of-00101.safetensors", + "model.layers.53.mlp.experts.96.gate_proj.weight": "model-00058-of-00101.safetensors", + "model.layers.53.mlp.experts.96.up_proj.weight": "model-00058-of-00101.safetensors", + "model.layers.53.mlp.experts.96.down_proj.weight": "model-00058-of-00101.safetensors", + "model.layers.53.mlp.experts.97.gate_proj.weight": "model-00058-of-00101.safetensors", + "model.layers.53.mlp.experts.97.up_proj.weight": "model-00058-of-00101.safetensors", + "model.layers.53.mlp.experts.97.down_proj.weight": "model-00058-of-00101.safetensors", + "model.layers.53.mlp.experts.98.gate_proj.weight": "model-00058-of-00101.safetensors", + "model.layers.53.mlp.experts.98.up_proj.weight": "model-00058-of-00101.safetensors", + "model.layers.53.mlp.experts.98.down_proj.weight": "model-00058-of-00101.safetensors", + "model.layers.53.mlp.experts.99.gate_proj.weight": "model-00058-of-00101.safetensors", + "model.layers.53.mlp.experts.99.up_proj.weight": "model-00058-of-00101.safetensors", + "model.layers.53.mlp.experts.99.down_proj.weight": "model-00058-of-00101.safetensors", + "model.layers.53.mlp.experts.100.gate_proj.weight": "model-00058-of-00101.safetensors", + "model.layers.53.mlp.experts.100.up_proj.weight": "model-00058-of-00101.safetensors", + "model.layers.53.mlp.experts.100.down_proj.weight": "model-00058-of-00101.safetensors", + "model.layers.53.mlp.experts.101.gate_proj.weight": "model-00058-of-00101.safetensors", + "model.layers.53.mlp.experts.101.up_proj.weight": "model-00058-of-00101.safetensors", + "model.layers.53.mlp.experts.101.down_proj.weight": "model-00058-of-00101.safetensors", + "model.layers.53.mlp.experts.102.gate_proj.weight": "model-00058-of-00101.safetensors", + "model.layers.53.mlp.experts.102.up_proj.weight": "model-00058-of-00101.safetensors", + "model.layers.53.mlp.experts.102.down_proj.weight": "model-00058-of-00101.safetensors", + "model.layers.53.mlp.experts.103.gate_proj.weight": "model-00058-of-00101.safetensors", + "model.layers.53.mlp.experts.103.up_proj.weight": "model-00058-of-00101.safetensors", + "model.layers.53.mlp.experts.103.down_proj.weight": "model-00058-of-00101.safetensors", + "model.layers.53.mlp.experts.104.gate_proj.weight": "model-00058-of-00101.safetensors", + "model.layers.53.mlp.experts.104.up_proj.weight": "model-00058-of-00101.safetensors", + "model.layers.53.mlp.experts.104.down_proj.weight": "model-00058-of-00101.safetensors", + "model.layers.53.mlp.experts.105.gate_proj.weight": "model-00058-of-00101.safetensors", + "model.layers.53.mlp.experts.105.up_proj.weight": "model-00058-of-00101.safetensors", + "model.layers.53.mlp.experts.105.down_proj.weight": "model-00058-of-00101.safetensors", + "model.layers.53.mlp.experts.106.gate_proj.weight": "model-00058-of-00101.safetensors", + "model.layers.53.mlp.experts.106.up_proj.weight": "model-00058-of-00101.safetensors", + "model.layers.53.mlp.experts.106.down_proj.weight": "model-00058-of-00101.safetensors", + "model.layers.53.mlp.experts.107.gate_proj.weight": "model-00058-of-00101.safetensors", + "model.layers.53.mlp.experts.107.up_proj.weight": "model-00058-of-00101.safetensors", + "model.layers.53.mlp.experts.107.down_proj.weight": "model-00058-of-00101.safetensors", + "model.layers.53.mlp.experts.108.gate_proj.weight": "model-00058-of-00101.safetensors", + "model.layers.53.mlp.experts.108.up_proj.weight": "model-00058-of-00101.safetensors", + "model.layers.53.mlp.experts.108.down_proj.weight": "model-00058-of-00101.safetensors", + "model.layers.53.mlp.experts.109.gate_proj.weight": "model-00058-of-00101.safetensors", + "model.layers.53.mlp.experts.109.up_proj.weight": "model-00058-of-00101.safetensors", + "model.layers.53.mlp.experts.109.down_proj.weight": "model-00058-of-00101.safetensors", + "model.layers.53.mlp.experts.110.gate_proj.weight": "model-00058-of-00101.safetensors", + "model.layers.53.mlp.experts.110.up_proj.weight": "model-00058-of-00101.safetensors", + "model.layers.53.mlp.experts.110.down_proj.weight": "model-00058-of-00101.safetensors", + "model.layers.53.mlp.experts.111.gate_proj.weight": "model-00058-of-00101.safetensors", + "model.layers.53.mlp.experts.111.up_proj.weight": "model-00058-of-00101.safetensors", + "model.layers.53.mlp.experts.111.down_proj.weight": "model-00058-of-00101.safetensors", + "model.layers.53.mlp.experts.112.gate_proj.weight": "model-00058-of-00101.safetensors", + "model.layers.53.mlp.experts.112.up_proj.weight": "model-00058-of-00101.safetensors", + "model.layers.53.mlp.experts.112.down_proj.weight": "model-00058-of-00101.safetensors", + "model.layers.53.mlp.experts.113.gate_proj.weight": "model-00058-of-00101.safetensors", + "model.layers.53.mlp.experts.113.up_proj.weight": "model-00058-of-00101.safetensors", + "model.layers.53.mlp.experts.113.down_proj.weight": "model-00058-of-00101.safetensors", + "model.layers.53.mlp.experts.114.gate_proj.weight": "model-00058-of-00101.safetensors", + "model.layers.53.mlp.experts.114.up_proj.weight": "model-00058-of-00101.safetensors", + "model.layers.53.mlp.experts.114.down_proj.weight": "model-00058-of-00101.safetensors", + "model.layers.53.mlp.experts.115.gate_proj.weight": "model-00058-of-00101.safetensors", + "model.layers.53.mlp.experts.115.up_proj.weight": "model-00058-of-00101.safetensors", + "model.layers.53.mlp.experts.115.down_proj.weight": "model-00058-of-00101.safetensors", + "model.layers.53.mlp.experts.116.gate_proj.weight": "model-00058-of-00101.safetensors", + "model.layers.53.mlp.experts.116.up_proj.weight": "model-00058-of-00101.safetensors", + "model.layers.53.mlp.experts.116.down_proj.weight": "model-00058-of-00101.safetensors", + "model.layers.53.mlp.experts.117.gate_proj.weight": "model-00058-of-00101.safetensors", + "model.layers.53.mlp.experts.117.up_proj.weight": "model-00058-of-00101.safetensors", + "model.layers.53.mlp.experts.117.down_proj.weight": "model-00058-of-00101.safetensors", + "model.layers.53.mlp.experts.118.gate_proj.weight": "model-00058-of-00101.safetensors", + "model.layers.53.mlp.experts.118.up_proj.weight": "model-00058-of-00101.safetensors", + "model.layers.53.mlp.experts.118.down_proj.weight": "model-00058-of-00101.safetensors", + "model.layers.53.mlp.experts.119.gate_proj.weight": "model-00058-of-00101.safetensors", + "model.layers.53.mlp.experts.119.up_proj.weight": "model-00058-of-00101.safetensors", + "model.layers.53.mlp.experts.119.down_proj.weight": "model-00058-of-00101.safetensors", + "model.layers.53.mlp.gate.weight": "model-00058-of-00101.safetensors", + "model.layers.53.mlp.gate.e_score_correction_bias": "model-00058-of-00101.safetensors", + "model.layers.53.mlp.shared_experts.gate_proj.weight": "model-00058-of-00101.safetensors", + "model.layers.53.mlp.shared_experts.up_proj.weight": "model-00058-of-00101.safetensors", + "model.layers.53.mlp.shared_experts.down_proj.weight": "model-00058-of-00101.safetensors", + "model.layers.53.input_layernorm.weight": "model-00058-of-00101.safetensors", + "model.layers.53.post_attention_layernorm.weight": "model-00058-of-00101.safetensors", + "model.layers.54.self_attn.q_proj.weight": "model-00058-of-00101.safetensors", + "model.layers.54.self_attn.q_proj.bias": "model-00058-of-00101.safetensors", + "model.layers.54.self_attn.k_proj.weight": "model-00058-of-00101.safetensors", + "model.layers.54.self_attn.k_proj.bias": "model-00058-of-00101.safetensors", + "model.layers.54.self_attn.v_proj.weight": "model-00058-of-00101.safetensors", + "model.layers.54.self_attn.v_proj.bias": "model-00058-of-00101.safetensors", + "model.layers.54.self_attn.o_proj.weight": "model-00058-of-00101.safetensors", + "model.layers.54.self_attn.q_norm.weight": "model-00058-of-00101.safetensors", + "model.layers.54.self_attn.k_norm.weight": "model-00058-of-00101.safetensors", + "model.layers.54.mlp.experts.0.gate_proj.weight": "model-00058-of-00101.safetensors", + "model.layers.54.mlp.experts.0.up_proj.weight": "model-00058-of-00101.safetensors", + "model.layers.54.mlp.experts.0.down_proj.weight": "model-00058-of-00101.safetensors", + "model.layers.54.mlp.experts.1.gate_proj.weight": "model-00058-of-00101.safetensors", + "model.layers.54.mlp.experts.1.up_proj.weight": "model-00058-of-00101.safetensors", + "model.layers.54.mlp.experts.1.down_proj.weight": "model-00058-of-00101.safetensors", + "model.layers.54.mlp.experts.2.gate_proj.weight": "model-00058-of-00101.safetensors", + "model.layers.54.mlp.experts.2.up_proj.weight": "model-00058-of-00101.safetensors", + "model.layers.54.mlp.experts.2.down_proj.weight": "model-00058-of-00101.safetensors", + "model.layers.54.mlp.experts.3.gate_proj.weight": "model-00058-of-00101.safetensors", + "model.layers.54.mlp.experts.3.up_proj.weight": "model-00058-of-00101.safetensors", + "model.layers.54.mlp.experts.3.down_proj.weight": "model-00058-of-00101.safetensors", + "model.layers.54.mlp.experts.4.gate_proj.weight": "model-00058-of-00101.safetensors", + "model.layers.54.mlp.experts.4.up_proj.weight": "model-00058-of-00101.safetensors", + "model.layers.54.mlp.experts.4.down_proj.weight": "model-00058-of-00101.safetensors", + "model.layers.54.mlp.experts.5.gate_proj.weight": "model-00058-of-00101.safetensors", + "model.layers.54.mlp.experts.5.up_proj.weight": "model-00058-of-00101.safetensors", + "model.layers.54.mlp.experts.5.down_proj.weight": "model-00058-of-00101.safetensors", + "model.layers.54.mlp.experts.6.gate_proj.weight": "model-00058-of-00101.safetensors", + "model.layers.54.mlp.experts.6.up_proj.weight": "model-00058-of-00101.safetensors", + "model.layers.54.mlp.experts.6.down_proj.weight": "model-00058-of-00101.safetensors", + "model.layers.54.mlp.experts.7.gate_proj.weight": "model-00058-of-00101.safetensors", + "model.layers.54.mlp.experts.7.up_proj.weight": "model-00058-of-00101.safetensors", + "model.layers.54.mlp.experts.7.down_proj.weight": "model-00058-of-00101.safetensors", + "model.layers.54.mlp.experts.8.gate_proj.weight": "model-00058-of-00101.safetensors", + "model.layers.54.mlp.experts.8.up_proj.weight": "model-00058-of-00101.safetensors", + "model.layers.54.mlp.experts.8.down_proj.weight": "model-00058-of-00101.safetensors", + "model.layers.54.mlp.experts.9.gate_proj.weight": "model-00058-of-00101.safetensors", + "model.layers.54.mlp.experts.9.up_proj.weight": "model-00058-of-00101.safetensors", + "model.layers.54.mlp.experts.9.down_proj.weight": "model-00058-of-00101.safetensors", + "model.layers.54.mlp.experts.10.gate_proj.weight": "model-00058-of-00101.safetensors", + "model.layers.54.mlp.experts.10.up_proj.weight": "model-00058-of-00101.safetensors", + "model.layers.54.mlp.experts.10.down_proj.weight": "model-00058-of-00101.safetensors", + "model.layers.54.mlp.experts.11.gate_proj.weight": "model-00058-of-00101.safetensors", + "model.layers.54.mlp.experts.11.up_proj.weight": "model-00058-of-00101.safetensors", + "model.layers.54.mlp.experts.11.down_proj.weight": "model-00058-of-00101.safetensors", + "model.layers.54.mlp.experts.12.gate_proj.weight": "model-00058-of-00101.safetensors", + "model.layers.54.mlp.experts.12.up_proj.weight": "model-00058-of-00101.safetensors", + "model.layers.54.mlp.experts.12.down_proj.weight": "model-00058-of-00101.safetensors", + "model.layers.54.mlp.experts.13.gate_proj.weight": "model-00058-of-00101.safetensors", + "model.layers.54.mlp.experts.13.up_proj.weight": "model-00058-of-00101.safetensors", + "model.layers.54.mlp.experts.13.down_proj.weight": "model-00058-of-00101.safetensors", + "model.layers.54.mlp.experts.14.gate_proj.weight": "model-00058-of-00101.safetensors", + "model.layers.54.mlp.experts.14.up_proj.weight": "model-00058-of-00101.safetensors", + "model.layers.54.mlp.experts.14.down_proj.weight": "model-00058-of-00101.safetensors", + "model.layers.54.mlp.experts.15.gate_proj.weight": "model-00058-of-00101.safetensors", + "model.layers.54.mlp.experts.15.up_proj.weight": "model-00058-of-00101.safetensors", + "model.layers.54.mlp.experts.15.down_proj.weight": "model-00058-of-00101.safetensors", + "model.layers.54.mlp.experts.16.gate_proj.weight": "model-00058-of-00101.safetensors", + "model.layers.54.mlp.experts.16.up_proj.weight": "model-00058-of-00101.safetensors", + "model.layers.54.mlp.experts.16.down_proj.weight": "model-00058-of-00101.safetensors", + "model.layers.54.mlp.experts.17.gate_proj.weight": "model-00058-of-00101.safetensors", + "model.layers.54.mlp.experts.17.up_proj.weight": "model-00058-of-00101.safetensors", + "model.layers.54.mlp.experts.17.down_proj.weight": "model-00058-of-00101.safetensors", + "model.layers.54.mlp.experts.18.gate_proj.weight": "model-00058-of-00101.safetensors", + "model.layers.54.mlp.experts.18.up_proj.weight": "model-00058-of-00101.safetensors", + "model.layers.54.mlp.experts.18.down_proj.weight": "model-00058-of-00101.safetensors", + "model.layers.54.mlp.experts.19.gate_proj.weight": "model-00058-of-00101.safetensors", + "model.layers.54.mlp.experts.19.up_proj.weight": "model-00058-of-00101.safetensors", + "model.layers.54.mlp.experts.19.down_proj.weight": "model-00058-of-00101.safetensors", + "model.layers.54.mlp.experts.20.gate_proj.weight": "model-00058-of-00101.safetensors", + "model.layers.54.mlp.experts.20.up_proj.weight": "model-00058-of-00101.safetensors", + "model.layers.54.mlp.experts.20.down_proj.weight": "model-00058-of-00101.safetensors", + "model.layers.54.mlp.experts.21.gate_proj.weight": "model-00058-of-00101.safetensors", + "model.layers.54.mlp.experts.21.up_proj.weight": "model-00058-of-00101.safetensors", + "model.layers.54.mlp.experts.21.down_proj.weight": "model-00058-of-00101.safetensors", + "model.layers.54.mlp.experts.22.gate_proj.weight": "model-00058-of-00101.safetensors", + "model.layers.54.mlp.experts.22.up_proj.weight": "model-00058-of-00101.safetensors", + "model.layers.54.mlp.experts.22.down_proj.weight": "model-00058-of-00101.safetensors", + "model.layers.54.mlp.experts.23.gate_proj.weight": "model-00058-of-00101.safetensors", + "model.layers.54.mlp.experts.23.up_proj.weight": "model-00058-of-00101.safetensors", + "model.layers.54.mlp.experts.23.down_proj.weight": "model-00058-of-00101.safetensors", + "model.layers.54.mlp.experts.24.gate_proj.weight": "model-00058-of-00101.safetensors", + "model.layers.54.mlp.experts.24.up_proj.weight": "model-00058-of-00101.safetensors", + "model.layers.54.mlp.experts.24.down_proj.weight": "model-00058-of-00101.safetensors", + "model.layers.54.mlp.experts.25.gate_proj.weight": "model-00058-of-00101.safetensors", + "model.layers.54.mlp.experts.25.up_proj.weight": "model-00058-of-00101.safetensors", + "model.layers.54.mlp.experts.25.down_proj.weight": "model-00058-of-00101.safetensors", + "model.layers.54.mlp.experts.26.gate_proj.weight": "model-00058-of-00101.safetensors", + "model.layers.54.mlp.experts.26.up_proj.weight": "model-00058-of-00101.safetensors", + "model.layers.54.mlp.experts.26.down_proj.weight": "model-00058-of-00101.safetensors", + "model.layers.54.mlp.experts.27.gate_proj.weight": "model-00058-of-00101.safetensors", + "model.layers.54.mlp.experts.27.up_proj.weight": "model-00058-of-00101.safetensors", + "model.layers.54.mlp.experts.27.down_proj.weight": "model-00058-of-00101.safetensors", + "model.layers.54.mlp.experts.28.gate_proj.weight": "model-00058-of-00101.safetensors", + "model.layers.54.mlp.experts.28.up_proj.weight": "model-00058-of-00101.safetensors", + "model.layers.54.mlp.experts.28.down_proj.weight": "model-00058-of-00101.safetensors", + "model.layers.54.mlp.experts.29.gate_proj.weight": "model-00058-of-00101.safetensors", + "model.layers.54.mlp.experts.29.up_proj.weight": "model-00058-of-00101.safetensors", + "model.layers.54.mlp.experts.29.down_proj.weight": "model-00058-of-00101.safetensors", + "model.layers.54.mlp.experts.30.gate_proj.weight": "model-00058-of-00101.safetensors", + "model.layers.54.mlp.experts.30.up_proj.weight": "model-00058-of-00101.safetensors", + "model.layers.54.mlp.experts.30.down_proj.weight": "model-00058-of-00101.safetensors", + "model.layers.54.mlp.experts.31.gate_proj.weight": "model-00058-of-00101.safetensors", + "model.layers.54.mlp.experts.31.up_proj.weight": "model-00058-of-00101.safetensors", + "model.layers.54.mlp.experts.31.down_proj.weight": "model-00058-of-00101.safetensors", + "model.layers.54.mlp.experts.32.gate_proj.weight": "model-00058-of-00101.safetensors", + "model.layers.54.mlp.experts.32.up_proj.weight": "model-00058-of-00101.safetensors", + "model.layers.54.mlp.experts.32.down_proj.weight": "model-00058-of-00101.safetensors", + "model.layers.54.mlp.experts.33.gate_proj.weight": "model-00058-of-00101.safetensors", + "model.layers.54.mlp.experts.33.up_proj.weight": "model-00058-of-00101.safetensors", + "model.layers.54.mlp.experts.33.down_proj.weight": "model-00058-of-00101.safetensors", + "model.layers.54.mlp.experts.34.gate_proj.weight": "model-00058-of-00101.safetensors", + "model.layers.54.mlp.experts.34.up_proj.weight": "model-00058-of-00101.safetensors", + "model.layers.54.mlp.experts.34.down_proj.weight": "model-00059-of-00101.safetensors", + "model.layers.54.mlp.experts.35.gate_proj.weight": "model-00059-of-00101.safetensors", + "model.layers.54.mlp.experts.35.up_proj.weight": "model-00059-of-00101.safetensors", + "model.layers.54.mlp.experts.35.down_proj.weight": "model-00059-of-00101.safetensors", + "model.layers.54.mlp.experts.36.gate_proj.weight": "model-00059-of-00101.safetensors", + "model.layers.54.mlp.experts.36.up_proj.weight": "model-00059-of-00101.safetensors", + "model.layers.54.mlp.experts.36.down_proj.weight": "model-00059-of-00101.safetensors", + "model.layers.54.mlp.experts.37.gate_proj.weight": "model-00059-of-00101.safetensors", + "model.layers.54.mlp.experts.37.up_proj.weight": "model-00059-of-00101.safetensors", + "model.layers.54.mlp.experts.37.down_proj.weight": "model-00059-of-00101.safetensors", + "model.layers.54.mlp.experts.38.gate_proj.weight": "model-00059-of-00101.safetensors", + "model.layers.54.mlp.experts.38.up_proj.weight": "model-00059-of-00101.safetensors", + "model.layers.54.mlp.experts.38.down_proj.weight": "model-00059-of-00101.safetensors", + "model.layers.54.mlp.experts.39.gate_proj.weight": "model-00059-of-00101.safetensors", + "model.layers.54.mlp.experts.39.up_proj.weight": "model-00059-of-00101.safetensors", + "model.layers.54.mlp.experts.39.down_proj.weight": "model-00059-of-00101.safetensors", + "model.layers.54.mlp.experts.40.gate_proj.weight": "model-00059-of-00101.safetensors", + "model.layers.54.mlp.experts.40.up_proj.weight": "model-00059-of-00101.safetensors", + "model.layers.54.mlp.experts.40.down_proj.weight": "model-00059-of-00101.safetensors", + "model.layers.54.mlp.experts.41.gate_proj.weight": "model-00059-of-00101.safetensors", + "model.layers.54.mlp.experts.41.up_proj.weight": "model-00059-of-00101.safetensors", + "model.layers.54.mlp.experts.41.down_proj.weight": "model-00059-of-00101.safetensors", + "model.layers.54.mlp.experts.42.gate_proj.weight": "model-00059-of-00101.safetensors", + "model.layers.54.mlp.experts.42.up_proj.weight": "model-00059-of-00101.safetensors", + "model.layers.54.mlp.experts.42.down_proj.weight": "model-00059-of-00101.safetensors", + "model.layers.54.mlp.experts.43.gate_proj.weight": "model-00059-of-00101.safetensors", + "model.layers.54.mlp.experts.43.up_proj.weight": "model-00059-of-00101.safetensors", + "model.layers.54.mlp.experts.43.down_proj.weight": "model-00059-of-00101.safetensors", + "model.layers.54.mlp.experts.44.gate_proj.weight": "model-00059-of-00101.safetensors", + "model.layers.54.mlp.experts.44.up_proj.weight": "model-00059-of-00101.safetensors", + "model.layers.54.mlp.experts.44.down_proj.weight": "model-00059-of-00101.safetensors", + "model.layers.54.mlp.experts.45.gate_proj.weight": "model-00059-of-00101.safetensors", + "model.layers.54.mlp.experts.45.up_proj.weight": "model-00059-of-00101.safetensors", + "model.layers.54.mlp.experts.45.down_proj.weight": "model-00059-of-00101.safetensors", + "model.layers.54.mlp.experts.46.gate_proj.weight": "model-00059-of-00101.safetensors", + "model.layers.54.mlp.experts.46.up_proj.weight": "model-00059-of-00101.safetensors", + "model.layers.54.mlp.experts.46.down_proj.weight": "model-00059-of-00101.safetensors", + "model.layers.54.mlp.experts.47.gate_proj.weight": "model-00059-of-00101.safetensors", + "model.layers.54.mlp.experts.47.up_proj.weight": "model-00059-of-00101.safetensors", + "model.layers.54.mlp.experts.47.down_proj.weight": "model-00059-of-00101.safetensors", + "model.layers.54.mlp.experts.48.gate_proj.weight": "model-00059-of-00101.safetensors", + "model.layers.54.mlp.experts.48.up_proj.weight": "model-00059-of-00101.safetensors", + "model.layers.54.mlp.experts.48.down_proj.weight": "model-00059-of-00101.safetensors", + "model.layers.54.mlp.experts.49.gate_proj.weight": "model-00059-of-00101.safetensors", + "model.layers.54.mlp.experts.49.up_proj.weight": "model-00059-of-00101.safetensors", + "model.layers.54.mlp.experts.49.down_proj.weight": "model-00059-of-00101.safetensors", + "model.layers.54.mlp.experts.50.gate_proj.weight": "model-00059-of-00101.safetensors", + "model.layers.54.mlp.experts.50.up_proj.weight": "model-00059-of-00101.safetensors", + "model.layers.54.mlp.experts.50.down_proj.weight": "model-00059-of-00101.safetensors", + "model.layers.54.mlp.experts.51.gate_proj.weight": "model-00059-of-00101.safetensors", + "model.layers.54.mlp.experts.51.up_proj.weight": "model-00059-of-00101.safetensors", + "model.layers.54.mlp.experts.51.down_proj.weight": "model-00059-of-00101.safetensors", + "model.layers.54.mlp.experts.52.gate_proj.weight": "model-00059-of-00101.safetensors", + "model.layers.54.mlp.experts.52.up_proj.weight": "model-00059-of-00101.safetensors", + "model.layers.54.mlp.experts.52.down_proj.weight": "model-00059-of-00101.safetensors", + "model.layers.54.mlp.experts.53.gate_proj.weight": "model-00059-of-00101.safetensors", + "model.layers.54.mlp.experts.53.up_proj.weight": "model-00059-of-00101.safetensors", + "model.layers.54.mlp.experts.53.down_proj.weight": "model-00059-of-00101.safetensors", + "model.layers.54.mlp.experts.54.gate_proj.weight": "model-00059-of-00101.safetensors", + "model.layers.54.mlp.experts.54.up_proj.weight": "model-00059-of-00101.safetensors", + "model.layers.54.mlp.experts.54.down_proj.weight": "model-00059-of-00101.safetensors", + "model.layers.54.mlp.experts.55.gate_proj.weight": "model-00059-of-00101.safetensors", + "model.layers.54.mlp.experts.55.up_proj.weight": "model-00059-of-00101.safetensors", + "model.layers.54.mlp.experts.55.down_proj.weight": "model-00059-of-00101.safetensors", + "model.layers.54.mlp.experts.56.gate_proj.weight": "model-00059-of-00101.safetensors", + "model.layers.54.mlp.experts.56.up_proj.weight": "model-00059-of-00101.safetensors", + "model.layers.54.mlp.experts.56.down_proj.weight": "model-00059-of-00101.safetensors", + "model.layers.54.mlp.experts.57.gate_proj.weight": "model-00059-of-00101.safetensors", + "model.layers.54.mlp.experts.57.up_proj.weight": "model-00059-of-00101.safetensors", + "model.layers.54.mlp.experts.57.down_proj.weight": "model-00059-of-00101.safetensors", + "model.layers.54.mlp.experts.58.gate_proj.weight": "model-00059-of-00101.safetensors", + "model.layers.54.mlp.experts.58.up_proj.weight": "model-00059-of-00101.safetensors", + "model.layers.54.mlp.experts.58.down_proj.weight": "model-00059-of-00101.safetensors", + "model.layers.54.mlp.experts.59.gate_proj.weight": "model-00059-of-00101.safetensors", + "model.layers.54.mlp.experts.59.up_proj.weight": "model-00059-of-00101.safetensors", + "model.layers.54.mlp.experts.59.down_proj.weight": "model-00059-of-00101.safetensors", + "model.layers.54.mlp.experts.60.gate_proj.weight": "model-00059-of-00101.safetensors", + "model.layers.54.mlp.experts.60.up_proj.weight": "model-00059-of-00101.safetensors", + "model.layers.54.mlp.experts.60.down_proj.weight": "model-00059-of-00101.safetensors", + "model.layers.54.mlp.experts.61.gate_proj.weight": "model-00059-of-00101.safetensors", + "model.layers.54.mlp.experts.61.up_proj.weight": "model-00059-of-00101.safetensors", + "model.layers.54.mlp.experts.61.down_proj.weight": "model-00059-of-00101.safetensors", + "model.layers.54.mlp.experts.62.gate_proj.weight": "model-00059-of-00101.safetensors", + "model.layers.54.mlp.experts.62.up_proj.weight": "model-00059-of-00101.safetensors", + "model.layers.54.mlp.experts.62.down_proj.weight": "model-00059-of-00101.safetensors", + "model.layers.54.mlp.experts.63.gate_proj.weight": "model-00059-of-00101.safetensors", + "model.layers.54.mlp.experts.63.up_proj.weight": "model-00059-of-00101.safetensors", + "model.layers.54.mlp.experts.63.down_proj.weight": "model-00059-of-00101.safetensors", + "model.layers.54.mlp.experts.64.gate_proj.weight": "model-00059-of-00101.safetensors", + "model.layers.54.mlp.experts.64.up_proj.weight": "model-00059-of-00101.safetensors", + "model.layers.54.mlp.experts.64.down_proj.weight": "model-00059-of-00101.safetensors", + "model.layers.54.mlp.experts.65.gate_proj.weight": "model-00059-of-00101.safetensors", + "model.layers.54.mlp.experts.65.up_proj.weight": "model-00059-of-00101.safetensors", + "model.layers.54.mlp.experts.65.down_proj.weight": "model-00059-of-00101.safetensors", + "model.layers.54.mlp.experts.66.gate_proj.weight": "model-00059-of-00101.safetensors", + "model.layers.54.mlp.experts.66.up_proj.weight": "model-00059-of-00101.safetensors", + "model.layers.54.mlp.experts.66.down_proj.weight": "model-00059-of-00101.safetensors", + "model.layers.54.mlp.experts.67.gate_proj.weight": "model-00059-of-00101.safetensors", + "model.layers.54.mlp.experts.67.up_proj.weight": "model-00059-of-00101.safetensors", + "model.layers.54.mlp.experts.67.down_proj.weight": "model-00059-of-00101.safetensors", + "model.layers.54.mlp.experts.68.gate_proj.weight": "model-00059-of-00101.safetensors", + "model.layers.54.mlp.experts.68.up_proj.weight": "model-00059-of-00101.safetensors", + "model.layers.54.mlp.experts.68.down_proj.weight": "model-00059-of-00101.safetensors", + "model.layers.54.mlp.experts.69.gate_proj.weight": "model-00059-of-00101.safetensors", + "model.layers.54.mlp.experts.69.up_proj.weight": "model-00059-of-00101.safetensors", + "model.layers.54.mlp.experts.69.down_proj.weight": "model-00059-of-00101.safetensors", + "model.layers.54.mlp.experts.70.gate_proj.weight": "model-00059-of-00101.safetensors", + "model.layers.54.mlp.experts.70.up_proj.weight": "model-00059-of-00101.safetensors", + "model.layers.54.mlp.experts.70.down_proj.weight": "model-00059-of-00101.safetensors", + "model.layers.54.mlp.experts.71.gate_proj.weight": "model-00059-of-00101.safetensors", + "model.layers.54.mlp.experts.71.up_proj.weight": "model-00059-of-00101.safetensors", + "model.layers.54.mlp.experts.71.down_proj.weight": "model-00059-of-00101.safetensors", + "model.layers.54.mlp.experts.72.gate_proj.weight": "model-00059-of-00101.safetensors", + "model.layers.54.mlp.experts.72.up_proj.weight": "model-00059-of-00101.safetensors", + "model.layers.54.mlp.experts.72.down_proj.weight": "model-00059-of-00101.safetensors", + "model.layers.54.mlp.experts.73.gate_proj.weight": "model-00059-of-00101.safetensors", + "model.layers.54.mlp.experts.73.up_proj.weight": "model-00059-of-00101.safetensors", + "model.layers.54.mlp.experts.73.down_proj.weight": "model-00059-of-00101.safetensors", + "model.layers.54.mlp.experts.74.gate_proj.weight": "model-00059-of-00101.safetensors", + "model.layers.54.mlp.experts.74.up_proj.weight": "model-00059-of-00101.safetensors", + "model.layers.54.mlp.experts.74.down_proj.weight": "model-00059-of-00101.safetensors", + "model.layers.54.mlp.experts.75.gate_proj.weight": "model-00059-of-00101.safetensors", + "model.layers.54.mlp.experts.75.up_proj.weight": "model-00059-of-00101.safetensors", + "model.layers.54.mlp.experts.75.down_proj.weight": "model-00059-of-00101.safetensors", + "model.layers.54.mlp.experts.76.gate_proj.weight": "model-00059-of-00101.safetensors", + "model.layers.54.mlp.experts.76.up_proj.weight": "model-00059-of-00101.safetensors", + "model.layers.54.mlp.experts.76.down_proj.weight": "model-00059-of-00101.safetensors", + "model.layers.54.mlp.experts.77.gate_proj.weight": "model-00059-of-00101.safetensors", + "model.layers.54.mlp.experts.77.up_proj.weight": "model-00059-of-00101.safetensors", + "model.layers.54.mlp.experts.77.down_proj.weight": "model-00059-of-00101.safetensors", + "model.layers.54.mlp.experts.78.gate_proj.weight": "model-00059-of-00101.safetensors", + "model.layers.54.mlp.experts.78.up_proj.weight": "model-00059-of-00101.safetensors", + "model.layers.54.mlp.experts.78.down_proj.weight": "model-00059-of-00101.safetensors", + "model.layers.54.mlp.experts.79.gate_proj.weight": "model-00059-of-00101.safetensors", + "model.layers.54.mlp.experts.79.up_proj.weight": "model-00059-of-00101.safetensors", + "model.layers.54.mlp.experts.79.down_proj.weight": "model-00059-of-00101.safetensors", + "model.layers.54.mlp.experts.80.gate_proj.weight": "model-00059-of-00101.safetensors", + "model.layers.54.mlp.experts.80.up_proj.weight": "model-00059-of-00101.safetensors", + "model.layers.54.mlp.experts.80.down_proj.weight": "model-00059-of-00101.safetensors", + "model.layers.54.mlp.experts.81.gate_proj.weight": "model-00059-of-00101.safetensors", + "model.layers.54.mlp.experts.81.up_proj.weight": "model-00059-of-00101.safetensors", + "model.layers.54.mlp.experts.81.down_proj.weight": "model-00059-of-00101.safetensors", + "model.layers.54.mlp.experts.82.gate_proj.weight": "model-00059-of-00101.safetensors", + "model.layers.54.mlp.experts.82.up_proj.weight": "model-00059-of-00101.safetensors", + "model.layers.54.mlp.experts.82.down_proj.weight": "model-00059-of-00101.safetensors", + "model.layers.54.mlp.experts.83.gate_proj.weight": "model-00059-of-00101.safetensors", + "model.layers.54.mlp.experts.83.up_proj.weight": "model-00059-of-00101.safetensors", + "model.layers.54.mlp.experts.83.down_proj.weight": "model-00059-of-00101.safetensors", + "model.layers.54.mlp.experts.84.gate_proj.weight": "model-00059-of-00101.safetensors", + "model.layers.54.mlp.experts.84.up_proj.weight": "model-00059-of-00101.safetensors", + "model.layers.54.mlp.experts.84.down_proj.weight": "model-00059-of-00101.safetensors", + "model.layers.54.mlp.experts.85.gate_proj.weight": "model-00059-of-00101.safetensors", + "model.layers.54.mlp.experts.85.up_proj.weight": "model-00059-of-00101.safetensors", + "model.layers.54.mlp.experts.85.down_proj.weight": "model-00059-of-00101.safetensors", + "model.layers.54.mlp.experts.86.gate_proj.weight": "model-00059-of-00101.safetensors", + "model.layers.54.mlp.experts.86.up_proj.weight": "model-00059-of-00101.safetensors", + "model.layers.54.mlp.experts.86.down_proj.weight": "model-00059-of-00101.safetensors", + "model.layers.54.mlp.experts.87.gate_proj.weight": "model-00059-of-00101.safetensors", + "model.layers.54.mlp.experts.87.up_proj.weight": "model-00059-of-00101.safetensors", + "model.layers.54.mlp.experts.87.down_proj.weight": "model-00059-of-00101.safetensors", + "model.layers.54.mlp.experts.88.gate_proj.weight": "model-00059-of-00101.safetensors", + "model.layers.54.mlp.experts.88.up_proj.weight": "model-00059-of-00101.safetensors", + "model.layers.54.mlp.experts.88.down_proj.weight": "model-00059-of-00101.safetensors", + "model.layers.54.mlp.experts.89.gate_proj.weight": "model-00059-of-00101.safetensors", + "model.layers.54.mlp.experts.89.up_proj.weight": "model-00059-of-00101.safetensors", + "model.layers.54.mlp.experts.89.down_proj.weight": "model-00059-of-00101.safetensors", + "model.layers.54.mlp.experts.90.gate_proj.weight": "model-00059-of-00101.safetensors", + "model.layers.54.mlp.experts.90.up_proj.weight": "model-00059-of-00101.safetensors", + "model.layers.54.mlp.experts.90.down_proj.weight": "model-00059-of-00101.safetensors", + "model.layers.54.mlp.experts.91.gate_proj.weight": "model-00059-of-00101.safetensors", + "model.layers.54.mlp.experts.91.up_proj.weight": "model-00059-of-00101.safetensors", + "model.layers.54.mlp.experts.91.down_proj.weight": "model-00059-of-00101.safetensors", + "model.layers.54.mlp.experts.92.gate_proj.weight": "model-00059-of-00101.safetensors", + "model.layers.54.mlp.experts.92.up_proj.weight": "model-00059-of-00101.safetensors", + "model.layers.54.mlp.experts.92.down_proj.weight": "model-00059-of-00101.safetensors", + "model.layers.54.mlp.experts.93.gate_proj.weight": "model-00059-of-00101.safetensors", + "model.layers.54.mlp.experts.93.up_proj.weight": "model-00059-of-00101.safetensors", + "model.layers.54.mlp.experts.93.down_proj.weight": "model-00059-of-00101.safetensors", + "model.layers.54.mlp.experts.94.gate_proj.weight": "model-00059-of-00101.safetensors", + "model.layers.54.mlp.experts.94.up_proj.weight": "model-00059-of-00101.safetensors", + "model.layers.54.mlp.experts.94.down_proj.weight": "model-00059-of-00101.safetensors", + "model.layers.54.mlp.experts.95.gate_proj.weight": "model-00059-of-00101.safetensors", + "model.layers.54.mlp.experts.95.up_proj.weight": "model-00059-of-00101.safetensors", + "model.layers.54.mlp.experts.95.down_proj.weight": "model-00059-of-00101.safetensors", + "model.layers.54.mlp.experts.96.gate_proj.weight": "model-00059-of-00101.safetensors", + "model.layers.54.mlp.experts.96.up_proj.weight": "model-00059-of-00101.safetensors", + "model.layers.54.mlp.experts.96.down_proj.weight": "model-00059-of-00101.safetensors", + "model.layers.54.mlp.experts.97.gate_proj.weight": "model-00059-of-00101.safetensors", + "model.layers.54.mlp.experts.97.up_proj.weight": "model-00059-of-00101.safetensors", + "model.layers.54.mlp.experts.97.down_proj.weight": "model-00059-of-00101.safetensors", + "model.layers.54.mlp.experts.98.gate_proj.weight": "model-00059-of-00101.safetensors", + "model.layers.54.mlp.experts.98.up_proj.weight": "model-00059-of-00101.safetensors", + "model.layers.54.mlp.experts.98.down_proj.weight": "model-00059-of-00101.safetensors", + "model.layers.54.mlp.experts.99.gate_proj.weight": "model-00059-of-00101.safetensors", + "model.layers.54.mlp.experts.99.up_proj.weight": "model-00059-of-00101.safetensors", + "model.layers.54.mlp.experts.99.down_proj.weight": "model-00059-of-00101.safetensors", + "model.layers.54.mlp.experts.100.gate_proj.weight": "model-00059-of-00101.safetensors", + "model.layers.54.mlp.experts.100.up_proj.weight": "model-00059-of-00101.safetensors", + "model.layers.54.mlp.experts.100.down_proj.weight": "model-00059-of-00101.safetensors", + "model.layers.54.mlp.experts.101.gate_proj.weight": "model-00059-of-00101.safetensors", + "model.layers.54.mlp.experts.101.up_proj.weight": "model-00059-of-00101.safetensors", + "model.layers.54.mlp.experts.101.down_proj.weight": "model-00059-of-00101.safetensors", + "model.layers.54.mlp.experts.102.gate_proj.weight": "model-00059-of-00101.safetensors", + "model.layers.54.mlp.experts.102.up_proj.weight": "model-00059-of-00101.safetensors", + "model.layers.54.mlp.experts.102.down_proj.weight": "model-00059-of-00101.safetensors", + "model.layers.54.mlp.experts.103.gate_proj.weight": "model-00059-of-00101.safetensors", + "model.layers.54.mlp.experts.103.up_proj.weight": "model-00059-of-00101.safetensors", + "model.layers.54.mlp.experts.103.down_proj.weight": "model-00059-of-00101.safetensors", + "model.layers.54.mlp.experts.104.gate_proj.weight": "model-00059-of-00101.safetensors", + "model.layers.54.mlp.experts.104.up_proj.weight": "model-00059-of-00101.safetensors", + "model.layers.54.mlp.experts.104.down_proj.weight": "model-00059-of-00101.safetensors", + "model.layers.54.mlp.experts.105.gate_proj.weight": "model-00059-of-00101.safetensors", + "model.layers.54.mlp.experts.105.up_proj.weight": "model-00059-of-00101.safetensors", + "model.layers.54.mlp.experts.105.down_proj.weight": "model-00059-of-00101.safetensors", + "model.layers.54.mlp.experts.106.gate_proj.weight": "model-00059-of-00101.safetensors", + "model.layers.54.mlp.experts.106.up_proj.weight": "model-00059-of-00101.safetensors", + "model.layers.54.mlp.experts.106.down_proj.weight": "model-00059-of-00101.safetensors", + "model.layers.54.mlp.experts.107.gate_proj.weight": "model-00059-of-00101.safetensors", + "model.layers.54.mlp.experts.107.up_proj.weight": "model-00059-of-00101.safetensors", + "model.layers.54.mlp.experts.107.down_proj.weight": "model-00059-of-00101.safetensors", + "model.layers.54.mlp.experts.108.gate_proj.weight": "model-00059-of-00101.safetensors", + "model.layers.54.mlp.experts.108.up_proj.weight": "model-00059-of-00101.safetensors", + "model.layers.54.mlp.experts.108.down_proj.weight": "model-00059-of-00101.safetensors", + "model.layers.54.mlp.experts.109.gate_proj.weight": "model-00059-of-00101.safetensors", + "model.layers.54.mlp.experts.109.up_proj.weight": "model-00059-of-00101.safetensors", + "model.layers.54.mlp.experts.109.down_proj.weight": "model-00059-of-00101.safetensors", + "model.layers.54.mlp.experts.110.gate_proj.weight": "model-00059-of-00101.safetensors", + "model.layers.54.mlp.experts.110.up_proj.weight": "model-00059-of-00101.safetensors", + "model.layers.54.mlp.experts.110.down_proj.weight": "model-00059-of-00101.safetensors", + "model.layers.54.mlp.experts.111.gate_proj.weight": "model-00059-of-00101.safetensors", + "model.layers.54.mlp.experts.111.up_proj.weight": "model-00059-of-00101.safetensors", + "model.layers.54.mlp.experts.111.down_proj.weight": "model-00059-of-00101.safetensors", + "model.layers.54.mlp.experts.112.gate_proj.weight": "model-00059-of-00101.safetensors", + "model.layers.54.mlp.experts.112.up_proj.weight": "model-00059-of-00101.safetensors", + "model.layers.54.mlp.experts.112.down_proj.weight": "model-00059-of-00101.safetensors", + "model.layers.54.mlp.experts.113.gate_proj.weight": "model-00059-of-00101.safetensors", + "model.layers.54.mlp.experts.113.up_proj.weight": "model-00059-of-00101.safetensors", + "model.layers.54.mlp.experts.113.down_proj.weight": "model-00059-of-00101.safetensors", + "model.layers.54.mlp.experts.114.gate_proj.weight": "model-00059-of-00101.safetensors", + "model.layers.54.mlp.experts.114.up_proj.weight": "model-00059-of-00101.safetensors", + "model.layers.54.mlp.experts.114.down_proj.weight": "model-00059-of-00101.safetensors", + "model.layers.54.mlp.experts.115.gate_proj.weight": "model-00059-of-00101.safetensors", + "model.layers.54.mlp.experts.115.up_proj.weight": "model-00059-of-00101.safetensors", + "model.layers.54.mlp.experts.115.down_proj.weight": "model-00059-of-00101.safetensors", + "model.layers.54.mlp.experts.116.gate_proj.weight": "model-00059-of-00101.safetensors", + "model.layers.54.mlp.experts.116.up_proj.weight": "model-00059-of-00101.safetensors", + "model.layers.54.mlp.experts.116.down_proj.weight": "model-00059-of-00101.safetensors", + "model.layers.54.mlp.experts.117.gate_proj.weight": "model-00059-of-00101.safetensors", + "model.layers.54.mlp.experts.117.up_proj.weight": "model-00059-of-00101.safetensors", + "model.layers.54.mlp.experts.117.down_proj.weight": "model-00059-of-00101.safetensors", + "model.layers.54.mlp.experts.118.gate_proj.weight": "model-00059-of-00101.safetensors", + "model.layers.54.mlp.experts.118.up_proj.weight": "model-00059-of-00101.safetensors", + "model.layers.54.mlp.experts.118.down_proj.weight": "model-00059-of-00101.safetensors", + "model.layers.54.mlp.experts.119.gate_proj.weight": "model-00059-of-00101.safetensors", + "model.layers.54.mlp.experts.119.up_proj.weight": "model-00059-of-00101.safetensors", + "model.layers.54.mlp.experts.119.down_proj.weight": "model-00059-of-00101.safetensors", + "model.layers.54.mlp.gate.weight": "model-00059-of-00101.safetensors", + "model.layers.54.mlp.gate.e_score_correction_bias": "model-00059-of-00101.safetensors", + "model.layers.54.mlp.shared_experts.gate_proj.weight": "model-00059-of-00101.safetensors", + "model.layers.54.mlp.shared_experts.up_proj.weight": "model-00059-of-00101.safetensors", + "model.layers.54.mlp.shared_experts.down_proj.weight": "model-00059-of-00101.safetensors", + "model.layers.54.input_layernorm.weight": "model-00059-of-00101.safetensors", + "model.layers.54.post_attention_layernorm.weight": "model-00059-of-00101.safetensors", + "model.layers.55.self_attn.q_proj.weight": "model-00059-of-00101.safetensors", + "model.layers.55.self_attn.q_proj.bias": "model-00059-of-00101.safetensors", + "model.layers.55.self_attn.k_proj.weight": "model-00059-of-00101.safetensors", + "model.layers.55.self_attn.k_proj.bias": "model-00059-of-00101.safetensors", + "model.layers.55.self_attn.v_proj.weight": "model-00059-of-00101.safetensors", + "model.layers.55.self_attn.v_proj.bias": "model-00059-of-00101.safetensors", + "model.layers.55.self_attn.o_proj.weight": "model-00059-of-00101.safetensors", + "model.layers.55.self_attn.q_norm.weight": "model-00059-of-00101.safetensors", + "model.layers.55.self_attn.k_norm.weight": "model-00059-of-00101.safetensors", + "model.layers.55.mlp.experts.0.gate_proj.weight": "model-00059-of-00101.safetensors", + "model.layers.55.mlp.experts.0.up_proj.weight": "model-00059-of-00101.safetensors", + "model.layers.55.mlp.experts.0.down_proj.weight": "model-00059-of-00101.safetensors", + "model.layers.55.mlp.experts.1.gate_proj.weight": "model-00059-of-00101.safetensors", + "model.layers.55.mlp.experts.1.up_proj.weight": "model-00059-of-00101.safetensors", + "model.layers.55.mlp.experts.1.down_proj.weight": "model-00059-of-00101.safetensors", + "model.layers.55.mlp.experts.2.gate_proj.weight": "model-00059-of-00101.safetensors", + "model.layers.55.mlp.experts.2.up_proj.weight": "model-00059-of-00101.safetensors", + "model.layers.55.mlp.experts.2.down_proj.weight": "model-00059-of-00101.safetensors", + "model.layers.55.mlp.experts.3.gate_proj.weight": "model-00059-of-00101.safetensors", + "model.layers.55.mlp.experts.3.up_proj.weight": "model-00059-of-00101.safetensors", + "model.layers.55.mlp.experts.3.down_proj.weight": "model-00059-of-00101.safetensors", + "model.layers.55.mlp.experts.4.gate_proj.weight": "model-00059-of-00101.safetensors", + "model.layers.55.mlp.experts.4.up_proj.weight": "model-00059-of-00101.safetensors", + "model.layers.55.mlp.experts.4.down_proj.weight": "model-00059-of-00101.safetensors", + "model.layers.55.mlp.experts.5.gate_proj.weight": "model-00059-of-00101.safetensors", + "model.layers.55.mlp.experts.5.up_proj.weight": "model-00059-of-00101.safetensors", + "model.layers.55.mlp.experts.5.down_proj.weight": "model-00059-of-00101.safetensors", + "model.layers.55.mlp.experts.6.gate_proj.weight": "model-00059-of-00101.safetensors", + "model.layers.55.mlp.experts.6.up_proj.weight": "model-00059-of-00101.safetensors", + "model.layers.55.mlp.experts.6.down_proj.weight": "model-00059-of-00101.safetensors", + "model.layers.55.mlp.experts.7.gate_proj.weight": "model-00059-of-00101.safetensors", + "model.layers.55.mlp.experts.7.up_proj.weight": "model-00059-of-00101.safetensors", + "model.layers.55.mlp.experts.7.down_proj.weight": "model-00059-of-00101.safetensors", + "model.layers.55.mlp.experts.8.gate_proj.weight": "model-00059-of-00101.safetensors", + "model.layers.55.mlp.experts.8.up_proj.weight": "model-00059-of-00101.safetensors", + "model.layers.55.mlp.experts.8.down_proj.weight": "model-00059-of-00101.safetensors", + "model.layers.55.mlp.experts.9.gate_proj.weight": "model-00059-of-00101.safetensors", + "model.layers.55.mlp.experts.9.up_proj.weight": "model-00059-of-00101.safetensors", + "model.layers.55.mlp.experts.9.down_proj.weight": "model-00059-of-00101.safetensors", + "model.layers.55.mlp.experts.10.gate_proj.weight": "model-00059-of-00101.safetensors", + "model.layers.55.mlp.experts.10.up_proj.weight": "model-00059-of-00101.safetensors", + "model.layers.55.mlp.experts.10.down_proj.weight": "model-00059-of-00101.safetensors", + "model.layers.55.mlp.experts.11.gate_proj.weight": "model-00059-of-00101.safetensors", + "model.layers.55.mlp.experts.11.up_proj.weight": "model-00059-of-00101.safetensors", + "model.layers.55.mlp.experts.11.down_proj.weight": "model-00059-of-00101.safetensors", + "model.layers.55.mlp.experts.12.gate_proj.weight": "model-00059-of-00101.safetensors", + "model.layers.55.mlp.experts.12.up_proj.weight": "model-00059-of-00101.safetensors", + "model.layers.55.mlp.experts.12.down_proj.weight": "model-00059-of-00101.safetensors", + "model.layers.55.mlp.experts.13.gate_proj.weight": "model-00059-of-00101.safetensors", + "model.layers.55.mlp.experts.13.up_proj.weight": "model-00059-of-00101.safetensors", + "model.layers.55.mlp.experts.13.down_proj.weight": "model-00059-of-00101.safetensors", + "model.layers.55.mlp.experts.14.gate_proj.weight": "model-00059-of-00101.safetensors", + "model.layers.55.mlp.experts.14.up_proj.weight": "model-00059-of-00101.safetensors", + "model.layers.55.mlp.experts.14.down_proj.weight": "model-00059-of-00101.safetensors", + "model.layers.55.mlp.experts.15.gate_proj.weight": "model-00059-of-00101.safetensors", + "model.layers.55.mlp.experts.15.up_proj.weight": "model-00059-of-00101.safetensors", + "model.layers.55.mlp.experts.15.down_proj.weight": "model-00059-of-00101.safetensors", + "model.layers.55.mlp.experts.16.gate_proj.weight": "model-00059-of-00101.safetensors", + "model.layers.55.mlp.experts.16.up_proj.weight": "model-00059-of-00101.safetensors", + "model.layers.55.mlp.experts.16.down_proj.weight": "model-00059-of-00101.safetensors", + "model.layers.55.mlp.experts.17.gate_proj.weight": "model-00059-of-00101.safetensors", + "model.layers.55.mlp.experts.17.up_proj.weight": "model-00059-of-00101.safetensors", + "model.layers.55.mlp.experts.17.down_proj.weight": "model-00059-of-00101.safetensors", + "model.layers.55.mlp.experts.18.gate_proj.weight": "model-00059-of-00101.safetensors", + "model.layers.55.mlp.experts.18.up_proj.weight": "model-00059-of-00101.safetensors", + "model.layers.55.mlp.experts.18.down_proj.weight": "model-00059-of-00101.safetensors", + "model.layers.55.mlp.experts.19.gate_proj.weight": "model-00059-of-00101.safetensors", + "model.layers.55.mlp.experts.19.up_proj.weight": "model-00059-of-00101.safetensors", + "model.layers.55.mlp.experts.19.down_proj.weight": "model-00059-of-00101.safetensors", + "model.layers.55.mlp.experts.20.gate_proj.weight": "model-00059-of-00101.safetensors", + "model.layers.55.mlp.experts.20.up_proj.weight": "model-00059-of-00101.safetensors", + "model.layers.55.mlp.experts.20.down_proj.weight": "model-00059-of-00101.safetensors", + "model.layers.55.mlp.experts.21.gate_proj.weight": "model-00059-of-00101.safetensors", + "model.layers.55.mlp.experts.21.up_proj.weight": "model-00060-of-00101.safetensors", + "model.layers.55.mlp.experts.21.down_proj.weight": "model-00060-of-00101.safetensors", + "model.layers.55.mlp.experts.22.gate_proj.weight": "model-00060-of-00101.safetensors", + "model.layers.55.mlp.experts.22.up_proj.weight": "model-00060-of-00101.safetensors", + "model.layers.55.mlp.experts.22.down_proj.weight": "model-00060-of-00101.safetensors", + "model.layers.55.mlp.experts.23.gate_proj.weight": "model-00060-of-00101.safetensors", + "model.layers.55.mlp.experts.23.up_proj.weight": "model-00060-of-00101.safetensors", + "model.layers.55.mlp.experts.23.down_proj.weight": "model-00060-of-00101.safetensors", + "model.layers.55.mlp.experts.24.gate_proj.weight": "model-00060-of-00101.safetensors", + "model.layers.55.mlp.experts.24.up_proj.weight": "model-00060-of-00101.safetensors", + "model.layers.55.mlp.experts.24.down_proj.weight": "model-00060-of-00101.safetensors", + "model.layers.55.mlp.experts.25.gate_proj.weight": "model-00060-of-00101.safetensors", + "model.layers.55.mlp.experts.25.up_proj.weight": "model-00060-of-00101.safetensors", + "model.layers.55.mlp.experts.25.down_proj.weight": "model-00060-of-00101.safetensors", + "model.layers.55.mlp.experts.26.gate_proj.weight": "model-00060-of-00101.safetensors", + "model.layers.55.mlp.experts.26.up_proj.weight": "model-00060-of-00101.safetensors", + "model.layers.55.mlp.experts.26.down_proj.weight": "model-00060-of-00101.safetensors", + "model.layers.55.mlp.experts.27.gate_proj.weight": "model-00060-of-00101.safetensors", + "model.layers.55.mlp.experts.27.up_proj.weight": "model-00060-of-00101.safetensors", + "model.layers.55.mlp.experts.27.down_proj.weight": "model-00060-of-00101.safetensors", + "model.layers.55.mlp.experts.28.gate_proj.weight": "model-00060-of-00101.safetensors", + "model.layers.55.mlp.experts.28.up_proj.weight": "model-00060-of-00101.safetensors", + "model.layers.55.mlp.experts.28.down_proj.weight": "model-00060-of-00101.safetensors", + "model.layers.55.mlp.experts.29.gate_proj.weight": "model-00060-of-00101.safetensors", + "model.layers.55.mlp.experts.29.up_proj.weight": "model-00060-of-00101.safetensors", + "model.layers.55.mlp.experts.29.down_proj.weight": "model-00060-of-00101.safetensors", + "model.layers.55.mlp.experts.30.gate_proj.weight": "model-00060-of-00101.safetensors", + "model.layers.55.mlp.experts.30.up_proj.weight": "model-00060-of-00101.safetensors", + "model.layers.55.mlp.experts.30.down_proj.weight": "model-00060-of-00101.safetensors", + "model.layers.55.mlp.experts.31.gate_proj.weight": "model-00060-of-00101.safetensors", + "model.layers.55.mlp.experts.31.up_proj.weight": "model-00060-of-00101.safetensors", + "model.layers.55.mlp.experts.31.down_proj.weight": "model-00060-of-00101.safetensors", + "model.layers.55.mlp.experts.32.gate_proj.weight": "model-00060-of-00101.safetensors", + "model.layers.55.mlp.experts.32.up_proj.weight": "model-00060-of-00101.safetensors", + "model.layers.55.mlp.experts.32.down_proj.weight": "model-00060-of-00101.safetensors", + "model.layers.55.mlp.experts.33.gate_proj.weight": "model-00060-of-00101.safetensors", + "model.layers.55.mlp.experts.33.up_proj.weight": "model-00060-of-00101.safetensors", + "model.layers.55.mlp.experts.33.down_proj.weight": "model-00060-of-00101.safetensors", + "model.layers.55.mlp.experts.34.gate_proj.weight": "model-00060-of-00101.safetensors", + "model.layers.55.mlp.experts.34.up_proj.weight": "model-00060-of-00101.safetensors", + "model.layers.55.mlp.experts.34.down_proj.weight": "model-00060-of-00101.safetensors", + "model.layers.55.mlp.experts.35.gate_proj.weight": "model-00060-of-00101.safetensors", + "model.layers.55.mlp.experts.35.up_proj.weight": "model-00060-of-00101.safetensors", + "model.layers.55.mlp.experts.35.down_proj.weight": "model-00060-of-00101.safetensors", + "model.layers.55.mlp.experts.36.gate_proj.weight": "model-00060-of-00101.safetensors", + "model.layers.55.mlp.experts.36.up_proj.weight": "model-00060-of-00101.safetensors", + "model.layers.55.mlp.experts.36.down_proj.weight": "model-00060-of-00101.safetensors", + "model.layers.55.mlp.experts.37.gate_proj.weight": "model-00060-of-00101.safetensors", + "model.layers.55.mlp.experts.37.up_proj.weight": "model-00060-of-00101.safetensors", + "model.layers.55.mlp.experts.37.down_proj.weight": "model-00060-of-00101.safetensors", + "model.layers.55.mlp.experts.38.gate_proj.weight": "model-00060-of-00101.safetensors", + "model.layers.55.mlp.experts.38.up_proj.weight": "model-00060-of-00101.safetensors", + "model.layers.55.mlp.experts.38.down_proj.weight": "model-00060-of-00101.safetensors", + "model.layers.55.mlp.experts.39.gate_proj.weight": "model-00060-of-00101.safetensors", + "model.layers.55.mlp.experts.39.up_proj.weight": "model-00060-of-00101.safetensors", + "model.layers.55.mlp.experts.39.down_proj.weight": "model-00060-of-00101.safetensors", + "model.layers.55.mlp.experts.40.gate_proj.weight": "model-00060-of-00101.safetensors", + "model.layers.55.mlp.experts.40.up_proj.weight": "model-00060-of-00101.safetensors", + "model.layers.55.mlp.experts.40.down_proj.weight": "model-00060-of-00101.safetensors", + "model.layers.55.mlp.experts.41.gate_proj.weight": "model-00060-of-00101.safetensors", + "model.layers.55.mlp.experts.41.up_proj.weight": "model-00060-of-00101.safetensors", + "model.layers.55.mlp.experts.41.down_proj.weight": "model-00060-of-00101.safetensors", + "model.layers.55.mlp.experts.42.gate_proj.weight": "model-00060-of-00101.safetensors", + "model.layers.55.mlp.experts.42.up_proj.weight": "model-00060-of-00101.safetensors", + "model.layers.55.mlp.experts.42.down_proj.weight": "model-00060-of-00101.safetensors", + "model.layers.55.mlp.experts.43.gate_proj.weight": "model-00060-of-00101.safetensors", + "model.layers.55.mlp.experts.43.up_proj.weight": "model-00060-of-00101.safetensors", + "model.layers.55.mlp.experts.43.down_proj.weight": "model-00060-of-00101.safetensors", + "model.layers.55.mlp.experts.44.gate_proj.weight": "model-00060-of-00101.safetensors", + "model.layers.55.mlp.experts.44.up_proj.weight": "model-00060-of-00101.safetensors", + "model.layers.55.mlp.experts.44.down_proj.weight": "model-00060-of-00101.safetensors", + "model.layers.55.mlp.experts.45.gate_proj.weight": "model-00060-of-00101.safetensors", + "model.layers.55.mlp.experts.45.up_proj.weight": "model-00060-of-00101.safetensors", + "model.layers.55.mlp.experts.45.down_proj.weight": "model-00060-of-00101.safetensors", + "model.layers.55.mlp.experts.46.gate_proj.weight": "model-00060-of-00101.safetensors", + "model.layers.55.mlp.experts.46.up_proj.weight": "model-00060-of-00101.safetensors", + "model.layers.55.mlp.experts.46.down_proj.weight": "model-00060-of-00101.safetensors", + "model.layers.55.mlp.experts.47.gate_proj.weight": "model-00060-of-00101.safetensors", + "model.layers.55.mlp.experts.47.up_proj.weight": "model-00060-of-00101.safetensors", + "model.layers.55.mlp.experts.47.down_proj.weight": "model-00060-of-00101.safetensors", + "model.layers.55.mlp.experts.48.gate_proj.weight": "model-00060-of-00101.safetensors", + "model.layers.55.mlp.experts.48.up_proj.weight": "model-00060-of-00101.safetensors", + "model.layers.55.mlp.experts.48.down_proj.weight": "model-00060-of-00101.safetensors", + "model.layers.55.mlp.experts.49.gate_proj.weight": "model-00060-of-00101.safetensors", + "model.layers.55.mlp.experts.49.up_proj.weight": "model-00060-of-00101.safetensors", + "model.layers.55.mlp.experts.49.down_proj.weight": "model-00060-of-00101.safetensors", + "model.layers.55.mlp.experts.50.gate_proj.weight": "model-00060-of-00101.safetensors", + "model.layers.55.mlp.experts.50.up_proj.weight": "model-00060-of-00101.safetensors", + "model.layers.55.mlp.experts.50.down_proj.weight": "model-00060-of-00101.safetensors", + "model.layers.55.mlp.experts.51.gate_proj.weight": "model-00060-of-00101.safetensors", + "model.layers.55.mlp.experts.51.up_proj.weight": "model-00060-of-00101.safetensors", + "model.layers.55.mlp.experts.51.down_proj.weight": "model-00060-of-00101.safetensors", + "model.layers.55.mlp.experts.52.gate_proj.weight": "model-00060-of-00101.safetensors", + "model.layers.55.mlp.experts.52.up_proj.weight": "model-00060-of-00101.safetensors", + "model.layers.55.mlp.experts.52.down_proj.weight": "model-00060-of-00101.safetensors", + "model.layers.55.mlp.experts.53.gate_proj.weight": "model-00060-of-00101.safetensors", + "model.layers.55.mlp.experts.53.up_proj.weight": "model-00060-of-00101.safetensors", + "model.layers.55.mlp.experts.53.down_proj.weight": "model-00060-of-00101.safetensors", + "model.layers.55.mlp.experts.54.gate_proj.weight": "model-00060-of-00101.safetensors", + "model.layers.55.mlp.experts.54.up_proj.weight": "model-00060-of-00101.safetensors", + "model.layers.55.mlp.experts.54.down_proj.weight": "model-00060-of-00101.safetensors", + "model.layers.55.mlp.experts.55.gate_proj.weight": "model-00060-of-00101.safetensors", + "model.layers.55.mlp.experts.55.up_proj.weight": "model-00060-of-00101.safetensors", + "model.layers.55.mlp.experts.55.down_proj.weight": "model-00060-of-00101.safetensors", + "model.layers.55.mlp.experts.56.gate_proj.weight": "model-00060-of-00101.safetensors", + "model.layers.55.mlp.experts.56.up_proj.weight": "model-00060-of-00101.safetensors", + "model.layers.55.mlp.experts.56.down_proj.weight": "model-00060-of-00101.safetensors", + "model.layers.55.mlp.experts.57.gate_proj.weight": "model-00060-of-00101.safetensors", + "model.layers.55.mlp.experts.57.up_proj.weight": "model-00060-of-00101.safetensors", + "model.layers.55.mlp.experts.57.down_proj.weight": "model-00060-of-00101.safetensors", + "model.layers.55.mlp.experts.58.gate_proj.weight": "model-00060-of-00101.safetensors", + "model.layers.55.mlp.experts.58.up_proj.weight": "model-00060-of-00101.safetensors", + "model.layers.55.mlp.experts.58.down_proj.weight": "model-00060-of-00101.safetensors", + "model.layers.55.mlp.experts.59.gate_proj.weight": "model-00060-of-00101.safetensors", + "model.layers.55.mlp.experts.59.up_proj.weight": "model-00060-of-00101.safetensors", + "model.layers.55.mlp.experts.59.down_proj.weight": "model-00060-of-00101.safetensors", + "model.layers.55.mlp.experts.60.gate_proj.weight": "model-00060-of-00101.safetensors", + "model.layers.55.mlp.experts.60.up_proj.weight": "model-00060-of-00101.safetensors", + "model.layers.55.mlp.experts.60.down_proj.weight": "model-00060-of-00101.safetensors", + "model.layers.55.mlp.experts.61.gate_proj.weight": "model-00060-of-00101.safetensors", + "model.layers.55.mlp.experts.61.up_proj.weight": "model-00060-of-00101.safetensors", + "model.layers.55.mlp.experts.61.down_proj.weight": "model-00060-of-00101.safetensors", + "model.layers.55.mlp.experts.62.gate_proj.weight": "model-00060-of-00101.safetensors", + "model.layers.55.mlp.experts.62.up_proj.weight": "model-00060-of-00101.safetensors", + "model.layers.55.mlp.experts.62.down_proj.weight": "model-00060-of-00101.safetensors", + "model.layers.55.mlp.experts.63.gate_proj.weight": "model-00060-of-00101.safetensors", + "model.layers.55.mlp.experts.63.up_proj.weight": "model-00060-of-00101.safetensors", + "model.layers.55.mlp.experts.63.down_proj.weight": "model-00060-of-00101.safetensors", + "model.layers.55.mlp.experts.64.gate_proj.weight": "model-00060-of-00101.safetensors", + "model.layers.55.mlp.experts.64.up_proj.weight": "model-00060-of-00101.safetensors", + "model.layers.55.mlp.experts.64.down_proj.weight": "model-00060-of-00101.safetensors", + "model.layers.55.mlp.experts.65.gate_proj.weight": "model-00060-of-00101.safetensors", + "model.layers.55.mlp.experts.65.up_proj.weight": "model-00060-of-00101.safetensors", + "model.layers.55.mlp.experts.65.down_proj.weight": "model-00060-of-00101.safetensors", + "model.layers.55.mlp.experts.66.gate_proj.weight": "model-00060-of-00101.safetensors", + "model.layers.55.mlp.experts.66.up_proj.weight": "model-00060-of-00101.safetensors", + "model.layers.55.mlp.experts.66.down_proj.weight": "model-00060-of-00101.safetensors", + "model.layers.55.mlp.experts.67.gate_proj.weight": "model-00060-of-00101.safetensors", + "model.layers.55.mlp.experts.67.up_proj.weight": "model-00060-of-00101.safetensors", + "model.layers.55.mlp.experts.67.down_proj.weight": "model-00060-of-00101.safetensors", + "model.layers.55.mlp.experts.68.gate_proj.weight": "model-00060-of-00101.safetensors", + "model.layers.55.mlp.experts.68.up_proj.weight": "model-00060-of-00101.safetensors", + "model.layers.55.mlp.experts.68.down_proj.weight": "model-00060-of-00101.safetensors", + "model.layers.55.mlp.experts.69.gate_proj.weight": "model-00060-of-00101.safetensors", + "model.layers.55.mlp.experts.69.up_proj.weight": "model-00060-of-00101.safetensors", + "model.layers.55.mlp.experts.69.down_proj.weight": "model-00060-of-00101.safetensors", + "model.layers.55.mlp.experts.70.gate_proj.weight": "model-00060-of-00101.safetensors", + "model.layers.55.mlp.experts.70.up_proj.weight": "model-00060-of-00101.safetensors", + "model.layers.55.mlp.experts.70.down_proj.weight": "model-00060-of-00101.safetensors", + "model.layers.55.mlp.experts.71.gate_proj.weight": "model-00060-of-00101.safetensors", + "model.layers.55.mlp.experts.71.up_proj.weight": "model-00060-of-00101.safetensors", + "model.layers.55.mlp.experts.71.down_proj.weight": "model-00060-of-00101.safetensors", + "model.layers.55.mlp.experts.72.gate_proj.weight": "model-00060-of-00101.safetensors", + "model.layers.55.mlp.experts.72.up_proj.weight": "model-00060-of-00101.safetensors", + "model.layers.55.mlp.experts.72.down_proj.weight": "model-00060-of-00101.safetensors", + "model.layers.55.mlp.experts.73.gate_proj.weight": "model-00060-of-00101.safetensors", + "model.layers.55.mlp.experts.73.up_proj.weight": "model-00060-of-00101.safetensors", + "model.layers.55.mlp.experts.73.down_proj.weight": "model-00060-of-00101.safetensors", + "model.layers.55.mlp.experts.74.gate_proj.weight": "model-00060-of-00101.safetensors", + "model.layers.55.mlp.experts.74.up_proj.weight": "model-00060-of-00101.safetensors", + "model.layers.55.mlp.experts.74.down_proj.weight": "model-00060-of-00101.safetensors", + "model.layers.55.mlp.experts.75.gate_proj.weight": "model-00060-of-00101.safetensors", + "model.layers.55.mlp.experts.75.up_proj.weight": "model-00060-of-00101.safetensors", + "model.layers.55.mlp.experts.75.down_proj.weight": "model-00060-of-00101.safetensors", + "model.layers.55.mlp.experts.76.gate_proj.weight": "model-00060-of-00101.safetensors", + "model.layers.55.mlp.experts.76.up_proj.weight": "model-00060-of-00101.safetensors", + "model.layers.55.mlp.experts.76.down_proj.weight": "model-00060-of-00101.safetensors", + "model.layers.55.mlp.experts.77.gate_proj.weight": "model-00060-of-00101.safetensors", + "model.layers.55.mlp.experts.77.up_proj.weight": "model-00060-of-00101.safetensors", + "model.layers.55.mlp.experts.77.down_proj.weight": "model-00060-of-00101.safetensors", + "model.layers.55.mlp.experts.78.gate_proj.weight": "model-00060-of-00101.safetensors", + "model.layers.55.mlp.experts.78.up_proj.weight": "model-00060-of-00101.safetensors", + "model.layers.55.mlp.experts.78.down_proj.weight": "model-00060-of-00101.safetensors", + "model.layers.55.mlp.experts.79.gate_proj.weight": "model-00060-of-00101.safetensors", + "model.layers.55.mlp.experts.79.up_proj.weight": "model-00060-of-00101.safetensors", + "model.layers.55.mlp.experts.79.down_proj.weight": "model-00060-of-00101.safetensors", + "model.layers.55.mlp.experts.80.gate_proj.weight": "model-00060-of-00101.safetensors", + "model.layers.55.mlp.experts.80.up_proj.weight": "model-00060-of-00101.safetensors", + "model.layers.55.mlp.experts.80.down_proj.weight": "model-00060-of-00101.safetensors", + "model.layers.55.mlp.experts.81.gate_proj.weight": "model-00060-of-00101.safetensors", + "model.layers.55.mlp.experts.81.up_proj.weight": "model-00060-of-00101.safetensors", + "model.layers.55.mlp.experts.81.down_proj.weight": "model-00060-of-00101.safetensors", + "model.layers.55.mlp.experts.82.gate_proj.weight": "model-00060-of-00101.safetensors", + "model.layers.55.mlp.experts.82.up_proj.weight": "model-00060-of-00101.safetensors", + "model.layers.55.mlp.experts.82.down_proj.weight": "model-00060-of-00101.safetensors", + "model.layers.55.mlp.experts.83.gate_proj.weight": "model-00060-of-00101.safetensors", + "model.layers.55.mlp.experts.83.up_proj.weight": "model-00060-of-00101.safetensors", + "model.layers.55.mlp.experts.83.down_proj.weight": "model-00060-of-00101.safetensors", + "model.layers.55.mlp.experts.84.gate_proj.weight": "model-00060-of-00101.safetensors", + "model.layers.55.mlp.experts.84.up_proj.weight": "model-00060-of-00101.safetensors", + "model.layers.55.mlp.experts.84.down_proj.weight": "model-00060-of-00101.safetensors", + "model.layers.55.mlp.experts.85.gate_proj.weight": "model-00060-of-00101.safetensors", + "model.layers.55.mlp.experts.85.up_proj.weight": "model-00060-of-00101.safetensors", + "model.layers.55.mlp.experts.85.down_proj.weight": "model-00060-of-00101.safetensors", + "model.layers.55.mlp.experts.86.gate_proj.weight": "model-00060-of-00101.safetensors", + "model.layers.55.mlp.experts.86.up_proj.weight": "model-00060-of-00101.safetensors", + "model.layers.55.mlp.experts.86.down_proj.weight": "model-00060-of-00101.safetensors", + "model.layers.55.mlp.experts.87.gate_proj.weight": "model-00060-of-00101.safetensors", + "model.layers.55.mlp.experts.87.up_proj.weight": "model-00060-of-00101.safetensors", + "model.layers.55.mlp.experts.87.down_proj.weight": "model-00060-of-00101.safetensors", + "model.layers.55.mlp.experts.88.gate_proj.weight": "model-00060-of-00101.safetensors", + "model.layers.55.mlp.experts.88.up_proj.weight": "model-00060-of-00101.safetensors", + "model.layers.55.mlp.experts.88.down_proj.weight": "model-00060-of-00101.safetensors", + "model.layers.55.mlp.experts.89.gate_proj.weight": "model-00060-of-00101.safetensors", + "model.layers.55.mlp.experts.89.up_proj.weight": "model-00060-of-00101.safetensors", + "model.layers.55.mlp.experts.89.down_proj.weight": "model-00060-of-00101.safetensors", + "model.layers.55.mlp.experts.90.gate_proj.weight": "model-00060-of-00101.safetensors", + "model.layers.55.mlp.experts.90.up_proj.weight": "model-00060-of-00101.safetensors", + "model.layers.55.mlp.experts.90.down_proj.weight": "model-00060-of-00101.safetensors", + "model.layers.55.mlp.experts.91.gate_proj.weight": "model-00060-of-00101.safetensors", + "model.layers.55.mlp.experts.91.up_proj.weight": "model-00060-of-00101.safetensors", + "model.layers.55.mlp.experts.91.down_proj.weight": "model-00060-of-00101.safetensors", + "model.layers.55.mlp.experts.92.gate_proj.weight": "model-00060-of-00101.safetensors", + "model.layers.55.mlp.experts.92.up_proj.weight": "model-00060-of-00101.safetensors", + "model.layers.55.mlp.experts.92.down_proj.weight": "model-00060-of-00101.safetensors", + "model.layers.55.mlp.experts.93.gate_proj.weight": "model-00060-of-00101.safetensors", + "model.layers.55.mlp.experts.93.up_proj.weight": "model-00060-of-00101.safetensors", + "model.layers.55.mlp.experts.93.down_proj.weight": "model-00060-of-00101.safetensors", + "model.layers.55.mlp.experts.94.gate_proj.weight": "model-00060-of-00101.safetensors", + "model.layers.55.mlp.experts.94.up_proj.weight": "model-00060-of-00101.safetensors", + "model.layers.55.mlp.experts.94.down_proj.weight": "model-00060-of-00101.safetensors", + "model.layers.55.mlp.experts.95.gate_proj.weight": "model-00060-of-00101.safetensors", + "model.layers.55.mlp.experts.95.up_proj.weight": "model-00060-of-00101.safetensors", + "model.layers.55.mlp.experts.95.down_proj.weight": "model-00060-of-00101.safetensors", + "model.layers.55.mlp.experts.96.gate_proj.weight": "model-00060-of-00101.safetensors", + "model.layers.55.mlp.experts.96.up_proj.weight": "model-00060-of-00101.safetensors", + "model.layers.55.mlp.experts.96.down_proj.weight": "model-00060-of-00101.safetensors", + "model.layers.55.mlp.experts.97.gate_proj.weight": "model-00060-of-00101.safetensors", + "model.layers.55.mlp.experts.97.up_proj.weight": "model-00060-of-00101.safetensors", + "model.layers.55.mlp.experts.97.down_proj.weight": "model-00060-of-00101.safetensors", + "model.layers.55.mlp.experts.98.gate_proj.weight": "model-00060-of-00101.safetensors", + "model.layers.55.mlp.experts.98.up_proj.weight": "model-00060-of-00101.safetensors", + "model.layers.55.mlp.experts.98.down_proj.weight": "model-00060-of-00101.safetensors", + "model.layers.55.mlp.experts.99.gate_proj.weight": "model-00060-of-00101.safetensors", + "model.layers.55.mlp.experts.99.up_proj.weight": "model-00060-of-00101.safetensors", + "model.layers.55.mlp.experts.99.down_proj.weight": "model-00060-of-00101.safetensors", + "model.layers.55.mlp.experts.100.gate_proj.weight": "model-00060-of-00101.safetensors", + "model.layers.55.mlp.experts.100.up_proj.weight": "model-00060-of-00101.safetensors", + "model.layers.55.mlp.experts.100.down_proj.weight": "model-00060-of-00101.safetensors", + "model.layers.55.mlp.experts.101.gate_proj.weight": "model-00060-of-00101.safetensors", + "model.layers.55.mlp.experts.101.up_proj.weight": "model-00060-of-00101.safetensors", + "model.layers.55.mlp.experts.101.down_proj.weight": "model-00060-of-00101.safetensors", + "model.layers.55.mlp.experts.102.gate_proj.weight": "model-00060-of-00101.safetensors", + "model.layers.55.mlp.experts.102.up_proj.weight": "model-00060-of-00101.safetensors", + "model.layers.55.mlp.experts.102.down_proj.weight": "model-00060-of-00101.safetensors", + "model.layers.55.mlp.experts.103.gate_proj.weight": "model-00060-of-00101.safetensors", + "model.layers.55.mlp.experts.103.up_proj.weight": "model-00060-of-00101.safetensors", + "model.layers.55.mlp.experts.103.down_proj.weight": "model-00060-of-00101.safetensors", + "model.layers.55.mlp.experts.104.gate_proj.weight": "model-00060-of-00101.safetensors", + "model.layers.55.mlp.experts.104.up_proj.weight": "model-00060-of-00101.safetensors", + "model.layers.55.mlp.experts.104.down_proj.weight": "model-00060-of-00101.safetensors", + "model.layers.55.mlp.experts.105.gate_proj.weight": "model-00060-of-00101.safetensors", + "model.layers.55.mlp.experts.105.up_proj.weight": "model-00060-of-00101.safetensors", + "model.layers.55.mlp.experts.105.down_proj.weight": "model-00060-of-00101.safetensors", + "model.layers.55.mlp.experts.106.gate_proj.weight": "model-00060-of-00101.safetensors", + "model.layers.55.mlp.experts.106.up_proj.weight": "model-00060-of-00101.safetensors", + "model.layers.55.mlp.experts.106.down_proj.weight": "model-00060-of-00101.safetensors", + "model.layers.55.mlp.experts.107.gate_proj.weight": "model-00060-of-00101.safetensors", + "model.layers.55.mlp.experts.107.up_proj.weight": "model-00060-of-00101.safetensors", + "model.layers.55.mlp.experts.107.down_proj.weight": "model-00060-of-00101.safetensors", + "model.layers.55.mlp.experts.108.gate_proj.weight": "model-00060-of-00101.safetensors", + "model.layers.55.mlp.experts.108.up_proj.weight": "model-00060-of-00101.safetensors", + "model.layers.55.mlp.experts.108.down_proj.weight": "model-00060-of-00101.safetensors", + "model.layers.55.mlp.experts.109.gate_proj.weight": "model-00060-of-00101.safetensors", + "model.layers.55.mlp.experts.109.up_proj.weight": "model-00060-of-00101.safetensors", + "model.layers.55.mlp.experts.109.down_proj.weight": "model-00060-of-00101.safetensors", + "model.layers.55.mlp.experts.110.gate_proj.weight": "model-00060-of-00101.safetensors", + "model.layers.55.mlp.experts.110.up_proj.weight": "model-00060-of-00101.safetensors", + "model.layers.55.mlp.experts.110.down_proj.weight": "model-00060-of-00101.safetensors", + "model.layers.55.mlp.experts.111.gate_proj.weight": "model-00060-of-00101.safetensors", + "model.layers.55.mlp.experts.111.up_proj.weight": "model-00060-of-00101.safetensors", + "model.layers.55.mlp.experts.111.down_proj.weight": "model-00060-of-00101.safetensors", + "model.layers.55.mlp.experts.112.gate_proj.weight": "model-00060-of-00101.safetensors", + "model.layers.55.mlp.experts.112.up_proj.weight": "model-00060-of-00101.safetensors", + "model.layers.55.mlp.experts.112.down_proj.weight": "model-00060-of-00101.safetensors", + "model.layers.55.mlp.experts.113.gate_proj.weight": "model-00060-of-00101.safetensors", + "model.layers.55.mlp.experts.113.up_proj.weight": "model-00060-of-00101.safetensors", + "model.layers.55.mlp.experts.113.down_proj.weight": "model-00060-of-00101.safetensors", + "model.layers.55.mlp.experts.114.gate_proj.weight": "model-00060-of-00101.safetensors", + "model.layers.55.mlp.experts.114.up_proj.weight": "model-00060-of-00101.safetensors", + "model.layers.55.mlp.experts.114.down_proj.weight": "model-00060-of-00101.safetensors", + "model.layers.55.mlp.experts.115.gate_proj.weight": "model-00060-of-00101.safetensors", + "model.layers.55.mlp.experts.115.up_proj.weight": "model-00060-of-00101.safetensors", + "model.layers.55.mlp.experts.115.down_proj.weight": "model-00060-of-00101.safetensors", + "model.layers.55.mlp.experts.116.gate_proj.weight": "model-00060-of-00101.safetensors", + "model.layers.55.mlp.experts.116.up_proj.weight": "model-00060-of-00101.safetensors", + "model.layers.55.mlp.experts.116.down_proj.weight": "model-00060-of-00101.safetensors", + "model.layers.55.mlp.experts.117.gate_proj.weight": "model-00060-of-00101.safetensors", + "model.layers.55.mlp.experts.117.up_proj.weight": "model-00060-of-00101.safetensors", + "model.layers.55.mlp.experts.117.down_proj.weight": "model-00060-of-00101.safetensors", + "model.layers.55.mlp.experts.118.gate_proj.weight": "model-00060-of-00101.safetensors", + "model.layers.55.mlp.experts.118.up_proj.weight": "model-00060-of-00101.safetensors", + "model.layers.55.mlp.experts.118.down_proj.weight": "model-00060-of-00101.safetensors", + "model.layers.55.mlp.experts.119.gate_proj.weight": "model-00060-of-00101.safetensors", + "model.layers.55.mlp.experts.119.up_proj.weight": "model-00060-of-00101.safetensors", + "model.layers.55.mlp.experts.119.down_proj.weight": "model-00060-of-00101.safetensors", + "model.layers.55.mlp.gate.weight": "model-00060-of-00101.safetensors", + "model.layers.55.mlp.gate.e_score_correction_bias": "model-00060-of-00101.safetensors", + "model.layers.55.mlp.shared_experts.gate_proj.weight": "model-00060-of-00101.safetensors", + "model.layers.55.mlp.shared_experts.up_proj.weight": "model-00060-of-00101.safetensors", + "model.layers.55.mlp.shared_experts.down_proj.weight": "model-00060-of-00101.safetensors", + "model.layers.55.input_layernorm.weight": "model-00060-of-00101.safetensors", + "model.layers.55.post_attention_layernorm.weight": "model-00060-of-00101.safetensors", + "model.layers.56.self_attn.q_proj.weight": "model-00060-of-00101.safetensors", + "model.layers.56.self_attn.q_proj.bias": "model-00060-of-00101.safetensors", + "model.layers.56.self_attn.k_proj.weight": "model-00060-of-00101.safetensors", + "model.layers.56.self_attn.k_proj.bias": "model-00060-of-00101.safetensors", + "model.layers.56.self_attn.v_proj.weight": "model-00060-of-00101.safetensors", + "model.layers.56.self_attn.v_proj.bias": "model-00060-of-00101.safetensors", + "model.layers.56.self_attn.o_proj.weight": "model-00060-of-00101.safetensors", + "model.layers.56.self_attn.q_norm.weight": "model-00060-of-00101.safetensors", + "model.layers.56.self_attn.k_norm.weight": "model-00060-of-00101.safetensors", + "model.layers.56.mlp.experts.0.gate_proj.weight": "model-00060-of-00101.safetensors", + "model.layers.56.mlp.experts.0.up_proj.weight": "model-00060-of-00101.safetensors", + "model.layers.56.mlp.experts.0.down_proj.weight": "model-00060-of-00101.safetensors", + "model.layers.56.mlp.experts.1.gate_proj.weight": "model-00060-of-00101.safetensors", + "model.layers.56.mlp.experts.1.up_proj.weight": "model-00060-of-00101.safetensors", + "model.layers.56.mlp.experts.1.down_proj.weight": "model-00060-of-00101.safetensors", + "model.layers.56.mlp.experts.2.gate_proj.weight": "model-00060-of-00101.safetensors", + "model.layers.56.mlp.experts.2.up_proj.weight": "model-00060-of-00101.safetensors", + "model.layers.56.mlp.experts.2.down_proj.weight": "model-00060-of-00101.safetensors", + "model.layers.56.mlp.experts.3.gate_proj.weight": "model-00060-of-00101.safetensors", + "model.layers.56.mlp.experts.3.up_proj.weight": "model-00060-of-00101.safetensors", + "model.layers.56.mlp.experts.3.down_proj.weight": "model-00060-of-00101.safetensors", + "model.layers.56.mlp.experts.4.gate_proj.weight": "model-00060-of-00101.safetensors", + "model.layers.56.mlp.experts.4.up_proj.weight": "model-00060-of-00101.safetensors", + "model.layers.56.mlp.experts.4.down_proj.weight": "model-00060-of-00101.safetensors", + "model.layers.56.mlp.experts.5.gate_proj.weight": "model-00060-of-00101.safetensors", + "model.layers.56.mlp.experts.5.up_proj.weight": "model-00060-of-00101.safetensors", + "model.layers.56.mlp.experts.5.down_proj.weight": "model-00060-of-00101.safetensors", + "model.layers.56.mlp.experts.6.gate_proj.weight": "model-00060-of-00101.safetensors", + "model.layers.56.mlp.experts.6.up_proj.weight": "model-00060-of-00101.safetensors", + "model.layers.56.mlp.experts.6.down_proj.weight": "model-00060-of-00101.safetensors", + "model.layers.56.mlp.experts.7.gate_proj.weight": "model-00060-of-00101.safetensors", + "model.layers.56.mlp.experts.7.up_proj.weight": "model-00060-of-00101.safetensors", + "model.layers.56.mlp.experts.7.down_proj.weight": "model-00060-of-00101.safetensors", + "model.layers.56.mlp.experts.8.gate_proj.weight": "model-00061-of-00101.safetensors", + "model.layers.56.mlp.experts.8.up_proj.weight": "model-00061-of-00101.safetensors", + "model.layers.56.mlp.experts.8.down_proj.weight": "model-00061-of-00101.safetensors", + "model.layers.56.mlp.experts.9.gate_proj.weight": "model-00061-of-00101.safetensors", + "model.layers.56.mlp.experts.9.up_proj.weight": "model-00061-of-00101.safetensors", + "model.layers.56.mlp.experts.9.down_proj.weight": "model-00061-of-00101.safetensors", + "model.layers.56.mlp.experts.10.gate_proj.weight": "model-00061-of-00101.safetensors", + "model.layers.56.mlp.experts.10.up_proj.weight": "model-00061-of-00101.safetensors", + "model.layers.56.mlp.experts.10.down_proj.weight": "model-00061-of-00101.safetensors", + "model.layers.56.mlp.experts.11.gate_proj.weight": "model-00061-of-00101.safetensors", + "model.layers.56.mlp.experts.11.up_proj.weight": "model-00061-of-00101.safetensors", + "model.layers.56.mlp.experts.11.down_proj.weight": "model-00061-of-00101.safetensors", + "model.layers.56.mlp.experts.12.gate_proj.weight": "model-00061-of-00101.safetensors", + "model.layers.56.mlp.experts.12.up_proj.weight": "model-00061-of-00101.safetensors", + "model.layers.56.mlp.experts.12.down_proj.weight": "model-00061-of-00101.safetensors", + "model.layers.56.mlp.experts.13.gate_proj.weight": "model-00061-of-00101.safetensors", + "model.layers.56.mlp.experts.13.up_proj.weight": "model-00061-of-00101.safetensors", + "model.layers.56.mlp.experts.13.down_proj.weight": "model-00061-of-00101.safetensors", + "model.layers.56.mlp.experts.14.gate_proj.weight": "model-00061-of-00101.safetensors", + "model.layers.56.mlp.experts.14.up_proj.weight": "model-00061-of-00101.safetensors", + "model.layers.56.mlp.experts.14.down_proj.weight": "model-00061-of-00101.safetensors", + "model.layers.56.mlp.experts.15.gate_proj.weight": "model-00061-of-00101.safetensors", + "model.layers.56.mlp.experts.15.up_proj.weight": "model-00061-of-00101.safetensors", + "model.layers.56.mlp.experts.15.down_proj.weight": "model-00061-of-00101.safetensors", + "model.layers.56.mlp.experts.16.gate_proj.weight": "model-00061-of-00101.safetensors", + "model.layers.56.mlp.experts.16.up_proj.weight": "model-00061-of-00101.safetensors", + "model.layers.56.mlp.experts.16.down_proj.weight": "model-00061-of-00101.safetensors", + "model.layers.56.mlp.experts.17.gate_proj.weight": "model-00061-of-00101.safetensors", + "model.layers.56.mlp.experts.17.up_proj.weight": "model-00061-of-00101.safetensors", + "model.layers.56.mlp.experts.17.down_proj.weight": "model-00061-of-00101.safetensors", + "model.layers.56.mlp.experts.18.gate_proj.weight": "model-00061-of-00101.safetensors", + "model.layers.56.mlp.experts.18.up_proj.weight": "model-00061-of-00101.safetensors", + "model.layers.56.mlp.experts.18.down_proj.weight": "model-00061-of-00101.safetensors", + "model.layers.56.mlp.experts.19.gate_proj.weight": "model-00061-of-00101.safetensors", + "model.layers.56.mlp.experts.19.up_proj.weight": "model-00061-of-00101.safetensors", + "model.layers.56.mlp.experts.19.down_proj.weight": "model-00061-of-00101.safetensors", + "model.layers.56.mlp.experts.20.gate_proj.weight": "model-00061-of-00101.safetensors", + "model.layers.56.mlp.experts.20.up_proj.weight": "model-00061-of-00101.safetensors", + "model.layers.56.mlp.experts.20.down_proj.weight": "model-00061-of-00101.safetensors", + "model.layers.56.mlp.experts.21.gate_proj.weight": "model-00061-of-00101.safetensors", + "model.layers.56.mlp.experts.21.up_proj.weight": "model-00061-of-00101.safetensors", + "model.layers.56.mlp.experts.21.down_proj.weight": "model-00061-of-00101.safetensors", + "model.layers.56.mlp.experts.22.gate_proj.weight": "model-00061-of-00101.safetensors", + "model.layers.56.mlp.experts.22.up_proj.weight": "model-00061-of-00101.safetensors", + "model.layers.56.mlp.experts.22.down_proj.weight": "model-00061-of-00101.safetensors", + "model.layers.56.mlp.experts.23.gate_proj.weight": "model-00061-of-00101.safetensors", + "model.layers.56.mlp.experts.23.up_proj.weight": "model-00061-of-00101.safetensors", + "model.layers.56.mlp.experts.23.down_proj.weight": "model-00061-of-00101.safetensors", + "model.layers.56.mlp.experts.24.gate_proj.weight": "model-00061-of-00101.safetensors", + "model.layers.56.mlp.experts.24.up_proj.weight": "model-00061-of-00101.safetensors", + "model.layers.56.mlp.experts.24.down_proj.weight": "model-00061-of-00101.safetensors", + "model.layers.56.mlp.experts.25.gate_proj.weight": "model-00061-of-00101.safetensors", + "model.layers.56.mlp.experts.25.up_proj.weight": "model-00061-of-00101.safetensors", + "model.layers.56.mlp.experts.25.down_proj.weight": "model-00061-of-00101.safetensors", + "model.layers.56.mlp.experts.26.gate_proj.weight": "model-00061-of-00101.safetensors", + "model.layers.56.mlp.experts.26.up_proj.weight": "model-00061-of-00101.safetensors", + "model.layers.56.mlp.experts.26.down_proj.weight": "model-00061-of-00101.safetensors", + "model.layers.56.mlp.experts.27.gate_proj.weight": "model-00061-of-00101.safetensors", + "model.layers.56.mlp.experts.27.up_proj.weight": "model-00061-of-00101.safetensors", + "model.layers.56.mlp.experts.27.down_proj.weight": "model-00061-of-00101.safetensors", + "model.layers.56.mlp.experts.28.gate_proj.weight": "model-00061-of-00101.safetensors", + "model.layers.56.mlp.experts.28.up_proj.weight": "model-00061-of-00101.safetensors", + "model.layers.56.mlp.experts.28.down_proj.weight": "model-00061-of-00101.safetensors", + "model.layers.56.mlp.experts.29.gate_proj.weight": "model-00061-of-00101.safetensors", + "model.layers.56.mlp.experts.29.up_proj.weight": "model-00061-of-00101.safetensors", + "model.layers.56.mlp.experts.29.down_proj.weight": "model-00061-of-00101.safetensors", + "model.layers.56.mlp.experts.30.gate_proj.weight": "model-00061-of-00101.safetensors", + "model.layers.56.mlp.experts.30.up_proj.weight": "model-00061-of-00101.safetensors", + "model.layers.56.mlp.experts.30.down_proj.weight": "model-00061-of-00101.safetensors", + "model.layers.56.mlp.experts.31.gate_proj.weight": "model-00061-of-00101.safetensors", + "model.layers.56.mlp.experts.31.up_proj.weight": "model-00061-of-00101.safetensors", + "model.layers.56.mlp.experts.31.down_proj.weight": "model-00061-of-00101.safetensors", + "model.layers.56.mlp.experts.32.gate_proj.weight": "model-00061-of-00101.safetensors", + "model.layers.56.mlp.experts.32.up_proj.weight": "model-00061-of-00101.safetensors", + "model.layers.56.mlp.experts.32.down_proj.weight": "model-00061-of-00101.safetensors", + "model.layers.56.mlp.experts.33.gate_proj.weight": "model-00061-of-00101.safetensors", + "model.layers.56.mlp.experts.33.up_proj.weight": "model-00061-of-00101.safetensors", + "model.layers.56.mlp.experts.33.down_proj.weight": "model-00061-of-00101.safetensors", + "model.layers.56.mlp.experts.34.gate_proj.weight": "model-00061-of-00101.safetensors", + "model.layers.56.mlp.experts.34.up_proj.weight": "model-00061-of-00101.safetensors", + "model.layers.56.mlp.experts.34.down_proj.weight": "model-00061-of-00101.safetensors", + "model.layers.56.mlp.experts.35.gate_proj.weight": "model-00061-of-00101.safetensors", + "model.layers.56.mlp.experts.35.up_proj.weight": "model-00061-of-00101.safetensors", + "model.layers.56.mlp.experts.35.down_proj.weight": "model-00061-of-00101.safetensors", + "model.layers.56.mlp.experts.36.gate_proj.weight": "model-00061-of-00101.safetensors", + "model.layers.56.mlp.experts.36.up_proj.weight": "model-00061-of-00101.safetensors", + "model.layers.56.mlp.experts.36.down_proj.weight": "model-00061-of-00101.safetensors", + "model.layers.56.mlp.experts.37.gate_proj.weight": "model-00061-of-00101.safetensors", + "model.layers.56.mlp.experts.37.up_proj.weight": "model-00061-of-00101.safetensors", + "model.layers.56.mlp.experts.37.down_proj.weight": "model-00061-of-00101.safetensors", + "model.layers.56.mlp.experts.38.gate_proj.weight": "model-00061-of-00101.safetensors", + "model.layers.56.mlp.experts.38.up_proj.weight": "model-00061-of-00101.safetensors", + "model.layers.56.mlp.experts.38.down_proj.weight": "model-00061-of-00101.safetensors", + "model.layers.56.mlp.experts.39.gate_proj.weight": "model-00061-of-00101.safetensors", + "model.layers.56.mlp.experts.39.up_proj.weight": "model-00061-of-00101.safetensors", + "model.layers.56.mlp.experts.39.down_proj.weight": "model-00061-of-00101.safetensors", + "model.layers.56.mlp.experts.40.gate_proj.weight": "model-00061-of-00101.safetensors", + "model.layers.56.mlp.experts.40.up_proj.weight": "model-00061-of-00101.safetensors", + "model.layers.56.mlp.experts.40.down_proj.weight": "model-00061-of-00101.safetensors", + "model.layers.56.mlp.experts.41.gate_proj.weight": "model-00061-of-00101.safetensors", + "model.layers.56.mlp.experts.41.up_proj.weight": "model-00061-of-00101.safetensors", + "model.layers.56.mlp.experts.41.down_proj.weight": "model-00061-of-00101.safetensors", + "model.layers.56.mlp.experts.42.gate_proj.weight": "model-00061-of-00101.safetensors", + "model.layers.56.mlp.experts.42.up_proj.weight": "model-00061-of-00101.safetensors", + "model.layers.56.mlp.experts.42.down_proj.weight": "model-00061-of-00101.safetensors", + "model.layers.56.mlp.experts.43.gate_proj.weight": "model-00061-of-00101.safetensors", + "model.layers.56.mlp.experts.43.up_proj.weight": "model-00061-of-00101.safetensors", + "model.layers.56.mlp.experts.43.down_proj.weight": "model-00061-of-00101.safetensors", + "model.layers.56.mlp.experts.44.gate_proj.weight": "model-00061-of-00101.safetensors", + "model.layers.56.mlp.experts.44.up_proj.weight": "model-00061-of-00101.safetensors", + "model.layers.56.mlp.experts.44.down_proj.weight": "model-00061-of-00101.safetensors", + "model.layers.56.mlp.experts.45.gate_proj.weight": "model-00061-of-00101.safetensors", + "model.layers.56.mlp.experts.45.up_proj.weight": "model-00061-of-00101.safetensors", + "model.layers.56.mlp.experts.45.down_proj.weight": "model-00061-of-00101.safetensors", + "model.layers.56.mlp.experts.46.gate_proj.weight": "model-00061-of-00101.safetensors", + "model.layers.56.mlp.experts.46.up_proj.weight": "model-00061-of-00101.safetensors", + "model.layers.56.mlp.experts.46.down_proj.weight": "model-00061-of-00101.safetensors", + "model.layers.56.mlp.experts.47.gate_proj.weight": "model-00061-of-00101.safetensors", + "model.layers.56.mlp.experts.47.up_proj.weight": "model-00061-of-00101.safetensors", + "model.layers.56.mlp.experts.47.down_proj.weight": "model-00061-of-00101.safetensors", + "model.layers.56.mlp.experts.48.gate_proj.weight": "model-00061-of-00101.safetensors", + "model.layers.56.mlp.experts.48.up_proj.weight": "model-00061-of-00101.safetensors", + "model.layers.56.mlp.experts.48.down_proj.weight": "model-00061-of-00101.safetensors", + "model.layers.56.mlp.experts.49.gate_proj.weight": "model-00061-of-00101.safetensors", + "model.layers.56.mlp.experts.49.up_proj.weight": "model-00061-of-00101.safetensors", + "model.layers.56.mlp.experts.49.down_proj.weight": "model-00061-of-00101.safetensors", + "model.layers.56.mlp.experts.50.gate_proj.weight": "model-00061-of-00101.safetensors", + "model.layers.56.mlp.experts.50.up_proj.weight": "model-00061-of-00101.safetensors", + "model.layers.56.mlp.experts.50.down_proj.weight": "model-00061-of-00101.safetensors", + "model.layers.56.mlp.experts.51.gate_proj.weight": "model-00061-of-00101.safetensors", + "model.layers.56.mlp.experts.51.up_proj.weight": "model-00061-of-00101.safetensors", + "model.layers.56.mlp.experts.51.down_proj.weight": "model-00061-of-00101.safetensors", + "model.layers.56.mlp.experts.52.gate_proj.weight": "model-00061-of-00101.safetensors", + "model.layers.56.mlp.experts.52.up_proj.weight": "model-00061-of-00101.safetensors", + "model.layers.56.mlp.experts.52.down_proj.weight": "model-00061-of-00101.safetensors", + "model.layers.56.mlp.experts.53.gate_proj.weight": "model-00061-of-00101.safetensors", + "model.layers.56.mlp.experts.53.up_proj.weight": "model-00061-of-00101.safetensors", + "model.layers.56.mlp.experts.53.down_proj.weight": "model-00061-of-00101.safetensors", + "model.layers.56.mlp.experts.54.gate_proj.weight": "model-00061-of-00101.safetensors", + "model.layers.56.mlp.experts.54.up_proj.weight": "model-00061-of-00101.safetensors", + "model.layers.56.mlp.experts.54.down_proj.weight": "model-00061-of-00101.safetensors", + "model.layers.56.mlp.experts.55.gate_proj.weight": "model-00061-of-00101.safetensors", + "model.layers.56.mlp.experts.55.up_proj.weight": "model-00061-of-00101.safetensors", + "model.layers.56.mlp.experts.55.down_proj.weight": "model-00061-of-00101.safetensors", + "model.layers.56.mlp.experts.56.gate_proj.weight": "model-00061-of-00101.safetensors", + "model.layers.56.mlp.experts.56.up_proj.weight": "model-00061-of-00101.safetensors", + "model.layers.56.mlp.experts.56.down_proj.weight": "model-00061-of-00101.safetensors", + "model.layers.56.mlp.experts.57.gate_proj.weight": "model-00061-of-00101.safetensors", + "model.layers.56.mlp.experts.57.up_proj.weight": "model-00061-of-00101.safetensors", + "model.layers.56.mlp.experts.57.down_proj.weight": "model-00061-of-00101.safetensors", + "model.layers.56.mlp.experts.58.gate_proj.weight": "model-00061-of-00101.safetensors", + "model.layers.56.mlp.experts.58.up_proj.weight": "model-00061-of-00101.safetensors", + "model.layers.56.mlp.experts.58.down_proj.weight": "model-00061-of-00101.safetensors", + "model.layers.56.mlp.experts.59.gate_proj.weight": "model-00061-of-00101.safetensors", + "model.layers.56.mlp.experts.59.up_proj.weight": "model-00061-of-00101.safetensors", + "model.layers.56.mlp.experts.59.down_proj.weight": "model-00061-of-00101.safetensors", + "model.layers.56.mlp.experts.60.gate_proj.weight": "model-00061-of-00101.safetensors", + "model.layers.56.mlp.experts.60.up_proj.weight": "model-00061-of-00101.safetensors", + "model.layers.56.mlp.experts.60.down_proj.weight": "model-00061-of-00101.safetensors", + "model.layers.56.mlp.experts.61.gate_proj.weight": "model-00061-of-00101.safetensors", + "model.layers.56.mlp.experts.61.up_proj.weight": "model-00061-of-00101.safetensors", + "model.layers.56.mlp.experts.61.down_proj.weight": "model-00061-of-00101.safetensors", + "model.layers.56.mlp.experts.62.gate_proj.weight": "model-00061-of-00101.safetensors", + "model.layers.56.mlp.experts.62.up_proj.weight": "model-00061-of-00101.safetensors", + "model.layers.56.mlp.experts.62.down_proj.weight": "model-00061-of-00101.safetensors", + "model.layers.56.mlp.experts.63.gate_proj.weight": "model-00061-of-00101.safetensors", + "model.layers.56.mlp.experts.63.up_proj.weight": "model-00061-of-00101.safetensors", + "model.layers.56.mlp.experts.63.down_proj.weight": "model-00061-of-00101.safetensors", + "model.layers.56.mlp.experts.64.gate_proj.weight": "model-00061-of-00101.safetensors", + "model.layers.56.mlp.experts.64.up_proj.weight": "model-00061-of-00101.safetensors", + "model.layers.56.mlp.experts.64.down_proj.weight": "model-00061-of-00101.safetensors", + "model.layers.56.mlp.experts.65.gate_proj.weight": "model-00061-of-00101.safetensors", + "model.layers.56.mlp.experts.65.up_proj.weight": "model-00061-of-00101.safetensors", + "model.layers.56.mlp.experts.65.down_proj.weight": "model-00061-of-00101.safetensors", + "model.layers.56.mlp.experts.66.gate_proj.weight": "model-00061-of-00101.safetensors", + "model.layers.56.mlp.experts.66.up_proj.weight": "model-00061-of-00101.safetensors", + "model.layers.56.mlp.experts.66.down_proj.weight": "model-00061-of-00101.safetensors", + "model.layers.56.mlp.experts.67.gate_proj.weight": "model-00061-of-00101.safetensors", + "model.layers.56.mlp.experts.67.up_proj.weight": "model-00061-of-00101.safetensors", + "model.layers.56.mlp.experts.67.down_proj.weight": "model-00061-of-00101.safetensors", + "model.layers.56.mlp.experts.68.gate_proj.weight": "model-00061-of-00101.safetensors", + "model.layers.56.mlp.experts.68.up_proj.weight": "model-00061-of-00101.safetensors", + "model.layers.56.mlp.experts.68.down_proj.weight": "model-00061-of-00101.safetensors", + "model.layers.56.mlp.experts.69.gate_proj.weight": "model-00061-of-00101.safetensors", + "model.layers.56.mlp.experts.69.up_proj.weight": "model-00061-of-00101.safetensors", + "model.layers.56.mlp.experts.69.down_proj.weight": "model-00061-of-00101.safetensors", + "model.layers.56.mlp.experts.70.gate_proj.weight": "model-00061-of-00101.safetensors", + "model.layers.56.mlp.experts.70.up_proj.weight": "model-00061-of-00101.safetensors", + "model.layers.56.mlp.experts.70.down_proj.weight": "model-00061-of-00101.safetensors", + "model.layers.56.mlp.experts.71.gate_proj.weight": "model-00061-of-00101.safetensors", + "model.layers.56.mlp.experts.71.up_proj.weight": "model-00061-of-00101.safetensors", + "model.layers.56.mlp.experts.71.down_proj.weight": "model-00061-of-00101.safetensors", + "model.layers.56.mlp.experts.72.gate_proj.weight": "model-00061-of-00101.safetensors", + "model.layers.56.mlp.experts.72.up_proj.weight": "model-00061-of-00101.safetensors", + "model.layers.56.mlp.experts.72.down_proj.weight": "model-00061-of-00101.safetensors", + "model.layers.56.mlp.experts.73.gate_proj.weight": "model-00061-of-00101.safetensors", + "model.layers.56.mlp.experts.73.up_proj.weight": "model-00061-of-00101.safetensors", + "model.layers.56.mlp.experts.73.down_proj.weight": "model-00061-of-00101.safetensors", + "model.layers.56.mlp.experts.74.gate_proj.weight": "model-00061-of-00101.safetensors", + "model.layers.56.mlp.experts.74.up_proj.weight": "model-00061-of-00101.safetensors", + "model.layers.56.mlp.experts.74.down_proj.weight": "model-00061-of-00101.safetensors", + "model.layers.56.mlp.experts.75.gate_proj.weight": "model-00061-of-00101.safetensors", + "model.layers.56.mlp.experts.75.up_proj.weight": "model-00061-of-00101.safetensors", + "model.layers.56.mlp.experts.75.down_proj.weight": "model-00061-of-00101.safetensors", + "model.layers.56.mlp.experts.76.gate_proj.weight": "model-00061-of-00101.safetensors", + "model.layers.56.mlp.experts.76.up_proj.weight": "model-00061-of-00101.safetensors", + "model.layers.56.mlp.experts.76.down_proj.weight": "model-00061-of-00101.safetensors", + "model.layers.56.mlp.experts.77.gate_proj.weight": "model-00061-of-00101.safetensors", + "model.layers.56.mlp.experts.77.up_proj.weight": "model-00061-of-00101.safetensors", + "model.layers.56.mlp.experts.77.down_proj.weight": "model-00061-of-00101.safetensors", + "model.layers.56.mlp.experts.78.gate_proj.weight": "model-00061-of-00101.safetensors", + "model.layers.56.mlp.experts.78.up_proj.weight": "model-00061-of-00101.safetensors", + "model.layers.56.mlp.experts.78.down_proj.weight": "model-00061-of-00101.safetensors", + "model.layers.56.mlp.experts.79.gate_proj.weight": "model-00061-of-00101.safetensors", + "model.layers.56.mlp.experts.79.up_proj.weight": "model-00061-of-00101.safetensors", + "model.layers.56.mlp.experts.79.down_proj.weight": "model-00061-of-00101.safetensors", + "model.layers.56.mlp.experts.80.gate_proj.weight": "model-00061-of-00101.safetensors", + "model.layers.56.mlp.experts.80.up_proj.weight": "model-00061-of-00101.safetensors", + "model.layers.56.mlp.experts.80.down_proj.weight": "model-00061-of-00101.safetensors", + "model.layers.56.mlp.experts.81.gate_proj.weight": "model-00061-of-00101.safetensors", + "model.layers.56.mlp.experts.81.up_proj.weight": "model-00061-of-00101.safetensors", + "model.layers.56.mlp.experts.81.down_proj.weight": "model-00061-of-00101.safetensors", + "model.layers.56.mlp.experts.82.gate_proj.weight": "model-00061-of-00101.safetensors", + "model.layers.56.mlp.experts.82.up_proj.weight": "model-00061-of-00101.safetensors", + "model.layers.56.mlp.experts.82.down_proj.weight": "model-00061-of-00101.safetensors", + "model.layers.56.mlp.experts.83.gate_proj.weight": "model-00061-of-00101.safetensors", + "model.layers.56.mlp.experts.83.up_proj.weight": "model-00061-of-00101.safetensors", + "model.layers.56.mlp.experts.83.down_proj.weight": "model-00061-of-00101.safetensors", + "model.layers.56.mlp.experts.84.gate_proj.weight": "model-00061-of-00101.safetensors", + "model.layers.56.mlp.experts.84.up_proj.weight": "model-00061-of-00101.safetensors", + "model.layers.56.mlp.experts.84.down_proj.weight": "model-00061-of-00101.safetensors", + "model.layers.56.mlp.experts.85.gate_proj.weight": "model-00061-of-00101.safetensors", + "model.layers.56.mlp.experts.85.up_proj.weight": "model-00061-of-00101.safetensors", + "model.layers.56.mlp.experts.85.down_proj.weight": "model-00061-of-00101.safetensors", + "model.layers.56.mlp.experts.86.gate_proj.weight": "model-00061-of-00101.safetensors", + "model.layers.56.mlp.experts.86.up_proj.weight": "model-00061-of-00101.safetensors", + "model.layers.56.mlp.experts.86.down_proj.weight": "model-00061-of-00101.safetensors", + "model.layers.56.mlp.experts.87.gate_proj.weight": "model-00061-of-00101.safetensors", + "model.layers.56.mlp.experts.87.up_proj.weight": "model-00061-of-00101.safetensors", + "model.layers.56.mlp.experts.87.down_proj.weight": "model-00061-of-00101.safetensors", + "model.layers.56.mlp.experts.88.gate_proj.weight": "model-00061-of-00101.safetensors", + "model.layers.56.mlp.experts.88.up_proj.weight": "model-00061-of-00101.safetensors", + "model.layers.56.mlp.experts.88.down_proj.weight": "model-00061-of-00101.safetensors", + "model.layers.56.mlp.experts.89.gate_proj.weight": "model-00061-of-00101.safetensors", + "model.layers.56.mlp.experts.89.up_proj.weight": "model-00061-of-00101.safetensors", + "model.layers.56.mlp.experts.89.down_proj.weight": "model-00061-of-00101.safetensors", + "model.layers.56.mlp.experts.90.gate_proj.weight": "model-00061-of-00101.safetensors", + "model.layers.56.mlp.experts.90.up_proj.weight": "model-00061-of-00101.safetensors", + "model.layers.56.mlp.experts.90.down_proj.weight": "model-00061-of-00101.safetensors", + "model.layers.56.mlp.experts.91.gate_proj.weight": "model-00061-of-00101.safetensors", + "model.layers.56.mlp.experts.91.up_proj.weight": "model-00061-of-00101.safetensors", + "model.layers.56.mlp.experts.91.down_proj.weight": "model-00061-of-00101.safetensors", + "model.layers.56.mlp.experts.92.gate_proj.weight": "model-00061-of-00101.safetensors", + "model.layers.56.mlp.experts.92.up_proj.weight": "model-00061-of-00101.safetensors", + "model.layers.56.mlp.experts.92.down_proj.weight": "model-00061-of-00101.safetensors", + "model.layers.56.mlp.experts.93.gate_proj.weight": "model-00061-of-00101.safetensors", + "model.layers.56.mlp.experts.93.up_proj.weight": "model-00061-of-00101.safetensors", + "model.layers.56.mlp.experts.93.down_proj.weight": "model-00061-of-00101.safetensors", + "model.layers.56.mlp.experts.94.gate_proj.weight": "model-00061-of-00101.safetensors", + "model.layers.56.mlp.experts.94.up_proj.weight": "model-00061-of-00101.safetensors", + "model.layers.56.mlp.experts.94.down_proj.weight": "model-00061-of-00101.safetensors", + "model.layers.56.mlp.experts.95.gate_proj.weight": "model-00061-of-00101.safetensors", + "model.layers.56.mlp.experts.95.up_proj.weight": "model-00061-of-00101.safetensors", + "model.layers.56.mlp.experts.95.down_proj.weight": "model-00061-of-00101.safetensors", + "model.layers.56.mlp.experts.96.gate_proj.weight": "model-00061-of-00101.safetensors", + "model.layers.56.mlp.experts.96.up_proj.weight": "model-00061-of-00101.safetensors", + "model.layers.56.mlp.experts.96.down_proj.weight": "model-00061-of-00101.safetensors", + "model.layers.56.mlp.experts.97.gate_proj.weight": "model-00061-of-00101.safetensors", + "model.layers.56.mlp.experts.97.up_proj.weight": "model-00061-of-00101.safetensors", + "model.layers.56.mlp.experts.97.down_proj.weight": "model-00061-of-00101.safetensors", + "model.layers.56.mlp.experts.98.gate_proj.weight": "model-00061-of-00101.safetensors", + "model.layers.56.mlp.experts.98.up_proj.weight": "model-00061-of-00101.safetensors", + "model.layers.56.mlp.experts.98.down_proj.weight": "model-00061-of-00101.safetensors", + "model.layers.56.mlp.experts.99.gate_proj.weight": "model-00061-of-00101.safetensors", + "model.layers.56.mlp.experts.99.up_proj.weight": "model-00061-of-00101.safetensors", + "model.layers.56.mlp.experts.99.down_proj.weight": "model-00061-of-00101.safetensors", + "model.layers.56.mlp.experts.100.gate_proj.weight": "model-00061-of-00101.safetensors", + "model.layers.56.mlp.experts.100.up_proj.weight": "model-00061-of-00101.safetensors", + "model.layers.56.mlp.experts.100.down_proj.weight": "model-00061-of-00101.safetensors", + "model.layers.56.mlp.experts.101.gate_proj.weight": "model-00061-of-00101.safetensors", + "model.layers.56.mlp.experts.101.up_proj.weight": "model-00061-of-00101.safetensors", + "model.layers.56.mlp.experts.101.down_proj.weight": "model-00061-of-00101.safetensors", + "model.layers.56.mlp.experts.102.gate_proj.weight": "model-00061-of-00101.safetensors", + "model.layers.56.mlp.experts.102.up_proj.weight": "model-00061-of-00101.safetensors", + "model.layers.56.mlp.experts.102.down_proj.weight": "model-00061-of-00101.safetensors", + "model.layers.56.mlp.experts.103.gate_proj.weight": "model-00061-of-00101.safetensors", + "model.layers.56.mlp.experts.103.up_proj.weight": "model-00061-of-00101.safetensors", + "model.layers.56.mlp.experts.103.down_proj.weight": "model-00061-of-00101.safetensors", + "model.layers.56.mlp.experts.104.gate_proj.weight": "model-00061-of-00101.safetensors", + "model.layers.56.mlp.experts.104.up_proj.weight": "model-00061-of-00101.safetensors", + "model.layers.56.mlp.experts.104.down_proj.weight": "model-00061-of-00101.safetensors", + "model.layers.56.mlp.experts.105.gate_proj.weight": "model-00061-of-00101.safetensors", + "model.layers.56.mlp.experts.105.up_proj.weight": "model-00061-of-00101.safetensors", + "model.layers.56.mlp.experts.105.down_proj.weight": "model-00061-of-00101.safetensors", + "model.layers.56.mlp.experts.106.gate_proj.weight": "model-00061-of-00101.safetensors", + "model.layers.56.mlp.experts.106.up_proj.weight": "model-00061-of-00101.safetensors", + "model.layers.56.mlp.experts.106.down_proj.weight": "model-00061-of-00101.safetensors", + "model.layers.56.mlp.experts.107.gate_proj.weight": "model-00061-of-00101.safetensors", + "model.layers.56.mlp.experts.107.up_proj.weight": "model-00061-of-00101.safetensors", + "model.layers.56.mlp.experts.107.down_proj.weight": "model-00061-of-00101.safetensors", + "model.layers.56.mlp.experts.108.gate_proj.weight": "model-00061-of-00101.safetensors", + "model.layers.56.mlp.experts.108.up_proj.weight": "model-00061-of-00101.safetensors", + "model.layers.56.mlp.experts.108.down_proj.weight": "model-00061-of-00101.safetensors", + "model.layers.56.mlp.experts.109.gate_proj.weight": "model-00061-of-00101.safetensors", + "model.layers.56.mlp.experts.109.up_proj.weight": "model-00061-of-00101.safetensors", + "model.layers.56.mlp.experts.109.down_proj.weight": "model-00061-of-00101.safetensors", + "model.layers.56.mlp.experts.110.gate_proj.weight": "model-00061-of-00101.safetensors", + "model.layers.56.mlp.experts.110.up_proj.weight": "model-00061-of-00101.safetensors", + "model.layers.56.mlp.experts.110.down_proj.weight": "model-00061-of-00101.safetensors", + "model.layers.56.mlp.experts.111.gate_proj.weight": "model-00061-of-00101.safetensors", + "model.layers.56.mlp.experts.111.up_proj.weight": "model-00061-of-00101.safetensors", + "model.layers.56.mlp.experts.111.down_proj.weight": "model-00061-of-00101.safetensors", + "model.layers.56.mlp.experts.112.gate_proj.weight": "model-00061-of-00101.safetensors", + "model.layers.56.mlp.experts.112.up_proj.weight": "model-00061-of-00101.safetensors", + "model.layers.56.mlp.experts.112.down_proj.weight": "model-00061-of-00101.safetensors", + "model.layers.56.mlp.experts.113.gate_proj.weight": "model-00061-of-00101.safetensors", + "model.layers.56.mlp.experts.113.up_proj.weight": "model-00061-of-00101.safetensors", + "model.layers.56.mlp.experts.113.down_proj.weight": "model-00061-of-00101.safetensors", + "model.layers.56.mlp.experts.114.gate_proj.weight": "model-00061-of-00101.safetensors", + "model.layers.56.mlp.experts.114.up_proj.weight": "model-00061-of-00101.safetensors", + "model.layers.56.mlp.experts.114.down_proj.weight": "model-00061-of-00101.safetensors", + "model.layers.56.mlp.experts.115.gate_proj.weight": "model-00061-of-00101.safetensors", + "model.layers.56.mlp.experts.115.up_proj.weight": "model-00061-of-00101.safetensors", + "model.layers.56.mlp.experts.115.down_proj.weight": "model-00061-of-00101.safetensors", + "model.layers.56.mlp.experts.116.gate_proj.weight": "model-00061-of-00101.safetensors", + "model.layers.56.mlp.experts.116.up_proj.weight": "model-00061-of-00101.safetensors", + "model.layers.56.mlp.experts.116.down_proj.weight": "model-00061-of-00101.safetensors", + "model.layers.56.mlp.experts.117.gate_proj.weight": "model-00061-of-00101.safetensors", + "model.layers.56.mlp.experts.117.up_proj.weight": "model-00061-of-00101.safetensors", + "model.layers.56.mlp.experts.117.down_proj.weight": "model-00061-of-00101.safetensors", + "model.layers.56.mlp.experts.118.gate_proj.weight": "model-00061-of-00101.safetensors", + "model.layers.56.mlp.experts.118.up_proj.weight": "model-00061-of-00101.safetensors", + "model.layers.56.mlp.experts.118.down_proj.weight": "model-00061-of-00101.safetensors", + "model.layers.56.mlp.experts.119.gate_proj.weight": "model-00061-of-00101.safetensors", + "model.layers.56.mlp.experts.119.up_proj.weight": "model-00061-of-00101.safetensors", + "model.layers.56.mlp.experts.119.down_proj.weight": "model-00061-of-00101.safetensors", + "model.layers.56.mlp.gate.weight": "model-00061-of-00101.safetensors", + "model.layers.56.mlp.gate.e_score_correction_bias": "model-00061-of-00101.safetensors", + "model.layers.56.mlp.shared_experts.gate_proj.weight": "model-00061-of-00101.safetensors", + "model.layers.56.mlp.shared_experts.up_proj.weight": "model-00061-of-00101.safetensors", + "model.layers.56.mlp.shared_experts.down_proj.weight": "model-00061-of-00101.safetensors", + "model.layers.56.input_layernorm.weight": "model-00061-of-00101.safetensors", + "model.layers.56.post_attention_layernorm.weight": "model-00061-of-00101.safetensors", + "model.layers.57.self_attn.q_proj.weight": "model-00062-of-00101.safetensors", + "model.layers.57.self_attn.q_proj.bias": "model-00062-of-00101.safetensors", + "model.layers.57.self_attn.k_proj.weight": "model-00062-of-00101.safetensors", + "model.layers.57.self_attn.k_proj.bias": "model-00062-of-00101.safetensors", + "model.layers.57.self_attn.v_proj.weight": "model-00062-of-00101.safetensors", + "model.layers.57.self_attn.v_proj.bias": "model-00062-of-00101.safetensors", + "model.layers.57.self_attn.o_proj.weight": "model-00062-of-00101.safetensors", + "model.layers.57.self_attn.q_norm.weight": "model-00062-of-00101.safetensors", + "model.layers.57.self_attn.k_norm.weight": "model-00062-of-00101.safetensors", + "model.layers.57.mlp.experts.0.gate_proj.weight": "model-00062-of-00101.safetensors", + "model.layers.57.mlp.experts.0.up_proj.weight": "model-00062-of-00101.safetensors", + "model.layers.57.mlp.experts.0.down_proj.weight": "model-00062-of-00101.safetensors", + "model.layers.57.mlp.experts.1.gate_proj.weight": "model-00062-of-00101.safetensors", + "model.layers.57.mlp.experts.1.up_proj.weight": "model-00062-of-00101.safetensors", + "model.layers.57.mlp.experts.1.down_proj.weight": "model-00062-of-00101.safetensors", + "model.layers.57.mlp.experts.2.gate_proj.weight": "model-00062-of-00101.safetensors", + "model.layers.57.mlp.experts.2.up_proj.weight": "model-00062-of-00101.safetensors", + "model.layers.57.mlp.experts.2.down_proj.weight": "model-00062-of-00101.safetensors", + "model.layers.57.mlp.experts.3.gate_proj.weight": "model-00062-of-00101.safetensors", + "model.layers.57.mlp.experts.3.up_proj.weight": "model-00062-of-00101.safetensors", + "model.layers.57.mlp.experts.3.down_proj.weight": "model-00062-of-00101.safetensors", + "model.layers.57.mlp.experts.4.gate_proj.weight": "model-00062-of-00101.safetensors", + "model.layers.57.mlp.experts.4.up_proj.weight": "model-00062-of-00101.safetensors", + "model.layers.57.mlp.experts.4.down_proj.weight": "model-00062-of-00101.safetensors", + "model.layers.57.mlp.experts.5.gate_proj.weight": "model-00062-of-00101.safetensors", + "model.layers.57.mlp.experts.5.up_proj.weight": "model-00062-of-00101.safetensors", + "model.layers.57.mlp.experts.5.down_proj.weight": "model-00062-of-00101.safetensors", + "model.layers.57.mlp.experts.6.gate_proj.weight": "model-00062-of-00101.safetensors", + "model.layers.57.mlp.experts.6.up_proj.weight": "model-00062-of-00101.safetensors", + "model.layers.57.mlp.experts.6.down_proj.weight": "model-00062-of-00101.safetensors", + "model.layers.57.mlp.experts.7.gate_proj.weight": "model-00062-of-00101.safetensors", + "model.layers.57.mlp.experts.7.up_proj.weight": "model-00062-of-00101.safetensors", + "model.layers.57.mlp.experts.7.down_proj.weight": "model-00062-of-00101.safetensors", + "model.layers.57.mlp.experts.8.gate_proj.weight": "model-00062-of-00101.safetensors", + "model.layers.57.mlp.experts.8.up_proj.weight": "model-00062-of-00101.safetensors", + "model.layers.57.mlp.experts.8.down_proj.weight": "model-00062-of-00101.safetensors", + "model.layers.57.mlp.experts.9.gate_proj.weight": "model-00062-of-00101.safetensors", + "model.layers.57.mlp.experts.9.up_proj.weight": "model-00062-of-00101.safetensors", + "model.layers.57.mlp.experts.9.down_proj.weight": "model-00062-of-00101.safetensors", + "model.layers.57.mlp.experts.10.gate_proj.weight": "model-00062-of-00101.safetensors", + "model.layers.57.mlp.experts.10.up_proj.weight": "model-00062-of-00101.safetensors", + "model.layers.57.mlp.experts.10.down_proj.weight": "model-00062-of-00101.safetensors", + "model.layers.57.mlp.experts.11.gate_proj.weight": "model-00062-of-00101.safetensors", + "model.layers.57.mlp.experts.11.up_proj.weight": "model-00062-of-00101.safetensors", + "model.layers.57.mlp.experts.11.down_proj.weight": "model-00062-of-00101.safetensors", + "model.layers.57.mlp.experts.12.gate_proj.weight": "model-00062-of-00101.safetensors", + "model.layers.57.mlp.experts.12.up_proj.weight": "model-00062-of-00101.safetensors", + "model.layers.57.mlp.experts.12.down_proj.weight": "model-00062-of-00101.safetensors", + "model.layers.57.mlp.experts.13.gate_proj.weight": "model-00062-of-00101.safetensors", + "model.layers.57.mlp.experts.13.up_proj.weight": "model-00062-of-00101.safetensors", + "model.layers.57.mlp.experts.13.down_proj.weight": "model-00062-of-00101.safetensors", + "model.layers.57.mlp.experts.14.gate_proj.weight": "model-00062-of-00101.safetensors", + "model.layers.57.mlp.experts.14.up_proj.weight": "model-00062-of-00101.safetensors", + "model.layers.57.mlp.experts.14.down_proj.weight": "model-00062-of-00101.safetensors", + "model.layers.57.mlp.experts.15.gate_proj.weight": "model-00062-of-00101.safetensors", + "model.layers.57.mlp.experts.15.up_proj.weight": "model-00062-of-00101.safetensors", + "model.layers.57.mlp.experts.15.down_proj.weight": "model-00062-of-00101.safetensors", + "model.layers.57.mlp.experts.16.gate_proj.weight": "model-00062-of-00101.safetensors", + "model.layers.57.mlp.experts.16.up_proj.weight": "model-00062-of-00101.safetensors", + "model.layers.57.mlp.experts.16.down_proj.weight": "model-00062-of-00101.safetensors", + "model.layers.57.mlp.experts.17.gate_proj.weight": "model-00062-of-00101.safetensors", + "model.layers.57.mlp.experts.17.up_proj.weight": "model-00062-of-00101.safetensors", + "model.layers.57.mlp.experts.17.down_proj.weight": "model-00062-of-00101.safetensors", + "model.layers.57.mlp.experts.18.gate_proj.weight": "model-00062-of-00101.safetensors", + "model.layers.57.mlp.experts.18.up_proj.weight": "model-00062-of-00101.safetensors", + "model.layers.57.mlp.experts.18.down_proj.weight": "model-00062-of-00101.safetensors", + "model.layers.57.mlp.experts.19.gate_proj.weight": "model-00062-of-00101.safetensors", + "model.layers.57.mlp.experts.19.up_proj.weight": "model-00062-of-00101.safetensors", + "model.layers.57.mlp.experts.19.down_proj.weight": "model-00062-of-00101.safetensors", + "model.layers.57.mlp.experts.20.gate_proj.weight": "model-00062-of-00101.safetensors", + "model.layers.57.mlp.experts.20.up_proj.weight": "model-00062-of-00101.safetensors", + "model.layers.57.mlp.experts.20.down_proj.weight": "model-00062-of-00101.safetensors", + "model.layers.57.mlp.experts.21.gate_proj.weight": "model-00062-of-00101.safetensors", + "model.layers.57.mlp.experts.21.up_proj.weight": "model-00062-of-00101.safetensors", + "model.layers.57.mlp.experts.21.down_proj.weight": "model-00062-of-00101.safetensors", + "model.layers.57.mlp.experts.22.gate_proj.weight": "model-00062-of-00101.safetensors", + "model.layers.57.mlp.experts.22.up_proj.weight": "model-00062-of-00101.safetensors", + "model.layers.57.mlp.experts.22.down_proj.weight": "model-00062-of-00101.safetensors", + "model.layers.57.mlp.experts.23.gate_proj.weight": "model-00062-of-00101.safetensors", + "model.layers.57.mlp.experts.23.up_proj.weight": "model-00062-of-00101.safetensors", + "model.layers.57.mlp.experts.23.down_proj.weight": "model-00062-of-00101.safetensors", + "model.layers.57.mlp.experts.24.gate_proj.weight": "model-00062-of-00101.safetensors", + "model.layers.57.mlp.experts.24.up_proj.weight": "model-00062-of-00101.safetensors", + "model.layers.57.mlp.experts.24.down_proj.weight": "model-00062-of-00101.safetensors", + "model.layers.57.mlp.experts.25.gate_proj.weight": "model-00062-of-00101.safetensors", + "model.layers.57.mlp.experts.25.up_proj.weight": "model-00062-of-00101.safetensors", + "model.layers.57.mlp.experts.25.down_proj.weight": "model-00062-of-00101.safetensors", + "model.layers.57.mlp.experts.26.gate_proj.weight": "model-00062-of-00101.safetensors", + "model.layers.57.mlp.experts.26.up_proj.weight": "model-00062-of-00101.safetensors", + "model.layers.57.mlp.experts.26.down_proj.weight": "model-00062-of-00101.safetensors", + "model.layers.57.mlp.experts.27.gate_proj.weight": "model-00062-of-00101.safetensors", + "model.layers.57.mlp.experts.27.up_proj.weight": "model-00062-of-00101.safetensors", + "model.layers.57.mlp.experts.27.down_proj.weight": "model-00062-of-00101.safetensors", + "model.layers.57.mlp.experts.28.gate_proj.weight": "model-00062-of-00101.safetensors", + "model.layers.57.mlp.experts.28.up_proj.weight": "model-00062-of-00101.safetensors", + "model.layers.57.mlp.experts.28.down_proj.weight": "model-00062-of-00101.safetensors", + "model.layers.57.mlp.experts.29.gate_proj.weight": "model-00062-of-00101.safetensors", + "model.layers.57.mlp.experts.29.up_proj.weight": "model-00062-of-00101.safetensors", + "model.layers.57.mlp.experts.29.down_proj.weight": "model-00062-of-00101.safetensors", + "model.layers.57.mlp.experts.30.gate_proj.weight": "model-00062-of-00101.safetensors", + "model.layers.57.mlp.experts.30.up_proj.weight": "model-00062-of-00101.safetensors", + "model.layers.57.mlp.experts.30.down_proj.weight": "model-00062-of-00101.safetensors", + "model.layers.57.mlp.experts.31.gate_proj.weight": "model-00062-of-00101.safetensors", + "model.layers.57.mlp.experts.31.up_proj.weight": "model-00062-of-00101.safetensors", + "model.layers.57.mlp.experts.31.down_proj.weight": "model-00062-of-00101.safetensors", + "model.layers.57.mlp.experts.32.gate_proj.weight": "model-00062-of-00101.safetensors", + "model.layers.57.mlp.experts.32.up_proj.weight": "model-00062-of-00101.safetensors", + "model.layers.57.mlp.experts.32.down_proj.weight": "model-00062-of-00101.safetensors", + "model.layers.57.mlp.experts.33.gate_proj.weight": "model-00062-of-00101.safetensors", + "model.layers.57.mlp.experts.33.up_proj.weight": "model-00062-of-00101.safetensors", + "model.layers.57.mlp.experts.33.down_proj.weight": "model-00062-of-00101.safetensors", + "model.layers.57.mlp.experts.34.gate_proj.weight": "model-00062-of-00101.safetensors", + "model.layers.57.mlp.experts.34.up_proj.weight": "model-00062-of-00101.safetensors", + "model.layers.57.mlp.experts.34.down_proj.weight": "model-00062-of-00101.safetensors", + "model.layers.57.mlp.experts.35.gate_proj.weight": "model-00062-of-00101.safetensors", + "model.layers.57.mlp.experts.35.up_proj.weight": "model-00062-of-00101.safetensors", + "model.layers.57.mlp.experts.35.down_proj.weight": "model-00062-of-00101.safetensors", + "model.layers.57.mlp.experts.36.gate_proj.weight": "model-00062-of-00101.safetensors", + "model.layers.57.mlp.experts.36.up_proj.weight": "model-00062-of-00101.safetensors", + "model.layers.57.mlp.experts.36.down_proj.weight": "model-00062-of-00101.safetensors", + "model.layers.57.mlp.experts.37.gate_proj.weight": "model-00062-of-00101.safetensors", + "model.layers.57.mlp.experts.37.up_proj.weight": "model-00062-of-00101.safetensors", + "model.layers.57.mlp.experts.37.down_proj.weight": "model-00062-of-00101.safetensors", + "model.layers.57.mlp.experts.38.gate_proj.weight": "model-00062-of-00101.safetensors", + "model.layers.57.mlp.experts.38.up_proj.weight": "model-00062-of-00101.safetensors", + "model.layers.57.mlp.experts.38.down_proj.weight": "model-00062-of-00101.safetensors", + "model.layers.57.mlp.experts.39.gate_proj.weight": "model-00062-of-00101.safetensors", + "model.layers.57.mlp.experts.39.up_proj.weight": "model-00062-of-00101.safetensors", + "model.layers.57.mlp.experts.39.down_proj.weight": "model-00062-of-00101.safetensors", + "model.layers.57.mlp.experts.40.gate_proj.weight": "model-00062-of-00101.safetensors", + "model.layers.57.mlp.experts.40.up_proj.weight": "model-00062-of-00101.safetensors", + "model.layers.57.mlp.experts.40.down_proj.weight": "model-00062-of-00101.safetensors", + "model.layers.57.mlp.experts.41.gate_proj.weight": "model-00062-of-00101.safetensors", + "model.layers.57.mlp.experts.41.up_proj.weight": "model-00062-of-00101.safetensors", + "model.layers.57.mlp.experts.41.down_proj.weight": "model-00062-of-00101.safetensors", + "model.layers.57.mlp.experts.42.gate_proj.weight": "model-00062-of-00101.safetensors", + "model.layers.57.mlp.experts.42.up_proj.weight": "model-00062-of-00101.safetensors", + "model.layers.57.mlp.experts.42.down_proj.weight": "model-00062-of-00101.safetensors", + "model.layers.57.mlp.experts.43.gate_proj.weight": "model-00062-of-00101.safetensors", + "model.layers.57.mlp.experts.43.up_proj.weight": "model-00062-of-00101.safetensors", + "model.layers.57.mlp.experts.43.down_proj.weight": "model-00062-of-00101.safetensors", + "model.layers.57.mlp.experts.44.gate_proj.weight": "model-00062-of-00101.safetensors", + "model.layers.57.mlp.experts.44.up_proj.weight": "model-00062-of-00101.safetensors", + "model.layers.57.mlp.experts.44.down_proj.weight": "model-00062-of-00101.safetensors", + "model.layers.57.mlp.experts.45.gate_proj.weight": "model-00062-of-00101.safetensors", + "model.layers.57.mlp.experts.45.up_proj.weight": "model-00062-of-00101.safetensors", + "model.layers.57.mlp.experts.45.down_proj.weight": "model-00062-of-00101.safetensors", + "model.layers.57.mlp.experts.46.gate_proj.weight": "model-00062-of-00101.safetensors", + "model.layers.57.mlp.experts.46.up_proj.weight": "model-00062-of-00101.safetensors", + "model.layers.57.mlp.experts.46.down_proj.weight": "model-00062-of-00101.safetensors", + "model.layers.57.mlp.experts.47.gate_proj.weight": "model-00062-of-00101.safetensors", + "model.layers.57.mlp.experts.47.up_proj.weight": "model-00062-of-00101.safetensors", + "model.layers.57.mlp.experts.47.down_proj.weight": "model-00062-of-00101.safetensors", + "model.layers.57.mlp.experts.48.gate_proj.weight": "model-00062-of-00101.safetensors", + "model.layers.57.mlp.experts.48.up_proj.weight": "model-00062-of-00101.safetensors", + "model.layers.57.mlp.experts.48.down_proj.weight": "model-00062-of-00101.safetensors", + "model.layers.57.mlp.experts.49.gate_proj.weight": "model-00062-of-00101.safetensors", + "model.layers.57.mlp.experts.49.up_proj.weight": "model-00062-of-00101.safetensors", + "model.layers.57.mlp.experts.49.down_proj.weight": "model-00062-of-00101.safetensors", + "model.layers.57.mlp.experts.50.gate_proj.weight": "model-00062-of-00101.safetensors", + "model.layers.57.mlp.experts.50.up_proj.weight": "model-00062-of-00101.safetensors", + "model.layers.57.mlp.experts.50.down_proj.weight": "model-00062-of-00101.safetensors", + "model.layers.57.mlp.experts.51.gate_proj.weight": "model-00062-of-00101.safetensors", + "model.layers.57.mlp.experts.51.up_proj.weight": "model-00062-of-00101.safetensors", + "model.layers.57.mlp.experts.51.down_proj.weight": "model-00062-of-00101.safetensors", + "model.layers.57.mlp.experts.52.gate_proj.weight": "model-00062-of-00101.safetensors", + "model.layers.57.mlp.experts.52.up_proj.weight": "model-00062-of-00101.safetensors", + "model.layers.57.mlp.experts.52.down_proj.weight": "model-00062-of-00101.safetensors", + "model.layers.57.mlp.experts.53.gate_proj.weight": "model-00062-of-00101.safetensors", + "model.layers.57.mlp.experts.53.up_proj.weight": "model-00062-of-00101.safetensors", + "model.layers.57.mlp.experts.53.down_proj.weight": "model-00062-of-00101.safetensors", + "model.layers.57.mlp.experts.54.gate_proj.weight": "model-00062-of-00101.safetensors", + "model.layers.57.mlp.experts.54.up_proj.weight": "model-00062-of-00101.safetensors", + "model.layers.57.mlp.experts.54.down_proj.weight": "model-00062-of-00101.safetensors", + "model.layers.57.mlp.experts.55.gate_proj.weight": "model-00062-of-00101.safetensors", + "model.layers.57.mlp.experts.55.up_proj.weight": "model-00062-of-00101.safetensors", + "model.layers.57.mlp.experts.55.down_proj.weight": "model-00062-of-00101.safetensors", + "model.layers.57.mlp.experts.56.gate_proj.weight": "model-00062-of-00101.safetensors", + "model.layers.57.mlp.experts.56.up_proj.weight": "model-00062-of-00101.safetensors", + "model.layers.57.mlp.experts.56.down_proj.weight": "model-00062-of-00101.safetensors", + "model.layers.57.mlp.experts.57.gate_proj.weight": "model-00062-of-00101.safetensors", + "model.layers.57.mlp.experts.57.up_proj.weight": "model-00062-of-00101.safetensors", + "model.layers.57.mlp.experts.57.down_proj.weight": "model-00062-of-00101.safetensors", + "model.layers.57.mlp.experts.58.gate_proj.weight": "model-00062-of-00101.safetensors", + "model.layers.57.mlp.experts.58.up_proj.weight": "model-00062-of-00101.safetensors", + "model.layers.57.mlp.experts.58.down_proj.weight": "model-00062-of-00101.safetensors", + "model.layers.57.mlp.experts.59.gate_proj.weight": "model-00062-of-00101.safetensors", + "model.layers.57.mlp.experts.59.up_proj.weight": "model-00062-of-00101.safetensors", + "model.layers.57.mlp.experts.59.down_proj.weight": "model-00062-of-00101.safetensors", + "model.layers.57.mlp.experts.60.gate_proj.weight": "model-00062-of-00101.safetensors", + "model.layers.57.mlp.experts.60.up_proj.weight": "model-00062-of-00101.safetensors", + "model.layers.57.mlp.experts.60.down_proj.weight": "model-00062-of-00101.safetensors", + "model.layers.57.mlp.experts.61.gate_proj.weight": "model-00062-of-00101.safetensors", + "model.layers.57.mlp.experts.61.up_proj.weight": "model-00062-of-00101.safetensors", + "model.layers.57.mlp.experts.61.down_proj.weight": "model-00062-of-00101.safetensors", + "model.layers.57.mlp.experts.62.gate_proj.weight": "model-00062-of-00101.safetensors", + "model.layers.57.mlp.experts.62.up_proj.weight": "model-00062-of-00101.safetensors", + "model.layers.57.mlp.experts.62.down_proj.weight": "model-00062-of-00101.safetensors", + "model.layers.57.mlp.experts.63.gate_proj.weight": "model-00062-of-00101.safetensors", + "model.layers.57.mlp.experts.63.up_proj.weight": "model-00062-of-00101.safetensors", + "model.layers.57.mlp.experts.63.down_proj.weight": "model-00062-of-00101.safetensors", + "model.layers.57.mlp.experts.64.gate_proj.weight": "model-00062-of-00101.safetensors", + "model.layers.57.mlp.experts.64.up_proj.weight": "model-00062-of-00101.safetensors", + "model.layers.57.mlp.experts.64.down_proj.weight": "model-00062-of-00101.safetensors", + "model.layers.57.mlp.experts.65.gate_proj.weight": "model-00062-of-00101.safetensors", + "model.layers.57.mlp.experts.65.up_proj.weight": "model-00062-of-00101.safetensors", + "model.layers.57.mlp.experts.65.down_proj.weight": "model-00062-of-00101.safetensors", + "model.layers.57.mlp.experts.66.gate_proj.weight": "model-00062-of-00101.safetensors", + "model.layers.57.mlp.experts.66.up_proj.weight": "model-00062-of-00101.safetensors", + "model.layers.57.mlp.experts.66.down_proj.weight": "model-00062-of-00101.safetensors", + "model.layers.57.mlp.experts.67.gate_proj.weight": "model-00062-of-00101.safetensors", + "model.layers.57.mlp.experts.67.up_proj.weight": "model-00062-of-00101.safetensors", + "model.layers.57.mlp.experts.67.down_proj.weight": "model-00062-of-00101.safetensors", + "model.layers.57.mlp.experts.68.gate_proj.weight": "model-00062-of-00101.safetensors", + "model.layers.57.mlp.experts.68.up_proj.weight": "model-00062-of-00101.safetensors", + "model.layers.57.mlp.experts.68.down_proj.weight": "model-00062-of-00101.safetensors", + "model.layers.57.mlp.experts.69.gate_proj.weight": "model-00062-of-00101.safetensors", + "model.layers.57.mlp.experts.69.up_proj.weight": "model-00062-of-00101.safetensors", + "model.layers.57.mlp.experts.69.down_proj.weight": "model-00062-of-00101.safetensors", + "model.layers.57.mlp.experts.70.gate_proj.weight": "model-00062-of-00101.safetensors", + "model.layers.57.mlp.experts.70.up_proj.weight": "model-00062-of-00101.safetensors", + "model.layers.57.mlp.experts.70.down_proj.weight": "model-00062-of-00101.safetensors", + "model.layers.57.mlp.experts.71.gate_proj.weight": "model-00062-of-00101.safetensors", + "model.layers.57.mlp.experts.71.up_proj.weight": "model-00062-of-00101.safetensors", + "model.layers.57.mlp.experts.71.down_proj.weight": "model-00062-of-00101.safetensors", + "model.layers.57.mlp.experts.72.gate_proj.weight": "model-00062-of-00101.safetensors", + "model.layers.57.mlp.experts.72.up_proj.weight": "model-00062-of-00101.safetensors", + "model.layers.57.mlp.experts.72.down_proj.weight": "model-00062-of-00101.safetensors", + "model.layers.57.mlp.experts.73.gate_proj.weight": "model-00062-of-00101.safetensors", + "model.layers.57.mlp.experts.73.up_proj.weight": "model-00062-of-00101.safetensors", + "model.layers.57.mlp.experts.73.down_proj.weight": "model-00062-of-00101.safetensors", + "model.layers.57.mlp.experts.74.gate_proj.weight": "model-00062-of-00101.safetensors", + "model.layers.57.mlp.experts.74.up_proj.weight": "model-00062-of-00101.safetensors", + "model.layers.57.mlp.experts.74.down_proj.weight": "model-00062-of-00101.safetensors", + "model.layers.57.mlp.experts.75.gate_proj.weight": "model-00062-of-00101.safetensors", + "model.layers.57.mlp.experts.75.up_proj.weight": "model-00062-of-00101.safetensors", + "model.layers.57.mlp.experts.75.down_proj.weight": "model-00062-of-00101.safetensors", + "model.layers.57.mlp.experts.76.gate_proj.weight": "model-00062-of-00101.safetensors", + "model.layers.57.mlp.experts.76.up_proj.weight": "model-00062-of-00101.safetensors", + "model.layers.57.mlp.experts.76.down_proj.weight": "model-00062-of-00101.safetensors", + "model.layers.57.mlp.experts.77.gate_proj.weight": "model-00062-of-00101.safetensors", + "model.layers.57.mlp.experts.77.up_proj.weight": "model-00062-of-00101.safetensors", + "model.layers.57.mlp.experts.77.down_proj.weight": "model-00062-of-00101.safetensors", + "model.layers.57.mlp.experts.78.gate_proj.weight": "model-00062-of-00101.safetensors", + "model.layers.57.mlp.experts.78.up_proj.weight": "model-00062-of-00101.safetensors", + "model.layers.57.mlp.experts.78.down_proj.weight": "model-00062-of-00101.safetensors", + "model.layers.57.mlp.experts.79.gate_proj.weight": "model-00062-of-00101.safetensors", + "model.layers.57.mlp.experts.79.up_proj.weight": "model-00062-of-00101.safetensors", + "model.layers.57.mlp.experts.79.down_proj.weight": "model-00062-of-00101.safetensors", + "model.layers.57.mlp.experts.80.gate_proj.weight": "model-00062-of-00101.safetensors", + "model.layers.57.mlp.experts.80.up_proj.weight": "model-00062-of-00101.safetensors", + "model.layers.57.mlp.experts.80.down_proj.weight": "model-00062-of-00101.safetensors", + "model.layers.57.mlp.experts.81.gate_proj.weight": "model-00062-of-00101.safetensors", + "model.layers.57.mlp.experts.81.up_proj.weight": "model-00062-of-00101.safetensors", + "model.layers.57.mlp.experts.81.down_proj.weight": "model-00062-of-00101.safetensors", + "model.layers.57.mlp.experts.82.gate_proj.weight": "model-00062-of-00101.safetensors", + "model.layers.57.mlp.experts.82.up_proj.weight": "model-00062-of-00101.safetensors", + "model.layers.57.mlp.experts.82.down_proj.weight": "model-00062-of-00101.safetensors", + "model.layers.57.mlp.experts.83.gate_proj.weight": "model-00062-of-00101.safetensors", + "model.layers.57.mlp.experts.83.up_proj.weight": "model-00062-of-00101.safetensors", + "model.layers.57.mlp.experts.83.down_proj.weight": "model-00062-of-00101.safetensors", + "model.layers.57.mlp.experts.84.gate_proj.weight": "model-00062-of-00101.safetensors", + "model.layers.57.mlp.experts.84.up_proj.weight": "model-00062-of-00101.safetensors", + "model.layers.57.mlp.experts.84.down_proj.weight": "model-00062-of-00101.safetensors", + "model.layers.57.mlp.experts.85.gate_proj.weight": "model-00062-of-00101.safetensors", + "model.layers.57.mlp.experts.85.up_proj.weight": "model-00062-of-00101.safetensors", + "model.layers.57.mlp.experts.85.down_proj.weight": "model-00062-of-00101.safetensors", + "model.layers.57.mlp.experts.86.gate_proj.weight": "model-00062-of-00101.safetensors", + "model.layers.57.mlp.experts.86.up_proj.weight": "model-00062-of-00101.safetensors", + "model.layers.57.mlp.experts.86.down_proj.weight": "model-00062-of-00101.safetensors", + "model.layers.57.mlp.experts.87.gate_proj.weight": "model-00062-of-00101.safetensors", + "model.layers.57.mlp.experts.87.up_proj.weight": "model-00062-of-00101.safetensors", + "model.layers.57.mlp.experts.87.down_proj.weight": "model-00062-of-00101.safetensors", + "model.layers.57.mlp.experts.88.gate_proj.weight": "model-00062-of-00101.safetensors", + "model.layers.57.mlp.experts.88.up_proj.weight": "model-00062-of-00101.safetensors", + "model.layers.57.mlp.experts.88.down_proj.weight": "model-00062-of-00101.safetensors", + "model.layers.57.mlp.experts.89.gate_proj.weight": "model-00062-of-00101.safetensors", + "model.layers.57.mlp.experts.89.up_proj.weight": "model-00062-of-00101.safetensors", + "model.layers.57.mlp.experts.89.down_proj.weight": "model-00062-of-00101.safetensors", + "model.layers.57.mlp.experts.90.gate_proj.weight": "model-00062-of-00101.safetensors", + "model.layers.57.mlp.experts.90.up_proj.weight": "model-00062-of-00101.safetensors", + "model.layers.57.mlp.experts.90.down_proj.weight": "model-00062-of-00101.safetensors", + "model.layers.57.mlp.experts.91.gate_proj.weight": "model-00062-of-00101.safetensors", + "model.layers.57.mlp.experts.91.up_proj.weight": "model-00062-of-00101.safetensors", + "model.layers.57.mlp.experts.91.down_proj.weight": "model-00062-of-00101.safetensors", + "model.layers.57.mlp.experts.92.gate_proj.weight": "model-00062-of-00101.safetensors", + "model.layers.57.mlp.experts.92.up_proj.weight": "model-00062-of-00101.safetensors", + "model.layers.57.mlp.experts.92.down_proj.weight": "model-00062-of-00101.safetensors", + "model.layers.57.mlp.experts.93.gate_proj.weight": "model-00062-of-00101.safetensors", + "model.layers.57.mlp.experts.93.up_proj.weight": "model-00062-of-00101.safetensors", + "model.layers.57.mlp.experts.93.down_proj.weight": "model-00062-of-00101.safetensors", + "model.layers.57.mlp.experts.94.gate_proj.weight": "model-00062-of-00101.safetensors", + "model.layers.57.mlp.experts.94.up_proj.weight": "model-00062-of-00101.safetensors", + "model.layers.57.mlp.experts.94.down_proj.weight": "model-00062-of-00101.safetensors", + "model.layers.57.mlp.experts.95.gate_proj.weight": "model-00062-of-00101.safetensors", + "model.layers.57.mlp.experts.95.up_proj.weight": "model-00062-of-00101.safetensors", + "model.layers.57.mlp.experts.95.down_proj.weight": "model-00062-of-00101.safetensors", + "model.layers.57.mlp.experts.96.gate_proj.weight": "model-00062-of-00101.safetensors", + "model.layers.57.mlp.experts.96.up_proj.weight": "model-00062-of-00101.safetensors", + "model.layers.57.mlp.experts.96.down_proj.weight": "model-00062-of-00101.safetensors", + "model.layers.57.mlp.experts.97.gate_proj.weight": "model-00062-of-00101.safetensors", + "model.layers.57.mlp.experts.97.up_proj.weight": "model-00062-of-00101.safetensors", + "model.layers.57.mlp.experts.97.down_proj.weight": "model-00062-of-00101.safetensors", + "model.layers.57.mlp.experts.98.gate_proj.weight": "model-00062-of-00101.safetensors", + "model.layers.57.mlp.experts.98.up_proj.weight": "model-00062-of-00101.safetensors", + "model.layers.57.mlp.experts.98.down_proj.weight": "model-00062-of-00101.safetensors", + "model.layers.57.mlp.experts.99.gate_proj.weight": "model-00062-of-00101.safetensors", + "model.layers.57.mlp.experts.99.up_proj.weight": "model-00062-of-00101.safetensors", + "model.layers.57.mlp.experts.99.down_proj.weight": "model-00062-of-00101.safetensors", + "model.layers.57.mlp.experts.100.gate_proj.weight": "model-00062-of-00101.safetensors", + "model.layers.57.mlp.experts.100.up_proj.weight": "model-00062-of-00101.safetensors", + "model.layers.57.mlp.experts.100.down_proj.weight": "model-00062-of-00101.safetensors", + "model.layers.57.mlp.experts.101.gate_proj.weight": "model-00062-of-00101.safetensors", + "model.layers.57.mlp.experts.101.up_proj.weight": "model-00062-of-00101.safetensors", + "model.layers.57.mlp.experts.101.down_proj.weight": "model-00062-of-00101.safetensors", + "model.layers.57.mlp.experts.102.gate_proj.weight": "model-00062-of-00101.safetensors", + "model.layers.57.mlp.experts.102.up_proj.weight": "model-00062-of-00101.safetensors", + "model.layers.57.mlp.experts.102.down_proj.weight": "model-00062-of-00101.safetensors", + "model.layers.57.mlp.experts.103.gate_proj.weight": "model-00062-of-00101.safetensors", + "model.layers.57.mlp.experts.103.up_proj.weight": "model-00062-of-00101.safetensors", + "model.layers.57.mlp.experts.103.down_proj.weight": "model-00062-of-00101.safetensors", + "model.layers.57.mlp.experts.104.gate_proj.weight": "model-00062-of-00101.safetensors", + "model.layers.57.mlp.experts.104.up_proj.weight": "model-00062-of-00101.safetensors", + "model.layers.57.mlp.experts.104.down_proj.weight": "model-00062-of-00101.safetensors", + "model.layers.57.mlp.experts.105.gate_proj.weight": "model-00062-of-00101.safetensors", + "model.layers.57.mlp.experts.105.up_proj.weight": "model-00062-of-00101.safetensors", + "model.layers.57.mlp.experts.105.down_proj.weight": "model-00062-of-00101.safetensors", + "model.layers.57.mlp.experts.106.gate_proj.weight": "model-00062-of-00101.safetensors", + "model.layers.57.mlp.experts.106.up_proj.weight": "model-00062-of-00101.safetensors", + "model.layers.57.mlp.experts.106.down_proj.weight": "model-00062-of-00101.safetensors", + "model.layers.57.mlp.experts.107.gate_proj.weight": "model-00062-of-00101.safetensors", + "model.layers.57.mlp.experts.107.up_proj.weight": "model-00062-of-00101.safetensors", + "model.layers.57.mlp.experts.107.down_proj.weight": "model-00063-of-00101.safetensors", + "model.layers.57.mlp.experts.108.gate_proj.weight": "model-00063-of-00101.safetensors", + "model.layers.57.mlp.experts.108.up_proj.weight": "model-00063-of-00101.safetensors", + "model.layers.57.mlp.experts.108.down_proj.weight": "model-00063-of-00101.safetensors", + "model.layers.57.mlp.experts.109.gate_proj.weight": "model-00063-of-00101.safetensors", + "model.layers.57.mlp.experts.109.up_proj.weight": "model-00063-of-00101.safetensors", + "model.layers.57.mlp.experts.109.down_proj.weight": "model-00063-of-00101.safetensors", + "model.layers.57.mlp.experts.110.gate_proj.weight": "model-00063-of-00101.safetensors", + "model.layers.57.mlp.experts.110.up_proj.weight": "model-00063-of-00101.safetensors", + "model.layers.57.mlp.experts.110.down_proj.weight": "model-00063-of-00101.safetensors", + "model.layers.57.mlp.experts.111.gate_proj.weight": "model-00063-of-00101.safetensors", + "model.layers.57.mlp.experts.111.up_proj.weight": "model-00063-of-00101.safetensors", + "model.layers.57.mlp.experts.111.down_proj.weight": "model-00063-of-00101.safetensors", + "model.layers.57.mlp.experts.112.gate_proj.weight": "model-00063-of-00101.safetensors", + "model.layers.57.mlp.experts.112.up_proj.weight": "model-00063-of-00101.safetensors", + "model.layers.57.mlp.experts.112.down_proj.weight": "model-00063-of-00101.safetensors", + "model.layers.57.mlp.experts.113.gate_proj.weight": "model-00063-of-00101.safetensors", + "model.layers.57.mlp.experts.113.up_proj.weight": "model-00063-of-00101.safetensors", + "model.layers.57.mlp.experts.113.down_proj.weight": "model-00063-of-00101.safetensors", + "model.layers.57.mlp.experts.114.gate_proj.weight": "model-00063-of-00101.safetensors", + "model.layers.57.mlp.experts.114.up_proj.weight": "model-00063-of-00101.safetensors", + "model.layers.57.mlp.experts.114.down_proj.weight": "model-00063-of-00101.safetensors", + "model.layers.57.mlp.experts.115.gate_proj.weight": "model-00063-of-00101.safetensors", + "model.layers.57.mlp.experts.115.up_proj.weight": "model-00063-of-00101.safetensors", + "model.layers.57.mlp.experts.115.down_proj.weight": "model-00063-of-00101.safetensors", + "model.layers.57.mlp.experts.116.gate_proj.weight": "model-00063-of-00101.safetensors", + "model.layers.57.mlp.experts.116.up_proj.weight": "model-00063-of-00101.safetensors", + "model.layers.57.mlp.experts.116.down_proj.weight": "model-00063-of-00101.safetensors", + "model.layers.57.mlp.experts.117.gate_proj.weight": "model-00063-of-00101.safetensors", + "model.layers.57.mlp.experts.117.up_proj.weight": "model-00063-of-00101.safetensors", + "model.layers.57.mlp.experts.117.down_proj.weight": "model-00063-of-00101.safetensors", + "model.layers.57.mlp.experts.118.gate_proj.weight": "model-00063-of-00101.safetensors", + "model.layers.57.mlp.experts.118.up_proj.weight": "model-00063-of-00101.safetensors", + "model.layers.57.mlp.experts.118.down_proj.weight": "model-00063-of-00101.safetensors", + "model.layers.57.mlp.experts.119.gate_proj.weight": "model-00063-of-00101.safetensors", + "model.layers.57.mlp.experts.119.up_proj.weight": "model-00063-of-00101.safetensors", + "model.layers.57.mlp.experts.119.down_proj.weight": "model-00063-of-00101.safetensors", + "model.layers.57.mlp.gate.weight": "model-00063-of-00101.safetensors", + "model.layers.57.mlp.gate.e_score_correction_bias": "model-00063-of-00101.safetensors", + "model.layers.57.mlp.shared_experts.gate_proj.weight": "model-00063-of-00101.safetensors", + "model.layers.57.mlp.shared_experts.up_proj.weight": "model-00063-of-00101.safetensors", + "model.layers.57.mlp.shared_experts.down_proj.weight": "model-00063-of-00101.safetensors", + "model.layers.57.input_layernorm.weight": "model-00063-of-00101.safetensors", + "model.layers.57.post_attention_layernorm.weight": "model-00063-of-00101.safetensors", + "model.layers.58.self_attn.q_proj.weight": "model-00063-of-00101.safetensors", + "model.layers.58.self_attn.q_proj.bias": "model-00063-of-00101.safetensors", + "model.layers.58.self_attn.k_proj.weight": "model-00063-of-00101.safetensors", + "model.layers.58.self_attn.k_proj.bias": "model-00063-of-00101.safetensors", + "model.layers.58.self_attn.v_proj.weight": "model-00063-of-00101.safetensors", + "model.layers.58.self_attn.v_proj.bias": "model-00063-of-00101.safetensors", + "model.layers.58.self_attn.o_proj.weight": "model-00063-of-00101.safetensors", + "model.layers.58.self_attn.q_norm.weight": "model-00063-of-00101.safetensors", + "model.layers.58.self_attn.k_norm.weight": "model-00063-of-00101.safetensors", + "model.layers.58.mlp.experts.0.gate_proj.weight": "model-00063-of-00101.safetensors", + "model.layers.58.mlp.experts.0.up_proj.weight": "model-00063-of-00101.safetensors", + "model.layers.58.mlp.experts.0.down_proj.weight": "model-00063-of-00101.safetensors", + "model.layers.58.mlp.experts.1.gate_proj.weight": "model-00063-of-00101.safetensors", + "model.layers.58.mlp.experts.1.up_proj.weight": "model-00063-of-00101.safetensors", + "model.layers.58.mlp.experts.1.down_proj.weight": "model-00063-of-00101.safetensors", + "model.layers.58.mlp.experts.2.gate_proj.weight": "model-00063-of-00101.safetensors", + "model.layers.58.mlp.experts.2.up_proj.weight": "model-00063-of-00101.safetensors", + "model.layers.58.mlp.experts.2.down_proj.weight": "model-00063-of-00101.safetensors", + "model.layers.58.mlp.experts.3.gate_proj.weight": "model-00063-of-00101.safetensors", + "model.layers.58.mlp.experts.3.up_proj.weight": "model-00063-of-00101.safetensors", + "model.layers.58.mlp.experts.3.down_proj.weight": "model-00063-of-00101.safetensors", + "model.layers.58.mlp.experts.4.gate_proj.weight": "model-00063-of-00101.safetensors", + "model.layers.58.mlp.experts.4.up_proj.weight": "model-00063-of-00101.safetensors", + "model.layers.58.mlp.experts.4.down_proj.weight": "model-00063-of-00101.safetensors", + "model.layers.58.mlp.experts.5.gate_proj.weight": "model-00063-of-00101.safetensors", + "model.layers.58.mlp.experts.5.up_proj.weight": "model-00063-of-00101.safetensors", + "model.layers.58.mlp.experts.5.down_proj.weight": "model-00063-of-00101.safetensors", + "model.layers.58.mlp.experts.6.gate_proj.weight": "model-00063-of-00101.safetensors", + "model.layers.58.mlp.experts.6.up_proj.weight": "model-00063-of-00101.safetensors", + "model.layers.58.mlp.experts.6.down_proj.weight": "model-00063-of-00101.safetensors", + "model.layers.58.mlp.experts.7.gate_proj.weight": "model-00063-of-00101.safetensors", + "model.layers.58.mlp.experts.7.up_proj.weight": "model-00063-of-00101.safetensors", + "model.layers.58.mlp.experts.7.down_proj.weight": "model-00063-of-00101.safetensors", + "model.layers.58.mlp.experts.8.gate_proj.weight": "model-00063-of-00101.safetensors", + "model.layers.58.mlp.experts.8.up_proj.weight": "model-00063-of-00101.safetensors", + "model.layers.58.mlp.experts.8.down_proj.weight": "model-00063-of-00101.safetensors", + "model.layers.58.mlp.experts.9.gate_proj.weight": "model-00063-of-00101.safetensors", + "model.layers.58.mlp.experts.9.up_proj.weight": "model-00063-of-00101.safetensors", + "model.layers.58.mlp.experts.9.down_proj.weight": "model-00063-of-00101.safetensors", + "model.layers.58.mlp.experts.10.gate_proj.weight": "model-00063-of-00101.safetensors", + "model.layers.58.mlp.experts.10.up_proj.weight": "model-00063-of-00101.safetensors", + "model.layers.58.mlp.experts.10.down_proj.weight": "model-00063-of-00101.safetensors", + "model.layers.58.mlp.experts.11.gate_proj.weight": "model-00063-of-00101.safetensors", + "model.layers.58.mlp.experts.11.up_proj.weight": "model-00063-of-00101.safetensors", + "model.layers.58.mlp.experts.11.down_proj.weight": "model-00063-of-00101.safetensors", + "model.layers.58.mlp.experts.12.gate_proj.weight": "model-00063-of-00101.safetensors", + "model.layers.58.mlp.experts.12.up_proj.weight": "model-00063-of-00101.safetensors", + "model.layers.58.mlp.experts.12.down_proj.weight": "model-00063-of-00101.safetensors", + "model.layers.58.mlp.experts.13.gate_proj.weight": "model-00063-of-00101.safetensors", + "model.layers.58.mlp.experts.13.up_proj.weight": "model-00063-of-00101.safetensors", + "model.layers.58.mlp.experts.13.down_proj.weight": "model-00063-of-00101.safetensors", + "model.layers.58.mlp.experts.14.gate_proj.weight": "model-00063-of-00101.safetensors", + "model.layers.58.mlp.experts.14.up_proj.weight": "model-00063-of-00101.safetensors", + "model.layers.58.mlp.experts.14.down_proj.weight": "model-00063-of-00101.safetensors", + "model.layers.58.mlp.experts.15.gate_proj.weight": "model-00063-of-00101.safetensors", + "model.layers.58.mlp.experts.15.up_proj.weight": "model-00063-of-00101.safetensors", + "model.layers.58.mlp.experts.15.down_proj.weight": "model-00063-of-00101.safetensors", + "model.layers.58.mlp.experts.16.gate_proj.weight": "model-00063-of-00101.safetensors", + "model.layers.58.mlp.experts.16.up_proj.weight": "model-00063-of-00101.safetensors", + "model.layers.58.mlp.experts.16.down_proj.weight": "model-00063-of-00101.safetensors", + "model.layers.58.mlp.experts.17.gate_proj.weight": "model-00063-of-00101.safetensors", + "model.layers.58.mlp.experts.17.up_proj.weight": "model-00063-of-00101.safetensors", + "model.layers.58.mlp.experts.17.down_proj.weight": "model-00063-of-00101.safetensors", + "model.layers.58.mlp.experts.18.gate_proj.weight": "model-00063-of-00101.safetensors", + "model.layers.58.mlp.experts.18.up_proj.weight": "model-00063-of-00101.safetensors", + "model.layers.58.mlp.experts.18.down_proj.weight": "model-00063-of-00101.safetensors", + "model.layers.58.mlp.experts.19.gate_proj.weight": "model-00063-of-00101.safetensors", + "model.layers.58.mlp.experts.19.up_proj.weight": "model-00063-of-00101.safetensors", + "model.layers.58.mlp.experts.19.down_proj.weight": "model-00063-of-00101.safetensors", + "model.layers.58.mlp.experts.20.gate_proj.weight": "model-00063-of-00101.safetensors", + "model.layers.58.mlp.experts.20.up_proj.weight": "model-00063-of-00101.safetensors", + "model.layers.58.mlp.experts.20.down_proj.weight": "model-00063-of-00101.safetensors", + "model.layers.58.mlp.experts.21.gate_proj.weight": "model-00063-of-00101.safetensors", + "model.layers.58.mlp.experts.21.up_proj.weight": "model-00063-of-00101.safetensors", + "model.layers.58.mlp.experts.21.down_proj.weight": "model-00063-of-00101.safetensors", + "model.layers.58.mlp.experts.22.gate_proj.weight": "model-00063-of-00101.safetensors", + "model.layers.58.mlp.experts.22.up_proj.weight": "model-00063-of-00101.safetensors", + "model.layers.58.mlp.experts.22.down_proj.weight": "model-00063-of-00101.safetensors", + "model.layers.58.mlp.experts.23.gate_proj.weight": "model-00063-of-00101.safetensors", + "model.layers.58.mlp.experts.23.up_proj.weight": "model-00063-of-00101.safetensors", + "model.layers.58.mlp.experts.23.down_proj.weight": "model-00063-of-00101.safetensors", + "model.layers.58.mlp.experts.24.gate_proj.weight": "model-00063-of-00101.safetensors", + "model.layers.58.mlp.experts.24.up_proj.weight": "model-00063-of-00101.safetensors", + "model.layers.58.mlp.experts.24.down_proj.weight": "model-00063-of-00101.safetensors", + "model.layers.58.mlp.experts.25.gate_proj.weight": "model-00063-of-00101.safetensors", + "model.layers.58.mlp.experts.25.up_proj.weight": "model-00063-of-00101.safetensors", + "model.layers.58.mlp.experts.25.down_proj.weight": "model-00063-of-00101.safetensors", + "model.layers.58.mlp.experts.26.gate_proj.weight": "model-00063-of-00101.safetensors", + "model.layers.58.mlp.experts.26.up_proj.weight": "model-00063-of-00101.safetensors", + "model.layers.58.mlp.experts.26.down_proj.weight": "model-00063-of-00101.safetensors", + "model.layers.58.mlp.experts.27.gate_proj.weight": "model-00063-of-00101.safetensors", + "model.layers.58.mlp.experts.27.up_proj.weight": "model-00063-of-00101.safetensors", + "model.layers.58.mlp.experts.27.down_proj.weight": "model-00063-of-00101.safetensors", + "model.layers.58.mlp.experts.28.gate_proj.weight": "model-00063-of-00101.safetensors", + "model.layers.58.mlp.experts.28.up_proj.weight": "model-00063-of-00101.safetensors", + "model.layers.58.mlp.experts.28.down_proj.weight": "model-00063-of-00101.safetensors", + "model.layers.58.mlp.experts.29.gate_proj.weight": "model-00063-of-00101.safetensors", + "model.layers.58.mlp.experts.29.up_proj.weight": "model-00063-of-00101.safetensors", + "model.layers.58.mlp.experts.29.down_proj.weight": "model-00063-of-00101.safetensors", + "model.layers.58.mlp.experts.30.gate_proj.weight": "model-00063-of-00101.safetensors", + "model.layers.58.mlp.experts.30.up_proj.weight": "model-00063-of-00101.safetensors", + "model.layers.58.mlp.experts.30.down_proj.weight": "model-00063-of-00101.safetensors", + "model.layers.58.mlp.experts.31.gate_proj.weight": "model-00063-of-00101.safetensors", + "model.layers.58.mlp.experts.31.up_proj.weight": "model-00063-of-00101.safetensors", + "model.layers.58.mlp.experts.31.down_proj.weight": "model-00063-of-00101.safetensors", + "model.layers.58.mlp.experts.32.gate_proj.weight": "model-00063-of-00101.safetensors", + "model.layers.58.mlp.experts.32.up_proj.weight": "model-00063-of-00101.safetensors", + "model.layers.58.mlp.experts.32.down_proj.weight": "model-00063-of-00101.safetensors", + "model.layers.58.mlp.experts.33.gate_proj.weight": "model-00063-of-00101.safetensors", + "model.layers.58.mlp.experts.33.up_proj.weight": "model-00063-of-00101.safetensors", + "model.layers.58.mlp.experts.33.down_proj.weight": "model-00063-of-00101.safetensors", + "model.layers.58.mlp.experts.34.gate_proj.weight": "model-00063-of-00101.safetensors", + "model.layers.58.mlp.experts.34.up_proj.weight": "model-00063-of-00101.safetensors", + "model.layers.58.mlp.experts.34.down_proj.weight": "model-00063-of-00101.safetensors", + "model.layers.58.mlp.experts.35.gate_proj.weight": "model-00063-of-00101.safetensors", + "model.layers.58.mlp.experts.35.up_proj.weight": "model-00063-of-00101.safetensors", + "model.layers.58.mlp.experts.35.down_proj.weight": "model-00063-of-00101.safetensors", + "model.layers.58.mlp.experts.36.gate_proj.weight": "model-00063-of-00101.safetensors", + "model.layers.58.mlp.experts.36.up_proj.weight": "model-00063-of-00101.safetensors", + "model.layers.58.mlp.experts.36.down_proj.weight": "model-00063-of-00101.safetensors", + "model.layers.58.mlp.experts.37.gate_proj.weight": "model-00063-of-00101.safetensors", + "model.layers.58.mlp.experts.37.up_proj.weight": "model-00063-of-00101.safetensors", + "model.layers.58.mlp.experts.37.down_proj.weight": "model-00063-of-00101.safetensors", + "model.layers.58.mlp.experts.38.gate_proj.weight": "model-00063-of-00101.safetensors", + "model.layers.58.mlp.experts.38.up_proj.weight": "model-00063-of-00101.safetensors", + "model.layers.58.mlp.experts.38.down_proj.weight": "model-00063-of-00101.safetensors", + "model.layers.58.mlp.experts.39.gate_proj.weight": "model-00063-of-00101.safetensors", + "model.layers.58.mlp.experts.39.up_proj.weight": "model-00063-of-00101.safetensors", + "model.layers.58.mlp.experts.39.down_proj.weight": "model-00063-of-00101.safetensors", + "model.layers.58.mlp.experts.40.gate_proj.weight": "model-00063-of-00101.safetensors", + "model.layers.58.mlp.experts.40.up_proj.weight": "model-00063-of-00101.safetensors", + "model.layers.58.mlp.experts.40.down_proj.weight": "model-00063-of-00101.safetensors", + "model.layers.58.mlp.experts.41.gate_proj.weight": "model-00063-of-00101.safetensors", + "model.layers.58.mlp.experts.41.up_proj.weight": "model-00063-of-00101.safetensors", + "model.layers.58.mlp.experts.41.down_proj.weight": "model-00063-of-00101.safetensors", + "model.layers.58.mlp.experts.42.gate_proj.weight": "model-00063-of-00101.safetensors", + "model.layers.58.mlp.experts.42.up_proj.weight": "model-00063-of-00101.safetensors", + "model.layers.58.mlp.experts.42.down_proj.weight": "model-00063-of-00101.safetensors", + "model.layers.58.mlp.experts.43.gate_proj.weight": "model-00063-of-00101.safetensors", + "model.layers.58.mlp.experts.43.up_proj.weight": "model-00063-of-00101.safetensors", + "model.layers.58.mlp.experts.43.down_proj.weight": "model-00063-of-00101.safetensors", + "model.layers.58.mlp.experts.44.gate_proj.weight": "model-00063-of-00101.safetensors", + "model.layers.58.mlp.experts.44.up_proj.weight": "model-00063-of-00101.safetensors", + "model.layers.58.mlp.experts.44.down_proj.weight": "model-00063-of-00101.safetensors", + "model.layers.58.mlp.experts.45.gate_proj.weight": "model-00063-of-00101.safetensors", + "model.layers.58.mlp.experts.45.up_proj.weight": "model-00063-of-00101.safetensors", + "model.layers.58.mlp.experts.45.down_proj.weight": "model-00063-of-00101.safetensors", + "model.layers.58.mlp.experts.46.gate_proj.weight": "model-00063-of-00101.safetensors", + "model.layers.58.mlp.experts.46.up_proj.weight": "model-00063-of-00101.safetensors", + "model.layers.58.mlp.experts.46.down_proj.weight": "model-00063-of-00101.safetensors", + "model.layers.58.mlp.experts.47.gate_proj.weight": "model-00063-of-00101.safetensors", + "model.layers.58.mlp.experts.47.up_proj.weight": "model-00063-of-00101.safetensors", + "model.layers.58.mlp.experts.47.down_proj.weight": "model-00063-of-00101.safetensors", + "model.layers.58.mlp.experts.48.gate_proj.weight": "model-00063-of-00101.safetensors", + "model.layers.58.mlp.experts.48.up_proj.weight": "model-00063-of-00101.safetensors", + "model.layers.58.mlp.experts.48.down_proj.weight": "model-00063-of-00101.safetensors", + "model.layers.58.mlp.experts.49.gate_proj.weight": "model-00063-of-00101.safetensors", + "model.layers.58.mlp.experts.49.up_proj.weight": "model-00063-of-00101.safetensors", + "model.layers.58.mlp.experts.49.down_proj.weight": "model-00063-of-00101.safetensors", + "model.layers.58.mlp.experts.50.gate_proj.weight": "model-00063-of-00101.safetensors", + "model.layers.58.mlp.experts.50.up_proj.weight": "model-00063-of-00101.safetensors", + "model.layers.58.mlp.experts.50.down_proj.weight": "model-00063-of-00101.safetensors", + "model.layers.58.mlp.experts.51.gate_proj.weight": "model-00063-of-00101.safetensors", + "model.layers.58.mlp.experts.51.up_proj.weight": "model-00063-of-00101.safetensors", + "model.layers.58.mlp.experts.51.down_proj.weight": "model-00063-of-00101.safetensors", + "model.layers.58.mlp.experts.52.gate_proj.weight": "model-00063-of-00101.safetensors", + "model.layers.58.mlp.experts.52.up_proj.weight": "model-00063-of-00101.safetensors", + "model.layers.58.mlp.experts.52.down_proj.weight": "model-00063-of-00101.safetensors", + "model.layers.58.mlp.experts.53.gate_proj.weight": "model-00063-of-00101.safetensors", + "model.layers.58.mlp.experts.53.up_proj.weight": "model-00063-of-00101.safetensors", + "model.layers.58.mlp.experts.53.down_proj.weight": "model-00063-of-00101.safetensors", + "model.layers.58.mlp.experts.54.gate_proj.weight": "model-00063-of-00101.safetensors", + "model.layers.58.mlp.experts.54.up_proj.weight": "model-00063-of-00101.safetensors", + "model.layers.58.mlp.experts.54.down_proj.weight": "model-00063-of-00101.safetensors", + "model.layers.58.mlp.experts.55.gate_proj.weight": "model-00063-of-00101.safetensors", + "model.layers.58.mlp.experts.55.up_proj.weight": "model-00063-of-00101.safetensors", + "model.layers.58.mlp.experts.55.down_proj.weight": "model-00063-of-00101.safetensors", + "model.layers.58.mlp.experts.56.gate_proj.weight": "model-00063-of-00101.safetensors", + "model.layers.58.mlp.experts.56.up_proj.weight": "model-00063-of-00101.safetensors", + "model.layers.58.mlp.experts.56.down_proj.weight": "model-00063-of-00101.safetensors", + "model.layers.58.mlp.experts.57.gate_proj.weight": "model-00063-of-00101.safetensors", + "model.layers.58.mlp.experts.57.up_proj.weight": "model-00063-of-00101.safetensors", + "model.layers.58.mlp.experts.57.down_proj.weight": "model-00063-of-00101.safetensors", + "model.layers.58.mlp.experts.58.gate_proj.weight": "model-00063-of-00101.safetensors", + "model.layers.58.mlp.experts.58.up_proj.weight": "model-00063-of-00101.safetensors", + "model.layers.58.mlp.experts.58.down_proj.weight": "model-00063-of-00101.safetensors", + "model.layers.58.mlp.experts.59.gate_proj.weight": "model-00063-of-00101.safetensors", + "model.layers.58.mlp.experts.59.up_proj.weight": "model-00063-of-00101.safetensors", + "model.layers.58.mlp.experts.59.down_proj.weight": "model-00063-of-00101.safetensors", + "model.layers.58.mlp.experts.60.gate_proj.weight": "model-00063-of-00101.safetensors", + "model.layers.58.mlp.experts.60.up_proj.weight": "model-00063-of-00101.safetensors", + "model.layers.58.mlp.experts.60.down_proj.weight": "model-00063-of-00101.safetensors", + "model.layers.58.mlp.experts.61.gate_proj.weight": "model-00063-of-00101.safetensors", + "model.layers.58.mlp.experts.61.up_proj.weight": "model-00063-of-00101.safetensors", + "model.layers.58.mlp.experts.61.down_proj.weight": "model-00063-of-00101.safetensors", + "model.layers.58.mlp.experts.62.gate_proj.weight": "model-00063-of-00101.safetensors", + "model.layers.58.mlp.experts.62.up_proj.weight": "model-00063-of-00101.safetensors", + "model.layers.58.mlp.experts.62.down_proj.weight": "model-00063-of-00101.safetensors", + "model.layers.58.mlp.experts.63.gate_proj.weight": "model-00063-of-00101.safetensors", + "model.layers.58.mlp.experts.63.up_proj.weight": "model-00063-of-00101.safetensors", + "model.layers.58.mlp.experts.63.down_proj.weight": "model-00063-of-00101.safetensors", + "model.layers.58.mlp.experts.64.gate_proj.weight": "model-00063-of-00101.safetensors", + "model.layers.58.mlp.experts.64.up_proj.weight": "model-00063-of-00101.safetensors", + "model.layers.58.mlp.experts.64.down_proj.weight": "model-00063-of-00101.safetensors", + "model.layers.58.mlp.experts.65.gate_proj.weight": "model-00063-of-00101.safetensors", + "model.layers.58.mlp.experts.65.up_proj.weight": "model-00063-of-00101.safetensors", + "model.layers.58.mlp.experts.65.down_proj.weight": "model-00063-of-00101.safetensors", + "model.layers.58.mlp.experts.66.gate_proj.weight": "model-00063-of-00101.safetensors", + "model.layers.58.mlp.experts.66.up_proj.weight": "model-00063-of-00101.safetensors", + "model.layers.58.mlp.experts.66.down_proj.weight": "model-00063-of-00101.safetensors", + "model.layers.58.mlp.experts.67.gate_proj.weight": "model-00063-of-00101.safetensors", + "model.layers.58.mlp.experts.67.up_proj.weight": "model-00063-of-00101.safetensors", + "model.layers.58.mlp.experts.67.down_proj.weight": "model-00063-of-00101.safetensors", + "model.layers.58.mlp.experts.68.gate_proj.weight": "model-00063-of-00101.safetensors", + "model.layers.58.mlp.experts.68.up_proj.weight": "model-00063-of-00101.safetensors", + "model.layers.58.mlp.experts.68.down_proj.weight": "model-00063-of-00101.safetensors", + "model.layers.58.mlp.experts.69.gate_proj.weight": "model-00063-of-00101.safetensors", + "model.layers.58.mlp.experts.69.up_proj.weight": "model-00063-of-00101.safetensors", + "model.layers.58.mlp.experts.69.down_proj.weight": "model-00063-of-00101.safetensors", + "model.layers.58.mlp.experts.70.gate_proj.weight": "model-00063-of-00101.safetensors", + "model.layers.58.mlp.experts.70.up_proj.weight": "model-00063-of-00101.safetensors", + "model.layers.58.mlp.experts.70.down_proj.weight": "model-00063-of-00101.safetensors", + "model.layers.58.mlp.experts.71.gate_proj.weight": "model-00063-of-00101.safetensors", + "model.layers.58.mlp.experts.71.up_proj.weight": "model-00063-of-00101.safetensors", + "model.layers.58.mlp.experts.71.down_proj.weight": "model-00063-of-00101.safetensors", + "model.layers.58.mlp.experts.72.gate_proj.weight": "model-00063-of-00101.safetensors", + "model.layers.58.mlp.experts.72.up_proj.weight": "model-00063-of-00101.safetensors", + "model.layers.58.mlp.experts.72.down_proj.weight": "model-00063-of-00101.safetensors", + "model.layers.58.mlp.experts.73.gate_proj.weight": "model-00063-of-00101.safetensors", + "model.layers.58.mlp.experts.73.up_proj.weight": "model-00063-of-00101.safetensors", + "model.layers.58.mlp.experts.73.down_proj.weight": "model-00063-of-00101.safetensors", + "model.layers.58.mlp.experts.74.gate_proj.weight": "model-00063-of-00101.safetensors", + "model.layers.58.mlp.experts.74.up_proj.weight": "model-00063-of-00101.safetensors", + "model.layers.58.mlp.experts.74.down_proj.weight": "model-00063-of-00101.safetensors", + "model.layers.58.mlp.experts.75.gate_proj.weight": "model-00063-of-00101.safetensors", + "model.layers.58.mlp.experts.75.up_proj.weight": "model-00063-of-00101.safetensors", + "model.layers.58.mlp.experts.75.down_proj.weight": "model-00063-of-00101.safetensors", + "model.layers.58.mlp.experts.76.gate_proj.weight": "model-00063-of-00101.safetensors", + "model.layers.58.mlp.experts.76.up_proj.weight": "model-00063-of-00101.safetensors", + "model.layers.58.mlp.experts.76.down_proj.weight": "model-00063-of-00101.safetensors", + "model.layers.58.mlp.experts.77.gate_proj.weight": "model-00063-of-00101.safetensors", + "model.layers.58.mlp.experts.77.up_proj.weight": "model-00063-of-00101.safetensors", + "model.layers.58.mlp.experts.77.down_proj.weight": "model-00063-of-00101.safetensors", + "model.layers.58.mlp.experts.78.gate_proj.weight": "model-00063-of-00101.safetensors", + "model.layers.58.mlp.experts.78.up_proj.weight": "model-00063-of-00101.safetensors", + "model.layers.58.mlp.experts.78.down_proj.weight": "model-00063-of-00101.safetensors", + "model.layers.58.mlp.experts.79.gate_proj.weight": "model-00063-of-00101.safetensors", + "model.layers.58.mlp.experts.79.up_proj.weight": "model-00063-of-00101.safetensors", + "model.layers.58.mlp.experts.79.down_proj.weight": "model-00063-of-00101.safetensors", + "model.layers.58.mlp.experts.80.gate_proj.weight": "model-00063-of-00101.safetensors", + "model.layers.58.mlp.experts.80.up_proj.weight": "model-00063-of-00101.safetensors", + "model.layers.58.mlp.experts.80.down_proj.weight": "model-00063-of-00101.safetensors", + "model.layers.58.mlp.experts.81.gate_proj.weight": "model-00063-of-00101.safetensors", + "model.layers.58.mlp.experts.81.up_proj.weight": "model-00063-of-00101.safetensors", + "model.layers.58.mlp.experts.81.down_proj.weight": "model-00063-of-00101.safetensors", + "model.layers.58.mlp.experts.82.gate_proj.weight": "model-00063-of-00101.safetensors", + "model.layers.58.mlp.experts.82.up_proj.weight": "model-00063-of-00101.safetensors", + "model.layers.58.mlp.experts.82.down_proj.weight": "model-00063-of-00101.safetensors", + "model.layers.58.mlp.experts.83.gate_proj.weight": "model-00063-of-00101.safetensors", + "model.layers.58.mlp.experts.83.up_proj.weight": "model-00063-of-00101.safetensors", + "model.layers.58.mlp.experts.83.down_proj.weight": "model-00063-of-00101.safetensors", + "model.layers.58.mlp.experts.84.gate_proj.weight": "model-00063-of-00101.safetensors", + "model.layers.58.mlp.experts.84.up_proj.weight": "model-00063-of-00101.safetensors", + "model.layers.58.mlp.experts.84.down_proj.weight": "model-00063-of-00101.safetensors", + "model.layers.58.mlp.experts.85.gate_proj.weight": "model-00063-of-00101.safetensors", + "model.layers.58.mlp.experts.85.up_proj.weight": "model-00063-of-00101.safetensors", + "model.layers.58.mlp.experts.85.down_proj.weight": "model-00063-of-00101.safetensors", + "model.layers.58.mlp.experts.86.gate_proj.weight": "model-00063-of-00101.safetensors", + "model.layers.58.mlp.experts.86.up_proj.weight": "model-00063-of-00101.safetensors", + "model.layers.58.mlp.experts.86.down_proj.weight": "model-00063-of-00101.safetensors", + "model.layers.58.mlp.experts.87.gate_proj.weight": "model-00063-of-00101.safetensors", + "model.layers.58.mlp.experts.87.up_proj.weight": "model-00063-of-00101.safetensors", + "model.layers.58.mlp.experts.87.down_proj.weight": "model-00063-of-00101.safetensors", + "model.layers.58.mlp.experts.88.gate_proj.weight": "model-00063-of-00101.safetensors", + "model.layers.58.mlp.experts.88.up_proj.weight": "model-00063-of-00101.safetensors", + "model.layers.58.mlp.experts.88.down_proj.weight": "model-00063-of-00101.safetensors", + "model.layers.58.mlp.experts.89.gate_proj.weight": "model-00063-of-00101.safetensors", + "model.layers.58.mlp.experts.89.up_proj.weight": "model-00063-of-00101.safetensors", + "model.layers.58.mlp.experts.89.down_proj.weight": "model-00063-of-00101.safetensors", + "model.layers.58.mlp.experts.90.gate_proj.weight": "model-00063-of-00101.safetensors", + "model.layers.58.mlp.experts.90.up_proj.weight": "model-00063-of-00101.safetensors", + "model.layers.58.mlp.experts.90.down_proj.weight": "model-00063-of-00101.safetensors", + "model.layers.58.mlp.experts.91.gate_proj.weight": "model-00063-of-00101.safetensors", + "model.layers.58.mlp.experts.91.up_proj.weight": "model-00063-of-00101.safetensors", + "model.layers.58.mlp.experts.91.down_proj.weight": "model-00063-of-00101.safetensors", + "model.layers.58.mlp.experts.92.gate_proj.weight": "model-00063-of-00101.safetensors", + "model.layers.58.mlp.experts.92.up_proj.weight": "model-00063-of-00101.safetensors", + "model.layers.58.mlp.experts.92.down_proj.weight": "model-00063-of-00101.safetensors", + "model.layers.58.mlp.experts.93.gate_proj.weight": "model-00063-of-00101.safetensors", + "model.layers.58.mlp.experts.93.up_proj.weight": "model-00063-of-00101.safetensors", + "model.layers.58.mlp.experts.93.down_proj.weight": "model-00063-of-00101.safetensors", + "model.layers.58.mlp.experts.94.gate_proj.weight": "model-00063-of-00101.safetensors", + "model.layers.58.mlp.experts.94.up_proj.weight": "model-00064-of-00101.safetensors", + "model.layers.58.mlp.experts.94.down_proj.weight": "model-00064-of-00101.safetensors", + "model.layers.58.mlp.experts.95.gate_proj.weight": "model-00064-of-00101.safetensors", + "model.layers.58.mlp.experts.95.up_proj.weight": "model-00064-of-00101.safetensors", + "model.layers.58.mlp.experts.95.down_proj.weight": "model-00064-of-00101.safetensors", + "model.layers.58.mlp.experts.96.gate_proj.weight": "model-00064-of-00101.safetensors", + "model.layers.58.mlp.experts.96.up_proj.weight": "model-00064-of-00101.safetensors", + "model.layers.58.mlp.experts.96.down_proj.weight": "model-00064-of-00101.safetensors", + "model.layers.58.mlp.experts.97.gate_proj.weight": "model-00064-of-00101.safetensors", + "model.layers.58.mlp.experts.97.up_proj.weight": "model-00064-of-00101.safetensors", + "model.layers.58.mlp.experts.97.down_proj.weight": "model-00064-of-00101.safetensors", + "model.layers.58.mlp.experts.98.gate_proj.weight": "model-00064-of-00101.safetensors", + "model.layers.58.mlp.experts.98.up_proj.weight": "model-00064-of-00101.safetensors", + "model.layers.58.mlp.experts.98.down_proj.weight": "model-00064-of-00101.safetensors", + "model.layers.58.mlp.experts.99.gate_proj.weight": "model-00064-of-00101.safetensors", + "model.layers.58.mlp.experts.99.up_proj.weight": "model-00064-of-00101.safetensors", + "model.layers.58.mlp.experts.99.down_proj.weight": "model-00064-of-00101.safetensors", + "model.layers.58.mlp.experts.100.gate_proj.weight": "model-00064-of-00101.safetensors", + "model.layers.58.mlp.experts.100.up_proj.weight": "model-00064-of-00101.safetensors", + "model.layers.58.mlp.experts.100.down_proj.weight": "model-00064-of-00101.safetensors", + "model.layers.58.mlp.experts.101.gate_proj.weight": "model-00064-of-00101.safetensors", + "model.layers.58.mlp.experts.101.up_proj.weight": "model-00064-of-00101.safetensors", + "model.layers.58.mlp.experts.101.down_proj.weight": "model-00064-of-00101.safetensors", + "model.layers.58.mlp.experts.102.gate_proj.weight": "model-00064-of-00101.safetensors", + "model.layers.58.mlp.experts.102.up_proj.weight": "model-00064-of-00101.safetensors", + "model.layers.58.mlp.experts.102.down_proj.weight": "model-00064-of-00101.safetensors", + "model.layers.58.mlp.experts.103.gate_proj.weight": "model-00064-of-00101.safetensors", + "model.layers.58.mlp.experts.103.up_proj.weight": "model-00064-of-00101.safetensors", + "model.layers.58.mlp.experts.103.down_proj.weight": "model-00064-of-00101.safetensors", + "model.layers.58.mlp.experts.104.gate_proj.weight": "model-00064-of-00101.safetensors", + "model.layers.58.mlp.experts.104.up_proj.weight": "model-00064-of-00101.safetensors", + "model.layers.58.mlp.experts.104.down_proj.weight": "model-00064-of-00101.safetensors", + "model.layers.58.mlp.experts.105.gate_proj.weight": "model-00064-of-00101.safetensors", + "model.layers.58.mlp.experts.105.up_proj.weight": "model-00064-of-00101.safetensors", + "model.layers.58.mlp.experts.105.down_proj.weight": "model-00064-of-00101.safetensors", + "model.layers.58.mlp.experts.106.gate_proj.weight": "model-00064-of-00101.safetensors", + "model.layers.58.mlp.experts.106.up_proj.weight": "model-00064-of-00101.safetensors", + "model.layers.58.mlp.experts.106.down_proj.weight": "model-00064-of-00101.safetensors", + "model.layers.58.mlp.experts.107.gate_proj.weight": "model-00064-of-00101.safetensors", + "model.layers.58.mlp.experts.107.up_proj.weight": "model-00064-of-00101.safetensors", + "model.layers.58.mlp.experts.107.down_proj.weight": "model-00064-of-00101.safetensors", + "model.layers.58.mlp.experts.108.gate_proj.weight": "model-00064-of-00101.safetensors", + "model.layers.58.mlp.experts.108.up_proj.weight": "model-00064-of-00101.safetensors", + "model.layers.58.mlp.experts.108.down_proj.weight": "model-00064-of-00101.safetensors", + "model.layers.58.mlp.experts.109.gate_proj.weight": "model-00064-of-00101.safetensors", + "model.layers.58.mlp.experts.109.up_proj.weight": "model-00064-of-00101.safetensors", + "model.layers.58.mlp.experts.109.down_proj.weight": "model-00064-of-00101.safetensors", + "model.layers.58.mlp.experts.110.gate_proj.weight": "model-00064-of-00101.safetensors", + "model.layers.58.mlp.experts.110.up_proj.weight": "model-00064-of-00101.safetensors", + "model.layers.58.mlp.experts.110.down_proj.weight": "model-00064-of-00101.safetensors", + "model.layers.58.mlp.experts.111.gate_proj.weight": "model-00064-of-00101.safetensors", + "model.layers.58.mlp.experts.111.up_proj.weight": "model-00064-of-00101.safetensors", + "model.layers.58.mlp.experts.111.down_proj.weight": "model-00064-of-00101.safetensors", + "model.layers.58.mlp.experts.112.gate_proj.weight": "model-00064-of-00101.safetensors", + "model.layers.58.mlp.experts.112.up_proj.weight": "model-00064-of-00101.safetensors", + "model.layers.58.mlp.experts.112.down_proj.weight": "model-00064-of-00101.safetensors", + "model.layers.58.mlp.experts.113.gate_proj.weight": "model-00064-of-00101.safetensors", + "model.layers.58.mlp.experts.113.up_proj.weight": "model-00064-of-00101.safetensors", + "model.layers.58.mlp.experts.113.down_proj.weight": "model-00064-of-00101.safetensors", + "model.layers.58.mlp.experts.114.gate_proj.weight": "model-00064-of-00101.safetensors", + "model.layers.58.mlp.experts.114.up_proj.weight": "model-00064-of-00101.safetensors", + "model.layers.58.mlp.experts.114.down_proj.weight": "model-00064-of-00101.safetensors", + "model.layers.58.mlp.experts.115.gate_proj.weight": "model-00064-of-00101.safetensors", + "model.layers.58.mlp.experts.115.up_proj.weight": "model-00064-of-00101.safetensors", + "model.layers.58.mlp.experts.115.down_proj.weight": "model-00064-of-00101.safetensors", + "model.layers.58.mlp.experts.116.gate_proj.weight": "model-00064-of-00101.safetensors", + "model.layers.58.mlp.experts.116.up_proj.weight": "model-00064-of-00101.safetensors", + "model.layers.58.mlp.experts.116.down_proj.weight": "model-00064-of-00101.safetensors", + "model.layers.58.mlp.experts.117.gate_proj.weight": "model-00064-of-00101.safetensors", + "model.layers.58.mlp.experts.117.up_proj.weight": "model-00064-of-00101.safetensors", + "model.layers.58.mlp.experts.117.down_proj.weight": "model-00064-of-00101.safetensors", + "model.layers.58.mlp.experts.118.gate_proj.weight": "model-00064-of-00101.safetensors", + "model.layers.58.mlp.experts.118.up_proj.weight": "model-00064-of-00101.safetensors", + "model.layers.58.mlp.experts.118.down_proj.weight": "model-00064-of-00101.safetensors", + "model.layers.58.mlp.experts.119.gate_proj.weight": "model-00064-of-00101.safetensors", + "model.layers.58.mlp.experts.119.up_proj.weight": "model-00064-of-00101.safetensors", + "model.layers.58.mlp.experts.119.down_proj.weight": "model-00064-of-00101.safetensors", + "model.layers.58.mlp.gate.weight": "model-00064-of-00101.safetensors", + "model.layers.58.mlp.gate.e_score_correction_bias": "model-00064-of-00101.safetensors", + "model.layers.58.mlp.shared_experts.gate_proj.weight": "model-00064-of-00101.safetensors", + "model.layers.58.mlp.shared_experts.up_proj.weight": "model-00064-of-00101.safetensors", + "model.layers.58.mlp.shared_experts.down_proj.weight": "model-00064-of-00101.safetensors", + "model.layers.58.input_layernorm.weight": "model-00064-of-00101.safetensors", + "model.layers.58.post_attention_layernorm.weight": "model-00064-of-00101.safetensors", + "model.layers.59.self_attn.q_proj.weight": "model-00064-of-00101.safetensors", + "model.layers.59.self_attn.q_proj.bias": "model-00064-of-00101.safetensors", + "model.layers.59.self_attn.k_proj.weight": "model-00064-of-00101.safetensors", + "model.layers.59.self_attn.k_proj.bias": "model-00064-of-00101.safetensors", + "model.layers.59.self_attn.v_proj.weight": "model-00064-of-00101.safetensors", + "model.layers.59.self_attn.v_proj.bias": "model-00064-of-00101.safetensors", + "model.layers.59.self_attn.o_proj.weight": "model-00064-of-00101.safetensors", + "model.layers.59.self_attn.q_norm.weight": "model-00064-of-00101.safetensors", + "model.layers.59.self_attn.k_norm.weight": "model-00064-of-00101.safetensors", + "model.layers.59.mlp.experts.0.gate_proj.weight": "model-00064-of-00101.safetensors", + "model.layers.59.mlp.experts.0.up_proj.weight": "model-00064-of-00101.safetensors", + "model.layers.59.mlp.experts.0.down_proj.weight": "model-00064-of-00101.safetensors", + "model.layers.59.mlp.experts.1.gate_proj.weight": "model-00064-of-00101.safetensors", + "model.layers.59.mlp.experts.1.up_proj.weight": "model-00064-of-00101.safetensors", + "model.layers.59.mlp.experts.1.down_proj.weight": "model-00064-of-00101.safetensors", + "model.layers.59.mlp.experts.2.gate_proj.weight": "model-00064-of-00101.safetensors", + "model.layers.59.mlp.experts.2.up_proj.weight": "model-00064-of-00101.safetensors", + "model.layers.59.mlp.experts.2.down_proj.weight": "model-00064-of-00101.safetensors", + "model.layers.59.mlp.experts.3.gate_proj.weight": "model-00064-of-00101.safetensors", + "model.layers.59.mlp.experts.3.up_proj.weight": "model-00064-of-00101.safetensors", + "model.layers.59.mlp.experts.3.down_proj.weight": "model-00064-of-00101.safetensors", + "model.layers.59.mlp.experts.4.gate_proj.weight": "model-00064-of-00101.safetensors", + "model.layers.59.mlp.experts.4.up_proj.weight": "model-00064-of-00101.safetensors", + "model.layers.59.mlp.experts.4.down_proj.weight": "model-00064-of-00101.safetensors", + "model.layers.59.mlp.experts.5.gate_proj.weight": "model-00064-of-00101.safetensors", + "model.layers.59.mlp.experts.5.up_proj.weight": "model-00064-of-00101.safetensors", + "model.layers.59.mlp.experts.5.down_proj.weight": "model-00064-of-00101.safetensors", + "model.layers.59.mlp.experts.6.gate_proj.weight": "model-00064-of-00101.safetensors", + "model.layers.59.mlp.experts.6.up_proj.weight": "model-00064-of-00101.safetensors", + "model.layers.59.mlp.experts.6.down_proj.weight": "model-00064-of-00101.safetensors", + "model.layers.59.mlp.experts.7.gate_proj.weight": "model-00064-of-00101.safetensors", + "model.layers.59.mlp.experts.7.up_proj.weight": "model-00064-of-00101.safetensors", + "model.layers.59.mlp.experts.7.down_proj.weight": "model-00064-of-00101.safetensors", + "model.layers.59.mlp.experts.8.gate_proj.weight": "model-00064-of-00101.safetensors", + "model.layers.59.mlp.experts.8.up_proj.weight": "model-00064-of-00101.safetensors", + "model.layers.59.mlp.experts.8.down_proj.weight": "model-00064-of-00101.safetensors", + "model.layers.59.mlp.experts.9.gate_proj.weight": "model-00064-of-00101.safetensors", + "model.layers.59.mlp.experts.9.up_proj.weight": "model-00064-of-00101.safetensors", + "model.layers.59.mlp.experts.9.down_proj.weight": "model-00064-of-00101.safetensors", + "model.layers.59.mlp.experts.10.gate_proj.weight": "model-00064-of-00101.safetensors", + "model.layers.59.mlp.experts.10.up_proj.weight": "model-00064-of-00101.safetensors", + "model.layers.59.mlp.experts.10.down_proj.weight": "model-00064-of-00101.safetensors", + "model.layers.59.mlp.experts.11.gate_proj.weight": "model-00064-of-00101.safetensors", + "model.layers.59.mlp.experts.11.up_proj.weight": "model-00064-of-00101.safetensors", + "model.layers.59.mlp.experts.11.down_proj.weight": "model-00064-of-00101.safetensors", + "model.layers.59.mlp.experts.12.gate_proj.weight": "model-00064-of-00101.safetensors", + "model.layers.59.mlp.experts.12.up_proj.weight": "model-00064-of-00101.safetensors", + "model.layers.59.mlp.experts.12.down_proj.weight": "model-00064-of-00101.safetensors", + "model.layers.59.mlp.experts.13.gate_proj.weight": "model-00064-of-00101.safetensors", + "model.layers.59.mlp.experts.13.up_proj.weight": "model-00064-of-00101.safetensors", + "model.layers.59.mlp.experts.13.down_proj.weight": "model-00064-of-00101.safetensors", + "model.layers.59.mlp.experts.14.gate_proj.weight": "model-00064-of-00101.safetensors", + "model.layers.59.mlp.experts.14.up_proj.weight": "model-00064-of-00101.safetensors", + "model.layers.59.mlp.experts.14.down_proj.weight": "model-00064-of-00101.safetensors", + "model.layers.59.mlp.experts.15.gate_proj.weight": "model-00064-of-00101.safetensors", + "model.layers.59.mlp.experts.15.up_proj.weight": "model-00064-of-00101.safetensors", + "model.layers.59.mlp.experts.15.down_proj.weight": "model-00064-of-00101.safetensors", + "model.layers.59.mlp.experts.16.gate_proj.weight": "model-00064-of-00101.safetensors", + "model.layers.59.mlp.experts.16.up_proj.weight": "model-00064-of-00101.safetensors", + "model.layers.59.mlp.experts.16.down_proj.weight": "model-00064-of-00101.safetensors", + "model.layers.59.mlp.experts.17.gate_proj.weight": "model-00064-of-00101.safetensors", + "model.layers.59.mlp.experts.17.up_proj.weight": "model-00064-of-00101.safetensors", + "model.layers.59.mlp.experts.17.down_proj.weight": "model-00064-of-00101.safetensors", + "model.layers.59.mlp.experts.18.gate_proj.weight": "model-00064-of-00101.safetensors", + "model.layers.59.mlp.experts.18.up_proj.weight": "model-00064-of-00101.safetensors", + "model.layers.59.mlp.experts.18.down_proj.weight": "model-00064-of-00101.safetensors", + "model.layers.59.mlp.experts.19.gate_proj.weight": "model-00064-of-00101.safetensors", + "model.layers.59.mlp.experts.19.up_proj.weight": "model-00064-of-00101.safetensors", + "model.layers.59.mlp.experts.19.down_proj.weight": "model-00064-of-00101.safetensors", + "model.layers.59.mlp.experts.20.gate_proj.weight": "model-00064-of-00101.safetensors", + "model.layers.59.mlp.experts.20.up_proj.weight": "model-00064-of-00101.safetensors", + "model.layers.59.mlp.experts.20.down_proj.weight": "model-00064-of-00101.safetensors", + "model.layers.59.mlp.experts.21.gate_proj.weight": "model-00064-of-00101.safetensors", + "model.layers.59.mlp.experts.21.up_proj.weight": "model-00064-of-00101.safetensors", + "model.layers.59.mlp.experts.21.down_proj.weight": "model-00064-of-00101.safetensors", + "model.layers.59.mlp.experts.22.gate_proj.weight": "model-00064-of-00101.safetensors", + "model.layers.59.mlp.experts.22.up_proj.weight": "model-00064-of-00101.safetensors", + "model.layers.59.mlp.experts.22.down_proj.weight": "model-00064-of-00101.safetensors", + "model.layers.59.mlp.experts.23.gate_proj.weight": "model-00064-of-00101.safetensors", + "model.layers.59.mlp.experts.23.up_proj.weight": "model-00064-of-00101.safetensors", + "model.layers.59.mlp.experts.23.down_proj.weight": "model-00064-of-00101.safetensors", + "model.layers.59.mlp.experts.24.gate_proj.weight": "model-00064-of-00101.safetensors", + "model.layers.59.mlp.experts.24.up_proj.weight": "model-00064-of-00101.safetensors", + "model.layers.59.mlp.experts.24.down_proj.weight": "model-00064-of-00101.safetensors", + "model.layers.59.mlp.experts.25.gate_proj.weight": "model-00064-of-00101.safetensors", + "model.layers.59.mlp.experts.25.up_proj.weight": "model-00064-of-00101.safetensors", + "model.layers.59.mlp.experts.25.down_proj.weight": "model-00064-of-00101.safetensors", + "model.layers.59.mlp.experts.26.gate_proj.weight": "model-00064-of-00101.safetensors", + "model.layers.59.mlp.experts.26.up_proj.weight": "model-00064-of-00101.safetensors", + "model.layers.59.mlp.experts.26.down_proj.weight": "model-00064-of-00101.safetensors", + "model.layers.59.mlp.experts.27.gate_proj.weight": "model-00064-of-00101.safetensors", + "model.layers.59.mlp.experts.27.up_proj.weight": "model-00064-of-00101.safetensors", + "model.layers.59.mlp.experts.27.down_proj.weight": "model-00064-of-00101.safetensors", + "model.layers.59.mlp.experts.28.gate_proj.weight": "model-00064-of-00101.safetensors", + "model.layers.59.mlp.experts.28.up_proj.weight": "model-00064-of-00101.safetensors", + "model.layers.59.mlp.experts.28.down_proj.weight": "model-00064-of-00101.safetensors", + "model.layers.59.mlp.experts.29.gate_proj.weight": "model-00064-of-00101.safetensors", + "model.layers.59.mlp.experts.29.up_proj.weight": "model-00064-of-00101.safetensors", + "model.layers.59.mlp.experts.29.down_proj.weight": "model-00064-of-00101.safetensors", + "model.layers.59.mlp.experts.30.gate_proj.weight": "model-00064-of-00101.safetensors", + "model.layers.59.mlp.experts.30.up_proj.weight": "model-00064-of-00101.safetensors", + "model.layers.59.mlp.experts.30.down_proj.weight": "model-00064-of-00101.safetensors", + "model.layers.59.mlp.experts.31.gate_proj.weight": "model-00064-of-00101.safetensors", + "model.layers.59.mlp.experts.31.up_proj.weight": "model-00064-of-00101.safetensors", + "model.layers.59.mlp.experts.31.down_proj.weight": "model-00064-of-00101.safetensors", + "model.layers.59.mlp.experts.32.gate_proj.weight": "model-00064-of-00101.safetensors", + "model.layers.59.mlp.experts.32.up_proj.weight": "model-00064-of-00101.safetensors", + "model.layers.59.mlp.experts.32.down_proj.weight": "model-00064-of-00101.safetensors", + "model.layers.59.mlp.experts.33.gate_proj.weight": "model-00064-of-00101.safetensors", + "model.layers.59.mlp.experts.33.up_proj.weight": "model-00064-of-00101.safetensors", + "model.layers.59.mlp.experts.33.down_proj.weight": "model-00064-of-00101.safetensors", + "model.layers.59.mlp.experts.34.gate_proj.weight": "model-00064-of-00101.safetensors", + "model.layers.59.mlp.experts.34.up_proj.weight": "model-00064-of-00101.safetensors", + "model.layers.59.mlp.experts.34.down_proj.weight": "model-00064-of-00101.safetensors", + "model.layers.59.mlp.experts.35.gate_proj.weight": "model-00064-of-00101.safetensors", + "model.layers.59.mlp.experts.35.up_proj.weight": "model-00064-of-00101.safetensors", + "model.layers.59.mlp.experts.35.down_proj.weight": "model-00064-of-00101.safetensors", + "model.layers.59.mlp.experts.36.gate_proj.weight": "model-00064-of-00101.safetensors", + "model.layers.59.mlp.experts.36.up_proj.weight": "model-00064-of-00101.safetensors", + "model.layers.59.mlp.experts.36.down_proj.weight": "model-00064-of-00101.safetensors", + "model.layers.59.mlp.experts.37.gate_proj.weight": "model-00064-of-00101.safetensors", + "model.layers.59.mlp.experts.37.up_proj.weight": "model-00064-of-00101.safetensors", + "model.layers.59.mlp.experts.37.down_proj.weight": "model-00064-of-00101.safetensors", + "model.layers.59.mlp.experts.38.gate_proj.weight": "model-00064-of-00101.safetensors", + "model.layers.59.mlp.experts.38.up_proj.weight": "model-00064-of-00101.safetensors", + "model.layers.59.mlp.experts.38.down_proj.weight": "model-00064-of-00101.safetensors", + "model.layers.59.mlp.experts.39.gate_proj.weight": "model-00064-of-00101.safetensors", + "model.layers.59.mlp.experts.39.up_proj.weight": "model-00064-of-00101.safetensors", + "model.layers.59.mlp.experts.39.down_proj.weight": "model-00064-of-00101.safetensors", + "model.layers.59.mlp.experts.40.gate_proj.weight": "model-00064-of-00101.safetensors", + "model.layers.59.mlp.experts.40.up_proj.weight": "model-00064-of-00101.safetensors", + "model.layers.59.mlp.experts.40.down_proj.weight": "model-00064-of-00101.safetensors", + "model.layers.59.mlp.experts.41.gate_proj.weight": "model-00064-of-00101.safetensors", + "model.layers.59.mlp.experts.41.up_proj.weight": "model-00064-of-00101.safetensors", + "model.layers.59.mlp.experts.41.down_proj.weight": "model-00064-of-00101.safetensors", + "model.layers.59.mlp.experts.42.gate_proj.weight": "model-00064-of-00101.safetensors", + "model.layers.59.mlp.experts.42.up_proj.weight": "model-00064-of-00101.safetensors", + "model.layers.59.mlp.experts.42.down_proj.weight": "model-00064-of-00101.safetensors", + "model.layers.59.mlp.experts.43.gate_proj.weight": "model-00064-of-00101.safetensors", + "model.layers.59.mlp.experts.43.up_proj.weight": "model-00064-of-00101.safetensors", + "model.layers.59.mlp.experts.43.down_proj.weight": "model-00064-of-00101.safetensors", + "model.layers.59.mlp.experts.44.gate_proj.weight": "model-00064-of-00101.safetensors", + "model.layers.59.mlp.experts.44.up_proj.weight": "model-00064-of-00101.safetensors", + "model.layers.59.mlp.experts.44.down_proj.weight": "model-00064-of-00101.safetensors", + "model.layers.59.mlp.experts.45.gate_proj.weight": "model-00064-of-00101.safetensors", + "model.layers.59.mlp.experts.45.up_proj.weight": "model-00064-of-00101.safetensors", + "model.layers.59.mlp.experts.45.down_proj.weight": "model-00064-of-00101.safetensors", + "model.layers.59.mlp.experts.46.gate_proj.weight": "model-00064-of-00101.safetensors", + "model.layers.59.mlp.experts.46.up_proj.weight": "model-00064-of-00101.safetensors", + "model.layers.59.mlp.experts.46.down_proj.weight": "model-00064-of-00101.safetensors", + "model.layers.59.mlp.experts.47.gate_proj.weight": "model-00064-of-00101.safetensors", + "model.layers.59.mlp.experts.47.up_proj.weight": "model-00064-of-00101.safetensors", + "model.layers.59.mlp.experts.47.down_proj.weight": "model-00064-of-00101.safetensors", + "model.layers.59.mlp.experts.48.gate_proj.weight": "model-00064-of-00101.safetensors", + "model.layers.59.mlp.experts.48.up_proj.weight": "model-00064-of-00101.safetensors", + "model.layers.59.mlp.experts.48.down_proj.weight": "model-00064-of-00101.safetensors", + "model.layers.59.mlp.experts.49.gate_proj.weight": "model-00064-of-00101.safetensors", + "model.layers.59.mlp.experts.49.up_proj.weight": "model-00064-of-00101.safetensors", + "model.layers.59.mlp.experts.49.down_proj.weight": "model-00064-of-00101.safetensors", + "model.layers.59.mlp.experts.50.gate_proj.weight": "model-00064-of-00101.safetensors", + "model.layers.59.mlp.experts.50.up_proj.weight": "model-00064-of-00101.safetensors", + "model.layers.59.mlp.experts.50.down_proj.weight": "model-00064-of-00101.safetensors", + "model.layers.59.mlp.experts.51.gate_proj.weight": "model-00064-of-00101.safetensors", + "model.layers.59.mlp.experts.51.up_proj.weight": "model-00064-of-00101.safetensors", + "model.layers.59.mlp.experts.51.down_proj.weight": "model-00064-of-00101.safetensors", + "model.layers.59.mlp.experts.52.gate_proj.weight": "model-00064-of-00101.safetensors", + "model.layers.59.mlp.experts.52.up_proj.weight": "model-00064-of-00101.safetensors", + "model.layers.59.mlp.experts.52.down_proj.weight": "model-00064-of-00101.safetensors", + "model.layers.59.mlp.experts.53.gate_proj.weight": "model-00064-of-00101.safetensors", + "model.layers.59.mlp.experts.53.up_proj.weight": "model-00064-of-00101.safetensors", + "model.layers.59.mlp.experts.53.down_proj.weight": "model-00064-of-00101.safetensors", + "model.layers.59.mlp.experts.54.gate_proj.weight": "model-00064-of-00101.safetensors", + "model.layers.59.mlp.experts.54.up_proj.weight": "model-00064-of-00101.safetensors", + "model.layers.59.mlp.experts.54.down_proj.weight": "model-00064-of-00101.safetensors", + "model.layers.59.mlp.experts.55.gate_proj.weight": "model-00064-of-00101.safetensors", + "model.layers.59.mlp.experts.55.up_proj.weight": "model-00064-of-00101.safetensors", + "model.layers.59.mlp.experts.55.down_proj.weight": "model-00064-of-00101.safetensors", + "model.layers.59.mlp.experts.56.gate_proj.weight": "model-00064-of-00101.safetensors", + "model.layers.59.mlp.experts.56.up_proj.weight": "model-00064-of-00101.safetensors", + "model.layers.59.mlp.experts.56.down_proj.weight": "model-00064-of-00101.safetensors", + "model.layers.59.mlp.experts.57.gate_proj.weight": "model-00064-of-00101.safetensors", + "model.layers.59.mlp.experts.57.up_proj.weight": "model-00064-of-00101.safetensors", + "model.layers.59.mlp.experts.57.down_proj.weight": "model-00064-of-00101.safetensors", + "model.layers.59.mlp.experts.58.gate_proj.weight": "model-00064-of-00101.safetensors", + "model.layers.59.mlp.experts.58.up_proj.weight": "model-00064-of-00101.safetensors", + "model.layers.59.mlp.experts.58.down_proj.weight": "model-00064-of-00101.safetensors", + "model.layers.59.mlp.experts.59.gate_proj.weight": "model-00064-of-00101.safetensors", + "model.layers.59.mlp.experts.59.up_proj.weight": "model-00064-of-00101.safetensors", + "model.layers.59.mlp.experts.59.down_proj.weight": "model-00064-of-00101.safetensors", + "model.layers.59.mlp.experts.60.gate_proj.weight": "model-00064-of-00101.safetensors", + "model.layers.59.mlp.experts.60.up_proj.weight": "model-00064-of-00101.safetensors", + "model.layers.59.mlp.experts.60.down_proj.weight": "model-00064-of-00101.safetensors", + "model.layers.59.mlp.experts.61.gate_proj.weight": "model-00064-of-00101.safetensors", + "model.layers.59.mlp.experts.61.up_proj.weight": "model-00064-of-00101.safetensors", + "model.layers.59.mlp.experts.61.down_proj.weight": "model-00064-of-00101.safetensors", + "model.layers.59.mlp.experts.62.gate_proj.weight": "model-00064-of-00101.safetensors", + "model.layers.59.mlp.experts.62.up_proj.weight": "model-00064-of-00101.safetensors", + "model.layers.59.mlp.experts.62.down_proj.weight": "model-00064-of-00101.safetensors", + "model.layers.59.mlp.experts.63.gate_proj.weight": "model-00064-of-00101.safetensors", + "model.layers.59.mlp.experts.63.up_proj.weight": "model-00064-of-00101.safetensors", + "model.layers.59.mlp.experts.63.down_proj.weight": "model-00064-of-00101.safetensors", + "model.layers.59.mlp.experts.64.gate_proj.weight": "model-00064-of-00101.safetensors", + "model.layers.59.mlp.experts.64.up_proj.weight": "model-00064-of-00101.safetensors", + "model.layers.59.mlp.experts.64.down_proj.weight": "model-00064-of-00101.safetensors", + "model.layers.59.mlp.experts.65.gate_proj.weight": "model-00064-of-00101.safetensors", + "model.layers.59.mlp.experts.65.up_proj.weight": "model-00064-of-00101.safetensors", + "model.layers.59.mlp.experts.65.down_proj.weight": "model-00064-of-00101.safetensors", + "model.layers.59.mlp.experts.66.gate_proj.weight": "model-00064-of-00101.safetensors", + "model.layers.59.mlp.experts.66.up_proj.weight": "model-00064-of-00101.safetensors", + "model.layers.59.mlp.experts.66.down_proj.weight": "model-00064-of-00101.safetensors", + "model.layers.59.mlp.experts.67.gate_proj.weight": "model-00064-of-00101.safetensors", + "model.layers.59.mlp.experts.67.up_proj.weight": "model-00064-of-00101.safetensors", + "model.layers.59.mlp.experts.67.down_proj.weight": "model-00064-of-00101.safetensors", + "model.layers.59.mlp.experts.68.gate_proj.weight": "model-00064-of-00101.safetensors", + "model.layers.59.mlp.experts.68.up_proj.weight": "model-00064-of-00101.safetensors", + "model.layers.59.mlp.experts.68.down_proj.weight": "model-00064-of-00101.safetensors", + "model.layers.59.mlp.experts.69.gate_proj.weight": "model-00064-of-00101.safetensors", + "model.layers.59.mlp.experts.69.up_proj.weight": "model-00064-of-00101.safetensors", + "model.layers.59.mlp.experts.69.down_proj.weight": "model-00064-of-00101.safetensors", + "model.layers.59.mlp.experts.70.gate_proj.weight": "model-00064-of-00101.safetensors", + "model.layers.59.mlp.experts.70.up_proj.weight": "model-00064-of-00101.safetensors", + "model.layers.59.mlp.experts.70.down_proj.weight": "model-00064-of-00101.safetensors", + "model.layers.59.mlp.experts.71.gate_proj.weight": "model-00064-of-00101.safetensors", + "model.layers.59.mlp.experts.71.up_proj.weight": "model-00064-of-00101.safetensors", + "model.layers.59.mlp.experts.71.down_proj.weight": "model-00064-of-00101.safetensors", + "model.layers.59.mlp.experts.72.gate_proj.weight": "model-00064-of-00101.safetensors", + "model.layers.59.mlp.experts.72.up_proj.weight": "model-00064-of-00101.safetensors", + "model.layers.59.mlp.experts.72.down_proj.weight": "model-00064-of-00101.safetensors", + "model.layers.59.mlp.experts.73.gate_proj.weight": "model-00064-of-00101.safetensors", + "model.layers.59.mlp.experts.73.up_proj.weight": "model-00064-of-00101.safetensors", + "model.layers.59.mlp.experts.73.down_proj.weight": "model-00064-of-00101.safetensors", + "model.layers.59.mlp.experts.74.gate_proj.weight": "model-00064-of-00101.safetensors", + "model.layers.59.mlp.experts.74.up_proj.weight": "model-00064-of-00101.safetensors", + "model.layers.59.mlp.experts.74.down_proj.weight": "model-00064-of-00101.safetensors", + "model.layers.59.mlp.experts.75.gate_proj.weight": "model-00064-of-00101.safetensors", + "model.layers.59.mlp.experts.75.up_proj.weight": "model-00064-of-00101.safetensors", + "model.layers.59.mlp.experts.75.down_proj.weight": "model-00064-of-00101.safetensors", + "model.layers.59.mlp.experts.76.gate_proj.weight": "model-00064-of-00101.safetensors", + "model.layers.59.mlp.experts.76.up_proj.weight": "model-00064-of-00101.safetensors", + "model.layers.59.mlp.experts.76.down_proj.weight": "model-00064-of-00101.safetensors", + "model.layers.59.mlp.experts.77.gate_proj.weight": "model-00064-of-00101.safetensors", + "model.layers.59.mlp.experts.77.up_proj.weight": "model-00064-of-00101.safetensors", + "model.layers.59.mlp.experts.77.down_proj.weight": "model-00064-of-00101.safetensors", + "model.layers.59.mlp.experts.78.gate_proj.weight": "model-00064-of-00101.safetensors", + "model.layers.59.mlp.experts.78.up_proj.weight": "model-00064-of-00101.safetensors", + "model.layers.59.mlp.experts.78.down_proj.weight": "model-00064-of-00101.safetensors", + "model.layers.59.mlp.experts.79.gate_proj.weight": "model-00064-of-00101.safetensors", + "model.layers.59.mlp.experts.79.up_proj.weight": "model-00064-of-00101.safetensors", + "model.layers.59.mlp.experts.79.down_proj.weight": "model-00064-of-00101.safetensors", + "model.layers.59.mlp.experts.80.gate_proj.weight": "model-00064-of-00101.safetensors", + "model.layers.59.mlp.experts.80.up_proj.weight": "model-00064-of-00101.safetensors", + "model.layers.59.mlp.experts.80.down_proj.weight": "model-00064-of-00101.safetensors", + "model.layers.59.mlp.experts.81.gate_proj.weight": "model-00065-of-00101.safetensors", + "model.layers.59.mlp.experts.81.up_proj.weight": "model-00065-of-00101.safetensors", + "model.layers.59.mlp.experts.81.down_proj.weight": "model-00065-of-00101.safetensors", + "model.layers.59.mlp.experts.82.gate_proj.weight": "model-00065-of-00101.safetensors", + "model.layers.59.mlp.experts.82.up_proj.weight": "model-00065-of-00101.safetensors", + "model.layers.59.mlp.experts.82.down_proj.weight": "model-00065-of-00101.safetensors", + "model.layers.59.mlp.experts.83.gate_proj.weight": "model-00065-of-00101.safetensors", + "model.layers.59.mlp.experts.83.up_proj.weight": "model-00065-of-00101.safetensors", + "model.layers.59.mlp.experts.83.down_proj.weight": "model-00065-of-00101.safetensors", + "model.layers.59.mlp.experts.84.gate_proj.weight": "model-00065-of-00101.safetensors", + "model.layers.59.mlp.experts.84.up_proj.weight": "model-00065-of-00101.safetensors", + "model.layers.59.mlp.experts.84.down_proj.weight": "model-00065-of-00101.safetensors", + "model.layers.59.mlp.experts.85.gate_proj.weight": "model-00065-of-00101.safetensors", + "model.layers.59.mlp.experts.85.up_proj.weight": "model-00065-of-00101.safetensors", + "model.layers.59.mlp.experts.85.down_proj.weight": "model-00065-of-00101.safetensors", + "model.layers.59.mlp.experts.86.gate_proj.weight": "model-00065-of-00101.safetensors", + "model.layers.59.mlp.experts.86.up_proj.weight": "model-00065-of-00101.safetensors", + "model.layers.59.mlp.experts.86.down_proj.weight": "model-00065-of-00101.safetensors", + "model.layers.59.mlp.experts.87.gate_proj.weight": "model-00065-of-00101.safetensors", + "model.layers.59.mlp.experts.87.up_proj.weight": "model-00065-of-00101.safetensors", + "model.layers.59.mlp.experts.87.down_proj.weight": "model-00065-of-00101.safetensors", + "model.layers.59.mlp.experts.88.gate_proj.weight": "model-00065-of-00101.safetensors", + "model.layers.59.mlp.experts.88.up_proj.weight": "model-00065-of-00101.safetensors", + "model.layers.59.mlp.experts.88.down_proj.weight": "model-00065-of-00101.safetensors", + "model.layers.59.mlp.experts.89.gate_proj.weight": "model-00065-of-00101.safetensors", + "model.layers.59.mlp.experts.89.up_proj.weight": "model-00065-of-00101.safetensors", + "model.layers.59.mlp.experts.89.down_proj.weight": "model-00065-of-00101.safetensors", + "model.layers.59.mlp.experts.90.gate_proj.weight": "model-00065-of-00101.safetensors", + "model.layers.59.mlp.experts.90.up_proj.weight": "model-00065-of-00101.safetensors", + "model.layers.59.mlp.experts.90.down_proj.weight": "model-00065-of-00101.safetensors", + "model.layers.59.mlp.experts.91.gate_proj.weight": "model-00065-of-00101.safetensors", + "model.layers.59.mlp.experts.91.up_proj.weight": "model-00065-of-00101.safetensors", + "model.layers.59.mlp.experts.91.down_proj.weight": "model-00065-of-00101.safetensors", + "model.layers.59.mlp.experts.92.gate_proj.weight": "model-00065-of-00101.safetensors", + "model.layers.59.mlp.experts.92.up_proj.weight": "model-00065-of-00101.safetensors", + "model.layers.59.mlp.experts.92.down_proj.weight": "model-00065-of-00101.safetensors", + "model.layers.59.mlp.experts.93.gate_proj.weight": "model-00065-of-00101.safetensors", + "model.layers.59.mlp.experts.93.up_proj.weight": "model-00065-of-00101.safetensors", + "model.layers.59.mlp.experts.93.down_proj.weight": "model-00065-of-00101.safetensors", + "model.layers.59.mlp.experts.94.gate_proj.weight": "model-00065-of-00101.safetensors", + "model.layers.59.mlp.experts.94.up_proj.weight": "model-00065-of-00101.safetensors", + "model.layers.59.mlp.experts.94.down_proj.weight": "model-00065-of-00101.safetensors", + "model.layers.59.mlp.experts.95.gate_proj.weight": "model-00065-of-00101.safetensors", + "model.layers.59.mlp.experts.95.up_proj.weight": "model-00065-of-00101.safetensors", + "model.layers.59.mlp.experts.95.down_proj.weight": "model-00065-of-00101.safetensors", + "model.layers.59.mlp.experts.96.gate_proj.weight": "model-00065-of-00101.safetensors", + "model.layers.59.mlp.experts.96.up_proj.weight": "model-00065-of-00101.safetensors", + "model.layers.59.mlp.experts.96.down_proj.weight": "model-00065-of-00101.safetensors", + "model.layers.59.mlp.experts.97.gate_proj.weight": "model-00065-of-00101.safetensors", + "model.layers.59.mlp.experts.97.up_proj.weight": "model-00065-of-00101.safetensors", + "model.layers.59.mlp.experts.97.down_proj.weight": "model-00065-of-00101.safetensors", + "model.layers.59.mlp.experts.98.gate_proj.weight": "model-00065-of-00101.safetensors", + "model.layers.59.mlp.experts.98.up_proj.weight": "model-00065-of-00101.safetensors", + "model.layers.59.mlp.experts.98.down_proj.weight": "model-00065-of-00101.safetensors", + "model.layers.59.mlp.experts.99.gate_proj.weight": "model-00065-of-00101.safetensors", + "model.layers.59.mlp.experts.99.up_proj.weight": "model-00065-of-00101.safetensors", + "model.layers.59.mlp.experts.99.down_proj.weight": "model-00065-of-00101.safetensors", + "model.layers.59.mlp.experts.100.gate_proj.weight": "model-00065-of-00101.safetensors", + "model.layers.59.mlp.experts.100.up_proj.weight": "model-00065-of-00101.safetensors", + "model.layers.59.mlp.experts.100.down_proj.weight": "model-00065-of-00101.safetensors", + "model.layers.59.mlp.experts.101.gate_proj.weight": "model-00065-of-00101.safetensors", + "model.layers.59.mlp.experts.101.up_proj.weight": "model-00065-of-00101.safetensors", + "model.layers.59.mlp.experts.101.down_proj.weight": "model-00065-of-00101.safetensors", + "model.layers.59.mlp.experts.102.gate_proj.weight": "model-00065-of-00101.safetensors", + "model.layers.59.mlp.experts.102.up_proj.weight": "model-00065-of-00101.safetensors", + "model.layers.59.mlp.experts.102.down_proj.weight": "model-00065-of-00101.safetensors", + "model.layers.59.mlp.experts.103.gate_proj.weight": "model-00065-of-00101.safetensors", + "model.layers.59.mlp.experts.103.up_proj.weight": "model-00065-of-00101.safetensors", + "model.layers.59.mlp.experts.103.down_proj.weight": "model-00065-of-00101.safetensors", + "model.layers.59.mlp.experts.104.gate_proj.weight": "model-00065-of-00101.safetensors", + "model.layers.59.mlp.experts.104.up_proj.weight": "model-00065-of-00101.safetensors", + "model.layers.59.mlp.experts.104.down_proj.weight": "model-00065-of-00101.safetensors", + "model.layers.59.mlp.experts.105.gate_proj.weight": "model-00065-of-00101.safetensors", + "model.layers.59.mlp.experts.105.up_proj.weight": "model-00065-of-00101.safetensors", + "model.layers.59.mlp.experts.105.down_proj.weight": "model-00065-of-00101.safetensors", + "model.layers.59.mlp.experts.106.gate_proj.weight": "model-00065-of-00101.safetensors", + "model.layers.59.mlp.experts.106.up_proj.weight": "model-00065-of-00101.safetensors", + "model.layers.59.mlp.experts.106.down_proj.weight": "model-00065-of-00101.safetensors", + "model.layers.59.mlp.experts.107.gate_proj.weight": "model-00065-of-00101.safetensors", + "model.layers.59.mlp.experts.107.up_proj.weight": "model-00065-of-00101.safetensors", + "model.layers.59.mlp.experts.107.down_proj.weight": "model-00065-of-00101.safetensors", + "model.layers.59.mlp.experts.108.gate_proj.weight": "model-00065-of-00101.safetensors", + "model.layers.59.mlp.experts.108.up_proj.weight": "model-00065-of-00101.safetensors", + "model.layers.59.mlp.experts.108.down_proj.weight": "model-00065-of-00101.safetensors", + "model.layers.59.mlp.experts.109.gate_proj.weight": "model-00065-of-00101.safetensors", + "model.layers.59.mlp.experts.109.up_proj.weight": "model-00065-of-00101.safetensors", + "model.layers.59.mlp.experts.109.down_proj.weight": "model-00065-of-00101.safetensors", + "model.layers.59.mlp.experts.110.gate_proj.weight": "model-00065-of-00101.safetensors", + "model.layers.59.mlp.experts.110.up_proj.weight": "model-00065-of-00101.safetensors", + "model.layers.59.mlp.experts.110.down_proj.weight": "model-00065-of-00101.safetensors", + "model.layers.59.mlp.experts.111.gate_proj.weight": "model-00065-of-00101.safetensors", + "model.layers.59.mlp.experts.111.up_proj.weight": "model-00065-of-00101.safetensors", + "model.layers.59.mlp.experts.111.down_proj.weight": "model-00065-of-00101.safetensors", + "model.layers.59.mlp.experts.112.gate_proj.weight": "model-00065-of-00101.safetensors", + "model.layers.59.mlp.experts.112.up_proj.weight": "model-00065-of-00101.safetensors", + "model.layers.59.mlp.experts.112.down_proj.weight": "model-00065-of-00101.safetensors", + "model.layers.59.mlp.experts.113.gate_proj.weight": "model-00065-of-00101.safetensors", + "model.layers.59.mlp.experts.113.up_proj.weight": "model-00065-of-00101.safetensors", + "model.layers.59.mlp.experts.113.down_proj.weight": "model-00065-of-00101.safetensors", + "model.layers.59.mlp.experts.114.gate_proj.weight": "model-00065-of-00101.safetensors", + "model.layers.59.mlp.experts.114.up_proj.weight": "model-00065-of-00101.safetensors", + "model.layers.59.mlp.experts.114.down_proj.weight": "model-00065-of-00101.safetensors", + "model.layers.59.mlp.experts.115.gate_proj.weight": "model-00065-of-00101.safetensors", + "model.layers.59.mlp.experts.115.up_proj.weight": "model-00065-of-00101.safetensors", + "model.layers.59.mlp.experts.115.down_proj.weight": "model-00065-of-00101.safetensors", + "model.layers.59.mlp.experts.116.gate_proj.weight": "model-00065-of-00101.safetensors", + "model.layers.59.mlp.experts.116.up_proj.weight": "model-00065-of-00101.safetensors", + "model.layers.59.mlp.experts.116.down_proj.weight": "model-00065-of-00101.safetensors", + "model.layers.59.mlp.experts.117.gate_proj.weight": "model-00065-of-00101.safetensors", + "model.layers.59.mlp.experts.117.up_proj.weight": "model-00065-of-00101.safetensors", + "model.layers.59.mlp.experts.117.down_proj.weight": "model-00065-of-00101.safetensors", + "model.layers.59.mlp.experts.118.gate_proj.weight": "model-00065-of-00101.safetensors", + "model.layers.59.mlp.experts.118.up_proj.weight": "model-00065-of-00101.safetensors", + "model.layers.59.mlp.experts.118.down_proj.weight": "model-00065-of-00101.safetensors", + "model.layers.59.mlp.experts.119.gate_proj.weight": "model-00065-of-00101.safetensors", + "model.layers.59.mlp.experts.119.up_proj.weight": "model-00065-of-00101.safetensors", + "model.layers.59.mlp.experts.119.down_proj.weight": "model-00065-of-00101.safetensors", + "model.layers.59.mlp.gate.weight": "model-00065-of-00101.safetensors", + "model.layers.59.mlp.gate.e_score_correction_bias": "model-00065-of-00101.safetensors", + "model.layers.59.mlp.shared_experts.gate_proj.weight": "model-00065-of-00101.safetensors", + "model.layers.59.mlp.shared_experts.up_proj.weight": "model-00065-of-00101.safetensors", + "model.layers.59.mlp.shared_experts.down_proj.weight": "model-00065-of-00101.safetensors", + "model.layers.59.input_layernorm.weight": "model-00065-of-00101.safetensors", + "model.layers.59.post_attention_layernorm.weight": "model-00065-of-00101.safetensors", + "model.layers.60.self_attn.q_proj.weight": "model-00065-of-00101.safetensors", + "model.layers.60.self_attn.q_proj.bias": "model-00065-of-00101.safetensors", + "model.layers.60.self_attn.k_proj.weight": "model-00065-of-00101.safetensors", + "model.layers.60.self_attn.k_proj.bias": "model-00065-of-00101.safetensors", + "model.layers.60.self_attn.v_proj.weight": "model-00065-of-00101.safetensors", + "model.layers.60.self_attn.v_proj.bias": "model-00065-of-00101.safetensors", + "model.layers.60.self_attn.o_proj.weight": "model-00065-of-00101.safetensors", + "model.layers.60.self_attn.q_norm.weight": "model-00065-of-00101.safetensors", + "model.layers.60.self_attn.k_norm.weight": "model-00065-of-00101.safetensors", + "model.layers.60.mlp.experts.0.gate_proj.weight": "model-00065-of-00101.safetensors", + "model.layers.60.mlp.experts.0.up_proj.weight": "model-00065-of-00101.safetensors", + "model.layers.60.mlp.experts.0.down_proj.weight": "model-00065-of-00101.safetensors", + "model.layers.60.mlp.experts.1.gate_proj.weight": "model-00065-of-00101.safetensors", + "model.layers.60.mlp.experts.1.up_proj.weight": "model-00065-of-00101.safetensors", + "model.layers.60.mlp.experts.1.down_proj.weight": "model-00065-of-00101.safetensors", + "model.layers.60.mlp.experts.2.gate_proj.weight": "model-00065-of-00101.safetensors", + "model.layers.60.mlp.experts.2.up_proj.weight": "model-00065-of-00101.safetensors", + "model.layers.60.mlp.experts.2.down_proj.weight": "model-00065-of-00101.safetensors", + "model.layers.60.mlp.experts.3.gate_proj.weight": "model-00065-of-00101.safetensors", + "model.layers.60.mlp.experts.3.up_proj.weight": "model-00065-of-00101.safetensors", + "model.layers.60.mlp.experts.3.down_proj.weight": "model-00065-of-00101.safetensors", + "model.layers.60.mlp.experts.4.gate_proj.weight": "model-00065-of-00101.safetensors", + "model.layers.60.mlp.experts.4.up_proj.weight": "model-00065-of-00101.safetensors", + "model.layers.60.mlp.experts.4.down_proj.weight": "model-00065-of-00101.safetensors", + "model.layers.60.mlp.experts.5.gate_proj.weight": "model-00065-of-00101.safetensors", + "model.layers.60.mlp.experts.5.up_proj.weight": "model-00065-of-00101.safetensors", + "model.layers.60.mlp.experts.5.down_proj.weight": "model-00065-of-00101.safetensors", + "model.layers.60.mlp.experts.6.gate_proj.weight": "model-00065-of-00101.safetensors", + "model.layers.60.mlp.experts.6.up_proj.weight": "model-00065-of-00101.safetensors", + "model.layers.60.mlp.experts.6.down_proj.weight": "model-00065-of-00101.safetensors", + "model.layers.60.mlp.experts.7.gate_proj.weight": "model-00065-of-00101.safetensors", + "model.layers.60.mlp.experts.7.up_proj.weight": "model-00065-of-00101.safetensors", + "model.layers.60.mlp.experts.7.down_proj.weight": "model-00065-of-00101.safetensors", + "model.layers.60.mlp.experts.8.gate_proj.weight": "model-00065-of-00101.safetensors", + "model.layers.60.mlp.experts.8.up_proj.weight": "model-00065-of-00101.safetensors", + "model.layers.60.mlp.experts.8.down_proj.weight": "model-00065-of-00101.safetensors", + "model.layers.60.mlp.experts.9.gate_proj.weight": "model-00065-of-00101.safetensors", + "model.layers.60.mlp.experts.9.up_proj.weight": "model-00065-of-00101.safetensors", + "model.layers.60.mlp.experts.9.down_proj.weight": "model-00065-of-00101.safetensors", + "model.layers.60.mlp.experts.10.gate_proj.weight": "model-00065-of-00101.safetensors", + "model.layers.60.mlp.experts.10.up_proj.weight": "model-00065-of-00101.safetensors", + "model.layers.60.mlp.experts.10.down_proj.weight": "model-00065-of-00101.safetensors", + "model.layers.60.mlp.experts.11.gate_proj.weight": "model-00065-of-00101.safetensors", + "model.layers.60.mlp.experts.11.up_proj.weight": "model-00065-of-00101.safetensors", + "model.layers.60.mlp.experts.11.down_proj.weight": "model-00065-of-00101.safetensors", + "model.layers.60.mlp.experts.12.gate_proj.weight": "model-00065-of-00101.safetensors", + "model.layers.60.mlp.experts.12.up_proj.weight": "model-00065-of-00101.safetensors", + "model.layers.60.mlp.experts.12.down_proj.weight": "model-00065-of-00101.safetensors", + "model.layers.60.mlp.experts.13.gate_proj.weight": "model-00065-of-00101.safetensors", + "model.layers.60.mlp.experts.13.up_proj.weight": "model-00065-of-00101.safetensors", + "model.layers.60.mlp.experts.13.down_proj.weight": "model-00065-of-00101.safetensors", + "model.layers.60.mlp.experts.14.gate_proj.weight": "model-00065-of-00101.safetensors", + "model.layers.60.mlp.experts.14.up_proj.weight": "model-00065-of-00101.safetensors", + "model.layers.60.mlp.experts.14.down_proj.weight": "model-00065-of-00101.safetensors", + "model.layers.60.mlp.experts.15.gate_proj.weight": "model-00065-of-00101.safetensors", + "model.layers.60.mlp.experts.15.up_proj.weight": "model-00065-of-00101.safetensors", + "model.layers.60.mlp.experts.15.down_proj.weight": "model-00065-of-00101.safetensors", + "model.layers.60.mlp.experts.16.gate_proj.weight": "model-00065-of-00101.safetensors", + "model.layers.60.mlp.experts.16.up_proj.weight": "model-00065-of-00101.safetensors", + "model.layers.60.mlp.experts.16.down_proj.weight": "model-00065-of-00101.safetensors", + "model.layers.60.mlp.experts.17.gate_proj.weight": "model-00065-of-00101.safetensors", + "model.layers.60.mlp.experts.17.up_proj.weight": "model-00065-of-00101.safetensors", + "model.layers.60.mlp.experts.17.down_proj.weight": "model-00065-of-00101.safetensors", + "model.layers.60.mlp.experts.18.gate_proj.weight": "model-00065-of-00101.safetensors", + "model.layers.60.mlp.experts.18.up_proj.weight": "model-00065-of-00101.safetensors", + "model.layers.60.mlp.experts.18.down_proj.weight": "model-00065-of-00101.safetensors", + "model.layers.60.mlp.experts.19.gate_proj.weight": "model-00065-of-00101.safetensors", + "model.layers.60.mlp.experts.19.up_proj.weight": "model-00065-of-00101.safetensors", + "model.layers.60.mlp.experts.19.down_proj.weight": "model-00065-of-00101.safetensors", + "model.layers.60.mlp.experts.20.gate_proj.weight": "model-00065-of-00101.safetensors", + "model.layers.60.mlp.experts.20.up_proj.weight": "model-00065-of-00101.safetensors", + "model.layers.60.mlp.experts.20.down_proj.weight": "model-00065-of-00101.safetensors", + "model.layers.60.mlp.experts.21.gate_proj.weight": "model-00065-of-00101.safetensors", + "model.layers.60.mlp.experts.21.up_proj.weight": "model-00065-of-00101.safetensors", + "model.layers.60.mlp.experts.21.down_proj.weight": "model-00065-of-00101.safetensors", + "model.layers.60.mlp.experts.22.gate_proj.weight": "model-00065-of-00101.safetensors", + "model.layers.60.mlp.experts.22.up_proj.weight": "model-00065-of-00101.safetensors", + "model.layers.60.mlp.experts.22.down_proj.weight": "model-00065-of-00101.safetensors", + "model.layers.60.mlp.experts.23.gate_proj.weight": "model-00065-of-00101.safetensors", + "model.layers.60.mlp.experts.23.up_proj.weight": "model-00065-of-00101.safetensors", + "model.layers.60.mlp.experts.23.down_proj.weight": "model-00065-of-00101.safetensors", + "model.layers.60.mlp.experts.24.gate_proj.weight": "model-00065-of-00101.safetensors", + "model.layers.60.mlp.experts.24.up_proj.weight": "model-00065-of-00101.safetensors", + "model.layers.60.mlp.experts.24.down_proj.weight": "model-00065-of-00101.safetensors", + "model.layers.60.mlp.experts.25.gate_proj.weight": "model-00065-of-00101.safetensors", + "model.layers.60.mlp.experts.25.up_proj.weight": "model-00065-of-00101.safetensors", + "model.layers.60.mlp.experts.25.down_proj.weight": "model-00065-of-00101.safetensors", + "model.layers.60.mlp.experts.26.gate_proj.weight": "model-00065-of-00101.safetensors", + "model.layers.60.mlp.experts.26.up_proj.weight": "model-00065-of-00101.safetensors", + "model.layers.60.mlp.experts.26.down_proj.weight": "model-00065-of-00101.safetensors", + "model.layers.60.mlp.experts.27.gate_proj.weight": "model-00065-of-00101.safetensors", + "model.layers.60.mlp.experts.27.up_proj.weight": "model-00065-of-00101.safetensors", + "model.layers.60.mlp.experts.27.down_proj.weight": "model-00065-of-00101.safetensors", + "model.layers.60.mlp.experts.28.gate_proj.weight": "model-00065-of-00101.safetensors", + "model.layers.60.mlp.experts.28.up_proj.weight": "model-00065-of-00101.safetensors", + "model.layers.60.mlp.experts.28.down_proj.weight": "model-00065-of-00101.safetensors", + "model.layers.60.mlp.experts.29.gate_proj.weight": "model-00065-of-00101.safetensors", + "model.layers.60.mlp.experts.29.up_proj.weight": "model-00065-of-00101.safetensors", + "model.layers.60.mlp.experts.29.down_proj.weight": "model-00065-of-00101.safetensors", + "model.layers.60.mlp.experts.30.gate_proj.weight": "model-00065-of-00101.safetensors", + "model.layers.60.mlp.experts.30.up_proj.weight": "model-00065-of-00101.safetensors", + "model.layers.60.mlp.experts.30.down_proj.weight": "model-00065-of-00101.safetensors", + "model.layers.60.mlp.experts.31.gate_proj.weight": "model-00065-of-00101.safetensors", + "model.layers.60.mlp.experts.31.up_proj.weight": "model-00065-of-00101.safetensors", + "model.layers.60.mlp.experts.31.down_proj.weight": "model-00065-of-00101.safetensors", + "model.layers.60.mlp.experts.32.gate_proj.weight": "model-00065-of-00101.safetensors", + "model.layers.60.mlp.experts.32.up_proj.weight": "model-00065-of-00101.safetensors", + "model.layers.60.mlp.experts.32.down_proj.weight": "model-00065-of-00101.safetensors", + "model.layers.60.mlp.experts.33.gate_proj.weight": "model-00065-of-00101.safetensors", + "model.layers.60.mlp.experts.33.up_proj.weight": "model-00065-of-00101.safetensors", + "model.layers.60.mlp.experts.33.down_proj.weight": "model-00065-of-00101.safetensors", + "model.layers.60.mlp.experts.34.gate_proj.weight": "model-00065-of-00101.safetensors", + "model.layers.60.mlp.experts.34.up_proj.weight": "model-00065-of-00101.safetensors", + "model.layers.60.mlp.experts.34.down_proj.weight": "model-00065-of-00101.safetensors", + "model.layers.60.mlp.experts.35.gate_proj.weight": "model-00065-of-00101.safetensors", + "model.layers.60.mlp.experts.35.up_proj.weight": "model-00065-of-00101.safetensors", + "model.layers.60.mlp.experts.35.down_proj.weight": "model-00065-of-00101.safetensors", + "model.layers.60.mlp.experts.36.gate_proj.weight": "model-00065-of-00101.safetensors", + "model.layers.60.mlp.experts.36.up_proj.weight": "model-00065-of-00101.safetensors", + "model.layers.60.mlp.experts.36.down_proj.weight": "model-00065-of-00101.safetensors", + "model.layers.60.mlp.experts.37.gate_proj.weight": "model-00065-of-00101.safetensors", + "model.layers.60.mlp.experts.37.up_proj.weight": "model-00065-of-00101.safetensors", + "model.layers.60.mlp.experts.37.down_proj.weight": "model-00065-of-00101.safetensors", + "model.layers.60.mlp.experts.38.gate_proj.weight": "model-00065-of-00101.safetensors", + "model.layers.60.mlp.experts.38.up_proj.weight": "model-00065-of-00101.safetensors", + "model.layers.60.mlp.experts.38.down_proj.weight": "model-00065-of-00101.safetensors", + "model.layers.60.mlp.experts.39.gate_proj.weight": "model-00065-of-00101.safetensors", + "model.layers.60.mlp.experts.39.up_proj.weight": "model-00065-of-00101.safetensors", + "model.layers.60.mlp.experts.39.down_proj.weight": "model-00065-of-00101.safetensors", + "model.layers.60.mlp.experts.40.gate_proj.weight": "model-00065-of-00101.safetensors", + "model.layers.60.mlp.experts.40.up_proj.weight": "model-00065-of-00101.safetensors", + "model.layers.60.mlp.experts.40.down_proj.weight": "model-00065-of-00101.safetensors", + "model.layers.60.mlp.experts.41.gate_proj.weight": "model-00065-of-00101.safetensors", + "model.layers.60.mlp.experts.41.up_proj.weight": "model-00065-of-00101.safetensors", + "model.layers.60.mlp.experts.41.down_proj.weight": "model-00065-of-00101.safetensors", + "model.layers.60.mlp.experts.42.gate_proj.weight": "model-00065-of-00101.safetensors", + "model.layers.60.mlp.experts.42.up_proj.weight": "model-00065-of-00101.safetensors", + "model.layers.60.mlp.experts.42.down_proj.weight": "model-00065-of-00101.safetensors", + "model.layers.60.mlp.experts.43.gate_proj.weight": "model-00065-of-00101.safetensors", + "model.layers.60.mlp.experts.43.up_proj.weight": "model-00065-of-00101.safetensors", + "model.layers.60.mlp.experts.43.down_proj.weight": "model-00065-of-00101.safetensors", + "model.layers.60.mlp.experts.44.gate_proj.weight": "model-00065-of-00101.safetensors", + "model.layers.60.mlp.experts.44.up_proj.weight": "model-00065-of-00101.safetensors", + "model.layers.60.mlp.experts.44.down_proj.weight": "model-00065-of-00101.safetensors", + "model.layers.60.mlp.experts.45.gate_proj.weight": "model-00065-of-00101.safetensors", + "model.layers.60.mlp.experts.45.up_proj.weight": "model-00065-of-00101.safetensors", + "model.layers.60.mlp.experts.45.down_proj.weight": "model-00065-of-00101.safetensors", + "model.layers.60.mlp.experts.46.gate_proj.weight": "model-00065-of-00101.safetensors", + "model.layers.60.mlp.experts.46.up_proj.weight": "model-00065-of-00101.safetensors", + "model.layers.60.mlp.experts.46.down_proj.weight": "model-00065-of-00101.safetensors", + "model.layers.60.mlp.experts.47.gate_proj.weight": "model-00065-of-00101.safetensors", + "model.layers.60.mlp.experts.47.up_proj.weight": "model-00065-of-00101.safetensors", + "model.layers.60.mlp.experts.47.down_proj.weight": "model-00065-of-00101.safetensors", + "model.layers.60.mlp.experts.48.gate_proj.weight": "model-00065-of-00101.safetensors", + "model.layers.60.mlp.experts.48.up_proj.weight": "model-00065-of-00101.safetensors", + "model.layers.60.mlp.experts.48.down_proj.weight": "model-00065-of-00101.safetensors", + "model.layers.60.mlp.experts.49.gate_proj.weight": "model-00065-of-00101.safetensors", + "model.layers.60.mlp.experts.49.up_proj.weight": "model-00065-of-00101.safetensors", + "model.layers.60.mlp.experts.49.down_proj.weight": "model-00065-of-00101.safetensors", + "model.layers.60.mlp.experts.50.gate_proj.weight": "model-00065-of-00101.safetensors", + "model.layers.60.mlp.experts.50.up_proj.weight": "model-00065-of-00101.safetensors", + "model.layers.60.mlp.experts.50.down_proj.weight": "model-00065-of-00101.safetensors", + "model.layers.60.mlp.experts.51.gate_proj.weight": "model-00065-of-00101.safetensors", + "model.layers.60.mlp.experts.51.up_proj.weight": "model-00065-of-00101.safetensors", + "model.layers.60.mlp.experts.51.down_proj.weight": "model-00065-of-00101.safetensors", + "model.layers.60.mlp.experts.52.gate_proj.weight": "model-00065-of-00101.safetensors", + "model.layers.60.mlp.experts.52.up_proj.weight": "model-00065-of-00101.safetensors", + "model.layers.60.mlp.experts.52.down_proj.weight": "model-00065-of-00101.safetensors", + "model.layers.60.mlp.experts.53.gate_proj.weight": "model-00065-of-00101.safetensors", + "model.layers.60.mlp.experts.53.up_proj.weight": "model-00065-of-00101.safetensors", + "model.layers.60.mlp.experts.53.down_proj.weight": "model-00065-of-00101.safetensors", + "model.layers.60.mlp.experts.54.gate_proj.weight": "model-00065-of-00101.safetensors", + "model.layers.60.mlp.experts.54.up_proj.weight": "model-00065-of-00101.safetensors", + "model.layers.60.mlp.experts.54.down_proj.weight": "model-00065-of-00101.safetensors", + "model.layers.60.mlp.experts.55.gate_proj.weight": "model-00065-of-00101.safetensors", + "model.layers.60.mlp.experts.55.up_proj.weight": "model-00065-of-00101.safetensors", + "model.layers.60.mlp.experts.55.down_proj.weight": "model-00065-of-00101.safetensors", + "model.layers.60.mlp.experts.56.gate_proj.weight": "model-00065-of-00101.safetensors", + "model.layers.60.mlp.experts.56.up_proj.weight": "model-00065-of-00101.safetensors", + "model.layers.60.mlp.experts.56.down_proj.weight": "model-00065-of-00101.safetensors", + "model.layers.60.mlp.experts.57.gate_proj.weight": "model-00065-of-00101.safetensors", + "model.layers.60.mlp.experts.57.up_proj.weight": "model-00065-of-00101.safetensors", + "model.layers.60.mlp.experts.57.down_proj.weight": "model-00065-of-00101.safetensors", + "model.layers.60.mlp.experts.58.gate_proj.weight": "model-00065-of-00101.safetensors", + "model.layers.60.mlp.experts.58.up_proj.weight": "model-00065-of-00101.safetensors", + "model.layers.60.mlp.experts.58.down_proj.weight": "model-00065-of-00101.safetensors", + "model.layers.60.mlp.experts.59.gate_proj.weight": "model-00065-of-00101.safetensors", + "model.layers.60.mlp.experts.59.up_proj.weight": "model-00065-of-00101.safetensors", + "model.layers.60.mlp.experts.59.down_proj.weight": "model-00065-of-00101.safetensors", + "model.layers.60.mlp.experts.60.gate_proj.weight": "model-00065-of-00101.safetensors", + "model.layers.60.mlp.experts.60.up_proj.weight": "model-00065-of-00101.safetensors", + "model.layers.60.mlp.experts.60.down_proj.weight": "model-00065-of-00101.safetensors", + "model.layers.60.mlp.experts.61.gate_proj.weight": "model-00065-of-00101.safetensors", + "model.layers.60.mlp.experts.61.up_proj.weight": "model-00065-of-00101.safetensors", + "model.layers.60.mlp.experts.61.down_proj.weight": "model-00065-of-00101.safetensors", + "model.layers.60.mlp.experts.62.gate_proj.weight": "model-00065-of-00101.safetensors", + "model.layers.60.mlp.experts.62.up_proj.weight": "model-00065-of-00101.safetensors", + "model.layers.60.mlp.experts.62.down_proj.weight": "model-00065-of-00101.safetensors", + "model.layers.60.mlp.experts.63.gate_proj.weight": "model-00065-of-00101.safetensors", + "model.layers.60.mlp.experts.63.up_proj.weight": "model-00065-of-00101.safetensors", + "model.layers.60.mlp.experts.63.down_proj.weight": "model-00065-of-00101.safetensors", + "model.layers.60.mlp.experts.64.gate_proj.weight": "model-00065-of-00101.safetensors", + "model.layers.60.mlp.experts.64.up_proj.weight": "model-00065-of-00101.safetensors", + "model.layers.60.mlp.experts.64.down_proj.weight": "model-00065-of-00101.safetensors", + "model.layers.60.mlp.experts.65.gate_proj.weight": "model-00065-of-00101.safetensors", + "model.layers.60.mlp.experts.65.up_proj.weight": "model-00065-of-00101.safetensors", + "model.layers.60.mlp.experts.65.down_proj.weight": "model-00065-of-00101.safetensors", + "model.layers.60.mlp.experts.66.gate_proj.weight": "model-00065-of-00101.safetensors", + "model.layers.60.mlp.experts.66.up_proj.weight": "model-00065-of-00101.safetensors", + "model.layers.60.mlp.experts.66.down_proj.weight": "model-00065-of-00101.safetensors", + "model.layers.60.mlp.experts.67.gate_proj.weight": "model-00065-of-00101.safetensors", + "model.layers.60.mlp.experts.67.up_proj.weight": "model-00065-of-00101.safetensors", + "model.layers.60.mlp.experts.67.down_proj.weight": "model-00066-of-00101.safetensors", + "model.layers.60.mlp.experts.68.gate_proj.weight": "model-00066-of-00101.safetensors", + "model.layers.60.mlp.experts.68.up_proj.weight": "model-00066-of-00101.safetensors", + "model.layers.60.mlp.experts.68.down_proj.weight": "model-00066-of-00101.safetensors", + "model.layers.60.mlp.experts.69.gate_proj.weight": "model-00066-of-00101.safetensors", + "model.layers.60.mlp.experts.69.up_proj.weight": "model-00066-of-00101.safetensors", + "model.layers.60.mlp.experts.69.down_proj.weight": "model-00066-of-00101.safetensors", + "model.layers.60.mlp.experts.70.gate_proj.weight": "model-00066-of-00101.safetensors", + "model.layers.60.mlp.experts.70.up_proj.weight": "model-00066-of-00101.safetensors", + "model.layers.60.mlp.experts.70.down_proj.weight": "model-00066-of-00101.safetensors", + "model.layers.60.mlp.experts.71.gate_proj.weight": "model-00066-of-00101.safetensors", + "model.layers.60.mlp.experts.71.up_proj.weight": "model-00066-of-00101.safetensors", + "model.layers.60.mlp.experts.71.down_proj.weight": "model-00066-of-00101.safetensors", + "model.layers.60.mlp.experts.72.gate_proj.weight": "model-00066-of-00101.safetensors", + "model.layers.60.mlp.experts.72.up_proj.weight": "model-00066-of-00101.safetensors", + "model.layers.60.mlp.experts.72.down_proj.weight": "model-00066-of-00101.safetensors", + "model.layers.60.mlp.experts.73.gate_proj.weight": "model-00066-of-00101.safetensors", + "model.layers.60.mlp.experts.73.up_proj.weight": "model-00066-of-00101.safetensors", + "model.layers.60.mlp.experts.73.down_proj.weight": "model-00066-of-00101.safetensors", + "model.layers.60.mlp.experts.74.gate_proj.weight": "model-00066-of-00101.safetensors", + "model.layers.60.mlp.experts.74.up_proj.weight": "model-00066-of-00101.safetensors", + "model.layers.60.mlp.experts.74.down_proj.weight": "model-00066-of-00101.safetensors", + "model.layers.60.mlp.experts.75.gate_proj.weight": "model-00066-of-00101.safetensors", + "model.layers.60.mlp.experts.75.up_proj.weight": "model-00066-of-00101.safetensors", + "model.layers.60.mlp.experts.75.down_proj.weight": "model-00066-of-00101.safetensors", + "model.layers.60.mlp.experts.76.gate_proj.weight": "model-00066-of-00101.safetensors", + "model.layers.60.mlp.experts.76.up_proj.weight": "model-00066-of-00101.safetensors", + "model.layers.60.mlp.experts.76.down_proj.weight": "model-00066-of-00101.safetensors", + "model.layers.60.mlp.experts.77.gate_proj.weight": "model-00066-of-00101.safetensors", + "model.layers.60.mlp.experts.77.up_proj.weight": "model-00066-of-00101.safetensors", + "model.layers.60.mlp.experts.77.down_proj.weight": "model-00066-of-00101.safetensors", + "model.layers.60.mlp.experts.78.gate_proj.weight": "model-00066-of-00101.safetensors", + "model.layers.60.mlp.experts.78.up_proj.weight": "model-00066-of-00101.safetensors", + "model.layers.60.mlp.experts.78.down_proj.weight": "model-00066-of-00101.safetensors", + "model.layers.60.mlp.experts.79.gate_proj.weight": "model-00066-of-00101.safetensors", + "model.layers.60.mlp.experts.79.up_proj.weight": "model-00066-of-00101.safetensors", + "model.layers.60.mlp.experts.79.down_proj.weight": "model-00066-of-00101.safetensors", + "model.layers.60.mlp.experts.80.gate_proj.weight": "model-00066-of-00101.safetensors", + "model.layers.60.mlp.experts.80.up_proj.weight": "model-00066-of-00101.safetensors", + "model.layers.60.mlp.experts.80.down_proj.weight": "model-00066-of-00101.safetensors", + "model.layers.60.mlp.experts.81.gate_proj.weight": "model-00066-of-00101.safetensors", + "model.layers.60.mlp.experts.81.up_proj.weight": "model-00066-of-00101.safetensors", + "model.layers.60.mlp.experts.81.down_proj.weight": "model-00066-of-00101.safetensors", + "model.layers.60.mlp.experts.82.gate_proj.weight": "model-00066-of-00101.safetensors", + "model.layers.60.mlp.experts.82.up_proj.weight": "model-00066-of-00101.safetensors", + "model.layers.60.mlp.experts.82.down_proj.weight": "model-00066-of-00101.safetensors", + "model.layers.60.mlp.experts.83.gate_proj.weight": "model-00066-of-00101.safetensors", + "model.layers.60.mlp.experts.83.up_proj.weight": "model-00066-of-00101.safetensors", + "model.layers.60.mlp.experts.83.down_proj.weight": "model-00066-of-00101.safetensors", + "model.layers.60.mlp.experts.84.gate_proj.weight": "model-00066-of-00101.safetensors", + "model.layers.60.mlp.experts.84.up_proj.weight": "model-00066-of-00101.safetensors", + "model.layers.60.mlp.experts.84.down_proj.weight": "model-00066-of-00101.safetensors", + "model.layers.60.mlp.experts.85.gate_proj.weight": "model-00066-of-00101.safetensors", + "model.layers.60.mlp.experts.85.up_proj.weight": "model-00066-of-00101.safetensors", + "model.layers.60.mlp.experts.85.down_proj.weight": "model-00066-of-00101.safetensors", + "model.layers.60.mlp.experts.86.gate_proj.weight": "model-00066-of-00101.safetensors", + "model.layers.60.mlp.experts.86.up_proj.weight": "model-00066-of-00101.safetensors", + "model.layers.60.mlp.experts.86.down_proj.weight": "model-00066-of-00101.safetensors", + "model.layers.60.mlp.experts.87.gate_proj.weight": "model-00066-of-00101.safetensors", + "model.layers.60.mlp.experts.87.up_proj.weight": "model-00066-of-00101.safetensors", + "model.layers.60.mlp.experts.87.down_proj.weight": "model-00066-of-00101.safetensors", + "model.layers.60.mlp.experts.88.gate_proj.weight": "model-00066-of-00101.safetensors", + "model.layers.60.mlp.experts.88.up_proj.weight": "model-00066-of-00101.safetensors", + "model.layers.60.mlp.experts.88.down_proj.weight": "model-00066-of-00101.safetensors", + "model.layers.60.mlp.experts.89.gate_proj.weight": "model-00066-of-00101.safetensors", + "model.layers.60.mlp.experts.89.up_proj.weight": "model-00066-of-00101.safetensors", + "model.layers.60.mlp.experts.89.down_proj.weight": "model-00066-of-00101.safetensors", + "model.layers.60.mlp.experts.90.gate_proj.weight": "model-00066-of-00101.safetensors", + "model.layers.60.mlp.experts.90.up_proj.weight": "model-00066-of-00101.safetensors", + "model.layers.60.mlp.experts.90.down_proj.weight": "model-00066-of-00101.safetensors", + "model.layers.60.mlp.experts.91.gate_proj.weight": "model-00066-of-00101.safetensors", + "model.layers.60.mlp.experts.91.up_proj.weight": "model-00066-of-00101.safetensors", + "model.layers.60.mlp.experts.91.down_proj.weight": "model-00066-of-00101.safetensors", + "model.layers.60.mlp.experts.92.gate_proj.weight": "model-00066-of-00101.safetensors", + "model.layers.60.mlp.experts.92.up_proj.weight": "model-00066-of-00101.safetensors", + "model.layers.60.mlp.experts.92.down_proj.weight": "model-00066-of-00101.safetensors", + "model.layers.60.mlp.experts.93.gate_proj.weight": "model-00066-of-00101.safetensors", + "model.layers.60.mlp.experts.93.up_proj.weight": "model-00066-of-00101.safetensors", + "model.layers.60.mlp.experts.93.down_proj.weight": "model-00066-of-00101.safetensors", + "model.layers.60.mlp.experts.94.gate_proj.weight": "model-00066-of-00101.safetensors", + "model.layers.60.mlp.experts.94.up_proj.weight": "model-00066-of-00101.safetensors", + "model.layers.60.mlp.experts.94.down_proj.weight": "model-00066-of-00101.safetensors", + "model.layers.60.mlp.experts.95.gate_proj.weight": "model-00066-of-00101.safetensors", + "model.layers.60.mlp.experts.95.up_proj.weight": "model-00066-of-00101.safetensors", + "model.layers.60.mlp.experts.95.down_proj.weight": "model-00066-of-00101.safetensors", + "model.layers.60.mlp.experts.96.gate_proj.weight": "model-00066-of-00101.safetensors", + "model.layers.60.mlp.experts.96.up_proj.weight": "model-00066-of-00101.safetensors", + "model.layers.60.mlp.experts.96.down_proj.weight": "model-00066-of-00101.safetensors", + "model.layers.60.mlp.experts.97.gate_proj.weight": "model-00066-of-00101.safetensors", + "model.layers.60.mlp.experts.97.up_proj.weight": "model-00066-of-00101.safetensors", + "model.layers.60.mlp.experts.97.down_proj.weight": "model-00066-of-00101.safetensors", + "model.layers.60.mlp.experts.98.gate_proj.weight": "model-00066-of-00101.safetensors", + "model.layers.60.mlp.experts.98.up_proj.weight": "model-00066-of-00101.safetensors", + "model.layers.60.mlp.experts.98.down_proj.weight": "model-00066-of-00101.safetensors", + "model.layers.60.mlp.experts.99.gate_proj.weight": "model-00066-of-00101.safetensors", + "model.layers.60.mlp.experts.99.up_proj.weight": "model-00066-of-00101.safetensors", + "model.layers.60.mlp.experts.99.down_proj.weight": "model-00066-of-00101.safetensors", + "model.layers.60.mlp.experts.100.gate_proj.weight": "model-00066-of-00101.safetensors", + "model.layers.60.mlp.experts.100.up_proj.weight": "model-00066-of-00101.safetensors", + "model.layers.60.mlp.experts.100.down_proj.weight": "model-00066-of-00101.safetensors", + "model.layers.60.mlp.experts.101.gate_proj.weight": "model-00066-of-00101.safetensors", + "model.layers.60.mlp.experts.101.up_proj.weight": "model-00066-of-00101.safetensors", + "model.layers.60.mlp.experts.101.down_proj.weight": "model-00066-of-00101.safetensors", + "model.layers.60.mlp.experts.102.gate_proj.weight": "model-00066-of-00101.safetensors", + "model.layers.60.mlp.experts.102.up_proj.weight": "model-00066-of-00101.safetensors", + "model.layers.60.mlp.experts.102.down_proj.weight": "model-00066-of-00101.safetensors", + "model.layers.60.mlp.experts.103.gate_proj.weight": "model-00066-of-00101.safetensors", + "model.layers.60.mlp.experts.103.up_proj.weight": "model-00066-of-00101.safetensors", + "model.layers.60.mlp.experts.103.down_proj.weight": "model-00066-of-00101.safetensors", + "model.layers.60.mlp.experts.104.gate_proj.weight": "model-00066-of-00101.safetensors", + "model.layers.60.mlp.experts.104.up_proj.weight": "model-00066-of-00101.safetensors", + "model.layers.60.mlp.experts.104.down_proj.weight": "model-00066-of-00101.safetensors", + "model.layers.60.mlp.experts.105.gate_proj.weight": "model-00066-of-00101.safetensors", + "model.layers.60.mlp.experts.105.up_proj.weight": "model-00066-of-00101.safetensors", + "model.layers.60.mlp.experts.105.down_proj.weight": "model-00066-of-00101.safetensors", + "model.layers.60.mlp.experts.106.gate_proj.weight": "model-00066-of-00101.safetensors", + "model.layers.60.mlp.experts.106.up_proj.weight": "model-00066-of-00101.safetensors", + "model.layers.60.mlp.experts.106.down_proj.weight": "model-00066-of-00101.safetensors", + "model.layers.60.mlp.experts.107.gate_proj.weight": "model-00066-of-00101.safetensors", + "model.layers.60.mlp.experts.107.up_proj.weight": "model-00066-of-00101.safetensors", + "model.layers.60.mlp.experts.107.down_proj.weight": "model-00066-of-00101.safetensors", + "model.layers.60.mlp.experts.108.gate_proj.weight": "model-00066-of-00101.safetensors", + "model.layers.60.mlp.experts.108.up_proj.weight": "model-00066-of-00101.safetensors", + "model.layers.60.mlp.experts.108.down_proj.weight": "model-00066-of-00101.safetensors", + "model.layers.60.mlp.experts.109.gate_proj.weight": "model-00066-of-00101.safetensors", + "model.layers.60.mlp.experts.109.up_proj.weight": "model-00066-of-00101.safetensors", + "model.layers.60.mlp.experts.109.down_proj.weight": "model-00066-of-00101.safetensors", + "model.layers.60.mlp.experts.110.gate_proj.weight": "model-00066-of-00101.safetensors", + "model.layers.60.mlp.experts.110.up_proj.weight": "model-00066-of-00101.safetensors", + "model.layers.60.mlp.experts.110.down_proj.weight": "model-00066-of-00101.safetensors", + "model.layers.60.mlp.experts.111.gate_proj.weight": "model-00066-of-00101.safetensors", + "model.layers.60.mlp.experts.111.up_proj.weight": "model-00066-of-00101.safetensors", + "model.layers.60.mlp.experts.111.down_proj.weight": "model-00066-of-00101.safetensors", + "model.layers.60.mlp.experts.112.gate_proj.weight": "model-00066-of-00101.safetensors", + "model.layers.60.mlp.experts.112.up_proj.weight": "model-00066-of-00101.safetensors", + "model.layers.60.mlp.experts.112.down_proj.weight": "model-00066-of-00101.safetensors", + "model.layers.60.mlp.experts.113.gate_proj.weight": "model-00066-of-00101.safetensors", + "model.layers.60.mlp.experts.113.up_proj.weight": "model-00066-of-00101.safetensors", + "model.layers.60.mlp.experts.113.down_proj.weight": "model-00066-of-00101.safetensors", + "model.layers.60.mlp.experts.114.gate_proj.weight": "model-00066-of-00101.safetensors", + "model.layers.60.mlp.experts.114.up_proj.weight": "model-00066-of-00101.safetensors", + "model.layers.60.mlp.experts.114.down_proj.weight": "model-00066-of-00101.safetensors", + "model.layers.60.mlp.experts.115.gate_proj.weight": "model-00066-of-00101.safetensors", + "model.layers.60.mlp.experts.115.up_proj.weight": "model-00066-of-00101.safetensors", + "model.layers.60.mlp.experts.115.down_proj.weight": "model-00066-of-00101.safetensors", + "model.layers.60.mlp.experts.116.gate_proj.weight": "model-00066-of-00101.safetensors", + "model.layers.60.mlp.experts.116.up_proj.weight": "model-00066-of-00101.safetensors", + "model.layers.60.mlp.experts.116.down_proj.weight": "model-00066-of-00101.safetensors", + "model.layers.60.mlp.experts.117.gate_proj.weight": "model-00066-of-00101.safetensors", + "model.layers.60.mlp.experts.117.up_proj.weight": "model-00066-of-00101.safetensors", + "model.layers.60.mlp.experts.117.down_proj.weight": "model-00066-of-00101.safetensors", + "model.layers.60.mlp.experts.118.gate_proj.weight": "model-00066-of-00101.safetensors", + "model.layers.60.mlp.experts.118.up_proj.weight": "model-00066-of-00101.safetensors", + "model.layers.60.mlp.experts.118.down_proj.weight": "model-00066-of-00101.safetensors", + "model.layers.60.mlp.experts.119.gate_proj.weight": "model-00066-of-00101.safetensors", + "model.layers.60.mlp.experts.119.up_proj.weight": "model-00066-of-00101.safetensors", + "model.layers.60.mlp.experts.119.down_proj.weight": "model-00066-of-00101.safetensors", + "model.layers.60.mlp.gate.weight": "model-00066-of-00101.safetensors", + "model.layers.60.mlp.gate.e_score_correction_bias": "model-00066-of-00101.safetensors", + "model.layers.60.mlp.shared_experts.gate_proj.weight": "model-00066-of-00101.safetensors", + "model.layers.60.mlp.shared_experts.up_proj.weight": "model-00066-of-00101.safetensors", + "model.layers.60.mlp.shared_experts.down_proj.weight": "model-00066-of-00101.safetensors", + "model.layers.60.input_layernorm.weight": "model-00066-of-00101.safetensors", + "model.layers.60.post_attention_layernorm.weight": "model-00066-of-00101.safetensors", + "model.layers.61.self_attn.q_proj.weight": "model-00066-of-00101.safetensors", + "model.layers.61.self_attn.q_proj.bias": "model-00066-of-00101.safetensors", + "model.layers.61.self_attn.k_proj.weight": "model-00066-of-00101.safetensors", + "model.layers.61.self_attn.k_proj.bias": "model-00066-of-00101.safetensors", + "model.layers.61.self_attn.v_proj.weight": "model-00066-of-00101.safetensors", + "model.layers.61.self_attn.v_proj.bias": "model-00066-of-00101.safetensors", + "model.layers.61.self_attn.o_proj.weight": "model-00066-of-00101.safetensors", + "model.layers.61.self_attn.q_norm.weight": "model-00066-of-00101.safetensors", + "model.layers.61.self_attn.k_norm.weight": "model-00066-of-00101.safetensors", + "model.layers.61.mlp.experts.0.gate_proj.weight": "model-00066-of-00101.safetensors", + "model.layers.61.mlp.experts.0.up_proj.weight": "model-00066-of-00101.safetensors", + "model.layers.61.mlp.experts.0.down_proj.weight": "model-00066-of-00101.safetensors", + "model.layers.61.mlp.experts.1.gate_proj.weight": "model-00066-of-00101.safetensors", + "model.layers.61.mlp.experts.1.up_proj.weight": "model-00066-of-00101.safetensors", + "model.layers.61.mlp.experts.1.down_proj.weight": "model-00066-of-00101.safetensors", + "model.layers.61.mlp.experts.2.gate_proj.weight": "model-00066-of-00101.safetensors", + "model.layers.61.mlp.experts.2.up_proj.weight": "model-00066-of-00101.safetensors", + "model.layers.61.mlp.experts.2.down_proj.weight": "model-00066-of-00101.safetensors", + "model.layers.61.mlp.experts.3.gate_proj.weight": "model-00066-of-00101.safetensors", + "model.layers.61.mlp.experts.3.up_proj.weight": "model-00066-of-00101.safetensors", + "model.layers.61.mlp.experts.3.down_proj.weight": "model-00066-of-00101.safetensors", + "model.layers.61.mlp.experts.4.gate_proj.weight": "model-00066-of-00101.safetensors", + "model.layers.61.mlp.experts.4.up_proj.weight": "model-00066-of-00101.safetensors", + "model.layers.61.mlp.experts.4.down_proj.weight": "model-00066-of-00101.safetensors", + "model.layers.61.mlp.experts.5.gate_proj.weight": "model-00066-of-00101.safetensors", + "model.layers.61.mlp.experts.5.up_proj.weight": "model-00066-of-00101.safetensors", + "model.layers.61.mlp.experts.5.down_proj.weight": "model-00066-of-00101.safetensors", + "model.layers.61.mlp.experts.6.gate_proj.weight": "model-00066-of-00101.safetensors", + "model.layers.61.mlp.experts.6.up_proj.weight": "model-00066-of-00101.safetensors", + "model.layers.61.mlp.experts.6.down_proj.weight": "model-00066-of-00101.safetensors", + "model.layers.61.mlp.experts.7.gate_proj.weight": "model-00066-of-00101.safetensors", + "model.layers.61.mlp.experts.7.up_proj.weight": "model-00066-of-00101.safetensors", + "model.layers.61.mlp.experts.7.down_proj.weight": "model-00066-of-00101.safetensors", + "model.layers.61.mlp.experts.8.gate_proj.weight": "model-00066-of-00101.safetensors", + "model.layers.61.mlp.experts.8.up_proj.weight": "model-00066-of-00101.safetensors", + "model.layers.61.mlp.experts.8.down_proj.weight": "model-00066-of-00101.safetensors", + "model.layers.61.mlp.experts.9.gate_proj.weight": "model-00066-of-00101.safetensors", + "model.layers.61.mlp.experts.9.up_proj.weight": "model-00066-of-00101.safetensors", + "model.layers.61.mlp.experts.9.down_proj.weight": "model-00066-of-00101.safetensors", + "model.layers.61.mlp.experts.10.gate_proj.weight": "model-00066-of-00101.safetensors", + "model.layers.61.mlp.experts.10.up_proj.weight": "model-00066-of-00101.safetensors", + "model.layers.61.mlp.experts.10.down_proj.weight": "model-00066-of-00101.safetensors", + "model.layers.61.mlp.experts.11.gate_proj.weight": "model-00066-of-00101.safetensors", + "model.layers.61.mlp.experts.11.up_proj.weight": "model-00066-of-00101.safetensors", + "model.layers.61.mlp.experts.11.down_proj.weight": "model-00066-of-00101.safetensors", + "model.layers.61.mlp.experts.12.gate_proj.weight": "model-00066-of-00101.safetensors", + "model.layers.61.mlp.experts.12.up_proj.weight": "model-00066-of-00101.safetensors", + "model.layers.61.mlp.experts.12.down_proj.weight": "model-00066-of-00101.safetensors", + "model.layers.61.mlp.experts.13.gate_proj.weight": "model-00066-of-00101.safetensors", + "model.layers.61.mlp.experts.13.up_proj.weight": "model-00066-of-00101.safetensors", + "model.layers.61.mlp.experts.13.down_proj.weight": "model-00066-of-00101.safetensors", + "model.layers.61.mlp.experts.14.gate_proj.weight": "model-00066-of-00101.safetensors", + "model.layers.61.mlp.experts.14.up_proj.weight": "model-00066-of-00101.safetensors", + "model.layers.61.mlp.experts.14.down_proj.weight": "model-00066-of-00101.safetensors", + "model.layers.61.mlp.experts.15.gate_proj.weight": "model-00066-of-00101.safetensors", + "model.layers.61.mlp.experts.15.up_proj.weight": "model-00066-of-00101.safetensors", + "model.layers.61.mlp.experts.15.down_proj.weight": "model-00066-of-00101.safetensors", + "model.layers.61.mlp.experts.16.gate_proj.weight": "model-00066-of-00101.safetensors", + "model.layers.61.mlp.experts.16.up_proj.weight": "model-00066-of-00101.safetensors", + "model.layers.61.mlp.experts.16.down_proj.weight": "model-00066-of-00101.safetensors", + "model.layers.61.mlp.experts.17.gate_proj.weight": "model-00066-of-00101.safetensors", + "model.layers.61.mlp.experts.17.up_proj.weight": "model-00066-of-00101.safetensors", + "model.layers.61.mlp.experts.17.down_proj.weight": "model-00066-of-00101.safetensors", + "model.layers.61.mlp.experts.18.gate_proj.weight": "model-00066-of-00101.safetensors", + "model.layers.61.mlp.experts.18.up_proj.weight": "model-00066-of-00101.safetensors", + "model.layers.61.mlp.experts.18.down_proj.weight": "model-00066-of-00101.safetensors", + "model.layers.61.mlp.experts.19.gate_proj.weight": "model-00066-of-00101.safetensors", + "model.layers.61.mlp.experts.19.up_proj.weight": "model-00066-of-00101.safetensors", + "model.layers.61.mlp.experts.19.down_proj.weight": "model-00066-of-00101.safetensors", + "model.layers.61.mlp.experts.20.gate_proj.weight": "model-00066-of-00101.safetensors", + "model.layers.61.mlp.experts.20.up_proj.weight": "model-00066-of-00101.safetensors", + "model.layers.61.mlp.experts.20.down_proj.weight": "model-00066-of-00101.safetensors", + "model.layers.61.mlp.experts.21.gate_proj.weight": "model-00066-of-00101.safetensors", + "model.layers.61.mlp.experts.21.up_proj.weight": "model-00066-of-00101.safetensors", + "model.layers.61.mlp.experts.21.down_proj.weight": "model-00066-of-00101.safetensors", + "model.layers.61.mlp.experts.22.gate_proj.weight": "model-00066-of-00101.safetensors", + "model.layers.61.mlp.experts.22.up_proj.weight": "model-00066-of-00101.safetensors", + "model.layers.61.mlp.experts.22.down_proj.weight": "model-00066-of-00101.safetensors", + "model.layers.61.mlp.experts.23.gate_proj.weight": "model-00066-of-00101.safetensors", + "model.layers.61.mlp.experts.23.up_proj.weight": "model-00066-of-00101.safetensors", + "model.layers.61.mlp.experts.23.down_proj.weight": "model-00066-of-00101.safetensors", + "model.layers.61.mlp.experts.24.gate_proj.weight": "model-00066-of-00101.safetensors", + "model.layers.61.mlp.experts.24.up_proj.weight": "model-00066-of-00101.safetensors", + "model.layers.61.mlp.experts.24.down_proj.weight": "model-00066-of-00101.safetensors", + "model.layers.61.mlp.experts.25.gate_proj.weight": "model-00066-of-00101.safetensors", + "model.layers.61.mlp.experts.25.up_proj.weight": "model-00066-of-00101.safetensors", + "model.layers.61.mlp.experts.25.down_proj.weight": "model-00066-of-00101.safetensors", + "model.layers.61.mlp.experts.26.gate_proj.weight": "model-00066-of-00101.safetensors", + "model.layers.61.mlp.experts.26.up_proj.weight": "model-00066-of-00101.safetensors", + "model.layers.61.mlp.experts.26.down_proj.weight": "model-00066-of-00101.safetensors", + "model.layers.61.mlp.experts.27.gate_proj.weight": "model-00066-of-00101.safetensors", + "model.layers.61.mlp.experts.27.up_proj.weight": "model-00066-of-00101.safetensors", + "model.layers.61.mlp.experts.27.down_proj.weight": "model-00066-of-00101.safetensors", + "model.layers.61.mlp.experts.28.gate_proj.weight": "model-00066-of-00101.safetensors", + "model.layers.61.mlp.experts.28.up_proj.weight": "model-00066-of-00101.safetensors", + "model.layers.61.mlp.experts.28.down_proj.weight": "model-00066-of-00101.safetensors", + "model.layers.61.mlp.experts.29.gate_proj.weight": "model-00066-of-00101.safetensors", + "model.layers.61.mlp.experts.29.up_proj.weight": "model-00066-of-00101.safetensors", + "model.layers.61.mlp.experts.29.down_proj.weight": "model-00066-of-00101.safetensors", + "model.layers.61.mlp.experts.30.gate_proj.weight": "model-00066-of-00101.safetensors", + "model.layers.61.mlp.experts.30.up_proj.weight": "model-00066-of-00101.safetensors", + "model.layers.61.mlp.experts.30.down_proj.weight": "model-00066-of-00101.safetensors", + "model.layers.61.mlp.experts.31.gate_proj.weight": "model-00066-of-00101.safetensors", + "model.layers.61.mlp.experts.31.up_proj.weight": "model-00066-of-00101.safetensors", + "model.layers.61.mlp.experts.31.down_proj.weight": "model-00066-of-00101.safetensors", + "model.layers.61.mlp.experts.32.gate_proj.weight": "model-00066-of-00101.safetensors", + "model.layers.61.mlp.experts.32.up_proj.weight": "model-00066-of-00101.safetensors", + "model.layers.61.mlp.experts.32.down_proj.weight": "model-00066-of-00101.safetensors", + "model.layers.61.mlp.experts.33.gate_proj.weight": "model-00066-of-00101.safetensors", + "model.layers.61.mlp.experts.33.up_proj.weight": "model-00066-of-00101.safetensors", + "model.layers.61.mlp.experts.33.down_proj.weight": "model-00066-of-00101.safetensors", + "model.layers.61.mlp.experts.34.gate_proj.weight": "model-00066-of-00101.safetensors", + "model.layers.61.mlp.experts.34.up_proj.weight": "model-00066-of-00101.safetensors", + "model.layers.61.mlp.experts.34.down_proj.weight": "model-00066-of-00101.safetensors", + "model.layers.61.mlp.experts.35.gate_proj.weight": "model-00066-of-00101.safetensors", + "model.layers.61.mlp.experts.35.up_proj.weight": "model-00066-of-00101.safetensors", + "model.layers.61.mlp.experts.35.down_proj.weight": "model-00066-of-00101.safetensors", + "model.layers.61.mlp.experts.36.gate_proj.weight": "model-00066-of-00101.safetensors", + "model.layers.61.mlp.experts.36.up_proj.weight": "model-00066-of-00101.safetensors", + "model.layers.61.mlp.experts.36.down_proj.weight": "model-00066-of-00101.safetensors", + "model.layers.61.mlp.experts.37.gate_proj.weight": "model-00066-of-00101.safetensors", + "model.layers.61.mlp.experts.37.up_proj.weight": "model-00066-of-00101.safetensors", + "model.layers.61.mlp.experts.37.down_proj.weight": "model-00066-of-00101.safetensors", + "model.layers.61.mlp.experts.38.gate_proj.weight": "model-00066-of-00101.safetensors", + "model.layers.61.mlp.experts.38.up_proj.weight": "model-00066-of-00101.safetensors", + "model.layers.61.mlp.experts.38.down_proj.weight": "model-00066-of-00101.safetensors", + "model.layers.61.mlp.experts.39.gate_proj.weight": "model-00066-of-00101.safetensors", + "model.layers.61.mlp.experts.39.up_proj.weight": "model-00066-of-00101.safetensors", + "model.layers.61.mlp.experts.39.down_proj.weight": "model-00066-of-00101.safetensors", + "model.layers.61.mlp.experts.40.gate_proj.weight": "model-00066-of-00101.safetensors", + "model.layers.61.mlp.experts.40.up_proj.weight": "model-00066-of-00101.safetensors", + "model.layers.61.mlp.experts.40.down_proj.weight": "model-00066-of-00101.safetensors", + "model.layers.61.mlp.experts.41.gate_proj.weight": "model-00066-of-00101.safetensors", + "model.layers.61.mlp.experts.41.up_proj.weight": "model-00066-of-00101.safetensors", + "model.layers.61.mlp.experts.41.down_proj.weight": "model-00066-of-00101.safetensors", + "model.layers.61.mlp.experts.42.gate_proj.weight": "model-00066-of-00101.safetensors", + "model.layers.61.mlp.experts.42.up_proj.weight": "model-00066-of-00101.safetensors", + "model.layers.61.mlp.experts.42.down_proj.weight": "model-00066-of-00101.safetensors", + "model.layers.61.mlp.experts.43.gate_proj.weight": "model-00066-of-00101.safetensors", + "model.layers.61.mlp.experts.43.up_proj.weight": "model-00066-of-00101.safetensors", + "model.layers.61.mlp.experts.43.down_proj.weight": "model-00066-of-00101.safetensors", + "model.layers.61.mlp.experts.44.gate_proj.weight": "model-00066-of-00101.safetensors", + "model.layers.61.mlp.experts.44.up_proj.weight": "model-00066-of-00101.safetensors", + "model.layers.61.mlp.experts.44.down_proj.weight": "model-00066-of-00101.safetensors", + "model.layers.61.mlp.experts.45.gate_proj.weight": "model-00066-of-00101.safetensors", + "model.layers.61.mlp.experts.45.up_proj.weight": "model-00066-of-00101.safetensors", + "model.layers.61.mlp.experts.45.down_proj.weight": "model-00066-of-00101.safetensors", + "model.layers.61.mlp.experts.46.gate_proj.weight": "model-00066-of-00101.safetensors", + "model.layers.61.mlp.experts.46.up_proj.weight": "model-00066-of-00101.safetensors", + "model.layers.61.mlp.experts.46.down_proj.weight": "model-00066-of-00101.safetensors", + "model.layers.61.mlp.experts.47.gate_proj.weight": "model-00066-of-00101.safetensors", + "model.layers.61.mlp.experts.47.up_proj.weight": "model-00066-of-00101.safetensors", + "model.layers.61.mlp.experts.47.down_proj.weight": "model-00066-of-00101.safetensors", + "model.layers.61.mlp.experts.48.gate_proj.weight": "model-00066-of-00101.safetensors", + "model.layers.61.mlp.experts.48.up_proj.weight": "model-00066-of-00101.safetensors", + "model.layers.61.mlp.experts.48.down_proj.weight": "model-00066-of-00101.safetensors", + "model.layers.61.mlp.experts.49.gate_proj.weight": "model-00066-of-00101.safetensors", + "model.layers.61.mlp.experts.49.up_proj.weight": "model-00066-of-00101.safetensors", + "model.layers.61.mlp.experts.49.down_proj.weight": "model-00066-of-00101.safetensors", + "model.layers.61.mlp.experts.50.gate_proj.weight": "model-00066-of-00101.safetensors", + "model.layers.61.mlp.experts.50.up_proj.weight": "model-00066-of-00101.safetensors", + "model.layers.61.mlp.experts.50.down_proj.weight": "model-00066-of-00101.safetensors", + "model.layers.61.mlp.experts.51.gate_proj.weight": "model-00066-of-00101.safetensors", + "model.layers.61.mlp.experts.51.up_proj.weight": "model-00066-of-00101.safetensors", + "model.layers.61.mlp.experts.51.down_proj.weight": "model-00066-of-00101.safetensors", + "model.layers.61.mlp.experts.52.gate_proj.weight": "model-00066-of-00101.safetensors", + "model.layers.61.mlp.experts.52.up_proj.weight": "model-00066-of-00101.safetensors", + "model.layers.61.mlp.experts.52.down_proj.weight": "model-00066-of-00101.safetensors", + "model.layers.61.mlp.experts.53.gate_proj.weight": "model-00066-of-00101.safetensors", + "model.layers.61.mlp.experts.53.up_proj.weight": "model-00066-of-00101.safetensors", + "model.layers.61.mlp.experts.53.down_proj.weight": "model-00066-of-00101.safetensors", + "model.layers.61.mlp.experts.54.gate_proj.weight": "model-00066-of-00101.safetensors", + "model.layers.61.mlp.experts.54.up_proj.weight": "model-00067-of-00101.safetensors", + "model.layers.61.mlp.experts.54.down_proj.weight": "model-00067-of-00101.safetensors", + "model.layers.61.mlp.experts.55.gate_proj.weight": "model-00067-of-00101.safetensors", + "model.layers.61.mlp.experts.55.up_proj.weight": "model-00067-of-00101.safetensors", + "model.layers.61.mlp.experts.55.down_proj.weight": "model-00067-of-00101.safetensors", + "model.layers.61.mlp.experts.56.gate_proj.weight": "model-00067-of-00101.safetensors", + "model.layers.61.mlp.experts.56.up_proj.weight": "model-00067-of-00101.safetensors", + "model.layers.61.mlp.experts.56.down_proj.weight": "model-00067-of-00101.safetensors", + "model.layers.61.mlp.experts.57.gate_proj.weight": "model-00067-of-00101.safetensors", + "model.layers.61.mlp.experts.57.up_proj.weight": "model-00067-of-00101.safetensors", + "model.layers.61.mlp.experts.57.down_proj.weight": "model-00067-of-00101.safetensors", + "model.layers.61.mlp.experts.58.gate_proj.weight": "model-00067-of-00101.safetensors", + "model.layers.61.mlp.experts.58.up_proj.weight": "model-00067-of-00101.safetensors", + "model.layers.61.mlp.experts.58.down_proj.weight": "model-00067-of-00101.safetensors", + "model.layers.61.mlp.experts.59.gate_proj.weight": "model-00067-of-00101.safetensors", + "model.layers.61.mlp.experts.59.up_proj.weight": "model-00067-of-00101.safetensors", + "model.layers.61.mlp.experts.59.down_proj.weight": "model-00067-of-00101.safetensors", + "model.layers.61.mlp.experts.60.gate_proj.weight": "model-00067-of-00101.safetensors", + "model.layers.61.mlp.experts.60.up_proj.weight": "model-00067-of-00101.safetensors", + "model.layers.61.mlp.experts.60.down_proj.weight": "model-00067-of-00101.safetensors", + "model.layers.61.mlp.experts.61.gate_proj.weight": "model-00067-of-00101.safetensors", + "model.layers.61.mlp.experts.61.up_proj.weight": "model-00067-of-00101.safetensors", + "model.layers.61.mlp.experts.61.down_proj.weight": "model-00067-of-00101.safetensors", + "model.layers.61.mlp.experts.62.gate_proj.weight": "model-00067-of-00101.safetensors", + "model.layers.61.mlp.experts.62.up_proj.weight": "model-00067-of-00101.safetensors", + "model.layers.61.mlp.experts.62.down_proj.weight": "model-00067-of-00101.safetensors", + "model.layers.61.mlp.experts.63.gate_proj.weight": "model-00067-of-00101.safetensors", + "model.layers.61.mlp.experts.63.up_proj.weight": "model-00067-of-00101.safetensors", + "model.layers.61.mlp.experts.63.down_proj.weight": "model-00067-of-00101.safetensors", + "model.layers.61.mlp.experts.64.gate_proj.weight": "model-00067-of-00101.safetensors", + "model.layers.61.mlp.experts.64.up_proj.weight": "model-00067-of-00101.safetensors", + "model.layers.61.mlp.experts.64.down_proj.weight": "model-00067-of-00101.safetensors", + "model.layers.61.mlp.experts.65.gate_proj.weight": "model-00067-of-00101.safetensors", + "model.layers.61.mlp.experts.65.up_proj.weight": "model-00067-of-00101.safetensors", + "model.layers.61.mlp.experts.65.down_proj.weight": "model-00067-of-00101.safetensors", + "model.layers.61.mlp.experts.66.gate_proj.weight": "model-00067-of-00101.safetensors", + "model.layers.61.mlp.experts.66.up_proj.weight": "model-00067-of-00101.safetensors", + "model.layers.61.mlp.experts.66.down_proj.weight": "model-00067-of-00101.safetensors", + "model.layers.61.mlp.experts.67.gate_proj.weight": "model-00067-of-00101.safetensors", + "model.layers.61.mlp.experts.67.up_proj.weight": "model-00067-of-00101.safetensors", + "model.layers.61.mlp.experts.67.down_proj.weight": "model-00067-of-00101.safetensors", + "model.layers.61.mlp.experts.68.gate_proj.weight": "model-00067-of-00101.safetensors", + "model.layers.61.mlp.experts.68.up_proj.weight": "model-00067-of-00101.safetensors", + "model.layers.61.mlp.experts.68.down_proj.weight": "model-00067-of-00101.safetensors", + "model.layers.61.mlp.experts.69.gate_proj.weight": "model-00067-of-00101.safetensors", + "model.layers.61.mlp.experts.69.up_proj.weight": "model-00067-of-00101.safetensors", + "model.layers.61.mlp.experts.69.down_proj.weight": "model-00067-of-00101.safetensors", + "model.layers.61.mlp.experts.70.gate_proj.weight": "model-00067-of-00101.safetensors", + "model.layers.61.mlp.experts.70.up_proj.weight": "model-00067-of-00101.safetensors", + "model.layers.61.mlp.experts.70.down_proj.weight": "model-00067-of-00101.safetensors", + "model.layers.61.mlp.experts.71.gate_proj.weight": "model-00067-of-00101.safetensors", + "model.layers.61.mlp.experts.71.up_proj.weight": "model-00067-of-00101.safetensors", + "model.layers.61.mlp.experts.71.down_proj.weight": "model-00067-of-00101.safetensors", + "model.layers.61.mlp.experts.72.gate_proj.weight": "model-00067-of-00101.safetensors", + "model.layers.61.mlp.experts.72.up_proj.weight": "model-00067-of-00101.safetensors", + "model.layers.61.mlp.experts.72.down_proj.weight": "model-00067-of-00101.safetensors", + "model.layers.61.mlp.experts.73.gate_proj.weight": "model-00067-of-00101.safetensors", + "model.layers.61.mlp.experts.73.up_proj.weight": "model-00067-of-00101.safetensors", + "model.layers.61.mlp.experts.73.down_proj.weight": "model-00067-of-00101.safetensors", + "model.layers.61.mlp.experts.74.gate_proj.weight": "model-00067-of-00101.safetensors", + "model.layers.61.mlp.experts.74.up_proj.weight": "model-00067-of-00101.safetensors", + "model.layers.61.mlp.experts.74.down_proj.weight": "model-00067-of-00101.safetensors", + "model.layers.61.mlp.experts.75.gate_proj.weight": "model-00067-of-00101.safetensors", + "model.layers.61.mlp.experts.75.up_proj.weight": "model-00067-of-00101.safetensors", + "model.layers.61.mlp.experts.75.down_proj.weight": "model-00067-of-00101.safetensors", + "model.layers.61.mlp.experts.76.gate_proj.weight": "model-00067-of-00101.safetensors", + "model.layers.61.mlp.experts.76.up_proj.weight": "model-00067-of-00101.safetensors", + "model.layers.61.mlp.experts.76.down_proj.weight": "model-00067-of-00101.safetensors", + "model.layers.61.mlp.experts.77.gate_proj.weight": "model-00067-of-00101.safetensors", + "model.layers.61.mlp.experts.77.up_proj.weight": "model-00067-of-00101.safetensors", + "model.layers.61.mlp.experts.77.down_proj.weight": "model-00067-of-00101.safetensors", + "model.layers.61.mlp.experts.78.gate_proj.weight": "model-00067-of-00101.safetensors", + "model.layers.61.mlp.experts.78.up_proj.weight": "model-00067-of-00101.safetensors", + "model.layers.61.mlp.experts.78.down_proj.weight": "model-00067-of-00101.safetensors", + "model.layers.61.mlp.experts.79.gate_proj.weight": "model-00067-of-00101.safetensors", + "model.layers.61.mlp.experts.79.up_proj.weight": "model-00067-of-00101.safetensors", + "model.layers.61.mlp.experts.79.down_proj.weight": "model-00067-of-00101.safetensors", + "model.layers.61.mlp.experts.80.gate_proj.weight": "model-00067-of-00101.safetensors", + "model.layers.61.mlp.experts.80.up_proj.weight": "model-00067-of-00101.safetensors", + "model.layers.61.mlp.experts.80.down_proj.weight": "model-00067-of-00101.safetensors", + "model.layers.61.mlp.experts.81.gate_proj.weight": "model-00067-of-00101.safetensors", + "model.layers.61.mlp.experts.81.up_proj.weight": "model-00067-of-00101.safetensors", + "model.layers.61.mlp.experts.81.down_proj.weight": "model-00067-of-00101.safetensors", + "model.layers.61.mlp.experts.82.gate_proj.weight": "model-00067-of-00101.safetensors", + "model.layers.61.mlp.experts.82.up_proj.weight": "model-00067-of-00101.safetensors", + "model.layers.61.mlp.experts.82.down_proj.weight": "model-00067-of-00101.safetensors", + "model.layers.61.mlp.experts.83.gate_proj.weight": "model-00067-of-00101.safetensors", + "model.layers.61.mlp.experts.83.up_proj.weight": "model-00067-of-00101.safetensors", + "model.layers.61.mlp.experts.83.down_proj.weight": "model-00067-of-00101.safetensors", + "model.layers.61.mlp.experts.84.gate_proj.weight": "model-00067-of-00101.safetensors", + "model.layers.61.mlp.experts.84.up_proj.weight": "model-00067-of-00101.safetensors", + "model.layers.61.mlp.experts.84.down_proj.weight": "model-00067-of-00101.safetensors", + "model.layers.61.mlp.experts.85.gate_proj.weight": "model-00067-of-00101.safetensors", + "model.layers.61.mlp.experts.85.up_proj.weight": "model-00067-of-00101.safetensors", + "model.layers.61.mlp.experts.85.down_proj.weight": "model-00067-of-00101.safetensors", + "model.layers.61.mlp.experts.86.gate_proj.weight": "model-00067-of-00101.safetensors", + "model.layers.61.mlp.experts.86.up_proj.weight": "model-00067-of-00101.safetensors", + "model.layers.61.mlp.experts.86.down_proj.weight": "model-00067-of-00101.safetensors", + "model.layers.61.mlp.experts.87.gate_proj.weight": "model-00067-of-00101.safetensors", + "model.layers.61.mlp.experts.87.up_proj.weight": "model-00067-of-00101.safetensors", + "model.layers.61.mlp.experts.87.down_proj.weight": "model-00067-of-00101.safetensors", + "model.layers.61.mlp.experts.88.gate_proj.weight": "model-00067-of-00101.safetensors", + "model.layers.61.mlp.experts.88.up_proj.weight": "model-00067-of-00101.safetensors", + "model.layers.61.mlp.experts.88.down_proj.weight": "model-00067-of-00101.safetensors", + "model.layers.61.mlp.experts.89.gate_proj.weight": "model-00067-of-00101.safetensors", + "model.layers.61.mlp.experts.89.up_proj.weight": "model-00067-of-00101.safetensors", + "model.layers.61.mlp.experts.89.down_proj.weight": "model-00067-of-00101.safetensors", + "model.layers.61.mlp.experts.90.gate_proj.weight": "model-00067-of-00101.safetensors", + "model.layers.61.mlp.experts.90.up_proj.weight": "model-00067-of-00101.safetensors", + "model.layers.61.mlp.experts.90.down_proj.weight": "model-00067-of-00101.safetensors", + "model.layers.61.mlp.experts.91.gate_proj.weight": "model-00067-of-00101.safetensors", + "model.layers.61.mlp.experts.91.up_proj.weight": "model-00067-of-00101.safetensors", + "model.layers.61.mlp.experts.91.down_proj.weight": "model-00067-of-00101.safetensors", + "model.layers.61.mlp.experts.92.gate_proj.weight": "model-00067-of-00101.safetensors", + "model.layers.61.mlp.experts.92.up_proj.weight": "model-00067-of-00101.safetensors", + "model.layers.61.mlp.experts.92.down_proj.weight": "model-00067-of-00101.safetensors", + "model.layers.61.mlp.experts.93.gate_proj.weight": "model-00067-of-00101.safetensors", + "model.layers.61.mlp.experts.93.up_proj.weight": "model-00067-of-00101.safetensors", + "model.layers.61.mlp.experts.93.down_proj.weight": "model-00067-of-00101.safetensors", + "model.layers.61.mlp.experts.94.gate_proj.weight": "model-00067-of-00101.safetensors", + "model.layers.61.mlp.experts.94.up_proj.weight": "model-00067-of-00101.safetensors", + "model.layers.61.mlp.experts.94.down_proj.weight": "model-00067-of-00101.safetensors", + "model.layers.61.mlp.experts.95.gate_proj.weight": "model-00067-of-00101.safetensors", + "model.layers.61.mlp.experts.95.up_proj.weight": "model-00067-of-00101.safetensors", + "model.layers.61.mlp.experts.95.down_proj.weight": "model-00067-of-00101.safetensors", + "model.layers.61.mlp.experts.96.gate_proj.weight": "model-00067-of-00101.safetensors", + "model.layers.61.mlp.experts.96.up_proj.weight": "model-00067-of-00101.safetensors", + "model.layers.61.mlp.experts.96.down_proj.weight": "model-00067-of-00101.safetensors", + "model.layers.61.mlp.experts.97.gate_proj.weight": "model-00067-of-00101.safetensors", + "model.layers.61.mlp.experts.97.up_proj.weight": "model-00067-of-00101.safetensors", + "model.layers.61.mlp.experts.97.down_proj.weight": "model-00067-of-00101.safetensors", + "model.layers.61.mlp.experts.98.gate_proj.weight": "model-00067-of-00101.safetensors", + "model.layers.61.mlp.experts.98.up_proj.weight": "model-00067-of-00101.safetensors", + "model.layers.61.mlp.experts.98.down_proj.weight": "model-00067-of-00101.safetensors", + "model.layers.61.mlp.experts.99.gate_proj.weight": "model-00067-of-00101.safetensors", + "model.layers.61.mlp.experts.99.up_proj.weight": "model-00067-of-00101.safetensors", + "model.layers.61.mlp.experts.99.down_proj.weight": "model-00067-of-00101.safetensors", + "model.layers.61.mlp.experts.100.gate_proj.weight": "model-00067-of-00101.safetensors", + "model.layers.61.mlp.experts.100.up_proj.weight": "model-00067-of-00101.safetensors", + "model.layers.61.mlp.experts.100.down_proj.weight": "model-00067-of-00101.safetensors", + "model.layers.61.mlp.experts.101.gate_proj.weight": "model-00067-of-00101.safetensors", + "model.layers.61.mlp.experts.101.up_proj.weight": "model-00067-of-00101.safetensors", + "model.layers.61.mlp.experts.101.down_proj.weight": "model-00067-of-00101.safetensors", + "model.layers.61.mlp.experts.102.gate_proj.weight": "model-00067-of-00101.safetensors", + "model.layers.61.mlp.experts.102.up_proj.weight": "model-00067-of-00101.safetensors", + "model.layers.61.mlp.experts.102.down_proj.weight": "model-00067-of-00101.safetensors", + "model.layers.61.mlp.experts.103.gate_proj.weight": "model-00067-of-00101.safetensors", + "model.layers.61.mlp.experts.103.up_proj.weight": "model-00067-of-00101.safetensors", + "model.layers.61.mlp.experts.103.down_proj.weight": "model-00067-of-00101.safetensors", + "model.layers.61.mlp.experts.104.gate_proj.weight": "model-00067-of-00101.safetensors", + "model.layers.61.mlp.experts.104.up_proj.weight": "model-00067-of-00101.safetensors", + "model.layers.61.mlp.experts.104.down_proj.weight": "model-00067-of-00101.safetensors", + "model.layers.61.mlp.experts.105.gate_proj.weight": "model-00067-of-00101.safetensors", + "model.layers.61.mlp.experts.105.up_proj.weight": "model-00067-of-00101.safetensors", + "model.layers.61.mlp.experts.105.down_proj.weight": "model-00067-of-00101.safetensors", + "model.layers.61.mlp.experts.106.gate_proj.weight": "model-00067-of-00101.safetensors", + "model.layers.61.mlp.experts.106.up_proj.weight": "model-00067-of-00101.safetensors", + "model.layers.61.mlp.experts.106.down_proj.weight": "model-00067-of-00101.safetensors", + "model.layers.61.mlp.experts.107.gate_proj.weight": "model-00067-of-00101.safetensors", + "model.layers.61.mlp.experts.107.up_proj.weight": "model-00067-of-00101.safetensors", + "model.layers.61.mlp.experts.107.down_proj.weight": "model-00067-of-00101.safetensors", + "model.layers.61.mlp.experts.108.gate_proj.weight": "model-00067-of-00101.safetensors", + "model.layers.61.mlp.experts.108.up_proj.weight": "model-00067-of-00101.safetensors", + "model.layers.61.mlp.experts.108.down_proj.weight": "model-00067-of-00101.safetensors", + "model.layers.61.mlp.experts.109.gate_proj.weight": "model-00067-of-00101.safetensors", + "model.layers.61.mlp.experts.109.up_proj.weight": "model-00067-of-00101.safetensors", + "model.layers.61.mlp.experts.109.down_proj.weight": "model-00067-of-00101.safetensors", + "model.layers.61.mlp.experts.110.gate_proj.weight": "model-00067-of-00101.safetensors", + "model.layers.61.mlp.experts.110.up_proj.weight": "model-00067-of-00101.safetensors", + "model.layers.61.mlp.experts.110.down_proj.weight": "model-00067-of-00101.safetensors", + "model.layers.61.mlp.experts.111.gate_proj.weight": "model-00067-of-00101.safetensors", + "model.layers.61.mlp.experts.111.up_proj.weight": "model-00067-of-00101.safetensors", + "model.layers.61.mlp.experts.111.down_proj.weight": "model-00067-of-00101.safetensors", + "model.layers.61.mlp.experts.112.gate_proj.weight": "model-00067-of-00101.safetensors", + "model.layers.61.mlp.experts.112.up_proj.weight": "model-00067-of-00101.safetensors", + "model.layers.61.mlp.experts.112.down_proj.weight": "model-00067-of-00101.safetensors", + "model.layers.61.mlp.experts.113.gate_proj.weight": "model-00067-of-00101.safetensors", + "model.layers.61.mlp.experts.113.up_proj.weight": "model-00067-of-00101.safetensors", + "model.layers.61.mlp.experts.113.down_proj.weight": "model-00067-of-00101.safetensors", + "model.layers.61.mlp.experts.114.gate_proj.weight": "model-00067-of-00101.safetensors", + "model.layers.61.mlp.experts.114.up_proj.weight": "model-00067-of-00101.safetensors", + "model.layers.61.mlp.experts.114.down_proj.weight": "model-00067-of-00101.safetensors", + "model.layers.61.mlp.experts.115.gate_proj.weight": "model-00067-of-00101.safetensors", + "model.layers.61.mlp.experts.115.up_proj.weight": "model-00067-of-00101.safetensors", + "model.layers.61.mlp.experts.115.down_proj.weight": "model-00067-of-00101.safetensors", + "model.layers.61.mlp.experts.116.gate_proj.weight": "model-00067-of-00101.safetensors", + "model.layers.61.mlp.experts.116.up_proj.weight": "model-00067-of-00101.safetensors", + "model.layers.61.mlp.experts.116.down_proj.weight": "model-00067-of-00101.safetensors", + "model.layers.61.mlp.experts.117.gate_proj.weight": "model-00067-of-00101.safetensors", + "model.layers.61.mlp.experts.117.up_proj.weight": "model-00067-of-00101.safetensors", + "model.layers.61.mlp.experts.117.down_proj.weight": "model-00067-of-00101.safetensors", + "model.layers.61.mlp.experts.118.gate_proj.weight": "model-00067-of-00101.safetensors", + "model.layers.61.mlp.experts.118.up_proj.weight": "model-00067-of-00101.safetensors", + "model.layers.61.mlp.experts.118.down_proj.weight": "model-00067-of-00101.safetensors", + "model.layers.61.mlp.experts.119.gate_proj.weight": "model-00067-of-00101.safetensors", + "model.layers.61.mlp.experts.119.up_proj.weight": "model-00067-of-00101.safetensors", + "model.layers.61.mlp.experts.119.down_proj.weight": "model-00067-of-00101.safetensors", + "model.layers.61.mlp.gate.weight": "model-00067-of-00101.safetensors", + "model.layers.61.mlp.gate.e_score_correction_bias": "model-00067-of-00101.safetensors", + "model.layers.61.mlp.shared_experts.gate_proj.weight": "model-00067-of-00101.safetensors", + "model.layers.61.mlp.shared_experts.up_proj.weight": "model-00067-of-00101.safetensors", + "model.layers.61.mlp.shared_experts.down_proj.weight": "model-00067-of-00101.safetensors", + "model.layers.61.input_layernorm.weight": "model-00067-of-00101.safetensors", + "model.layers.61.post_attention_layernorm.weight": "model-00067-of-00101.safetensors", + "model.layers.62.self_attn.q_proj.weight": "model-00067-of-00101.safetensors", + "model.layers.62.self_attn.q_proj.bias": "model-00067-of-00101.safetensors", + "model.layers.62.self_attn.k_proj.weight": "model-00067-of-00101.safetensors", + "model.layers.62.self_attn.k_proj.bias": "model-00067-of-00101.safetensors", + "model.layers.62.self_attn.v_proj.weight": "model-00067-of-00101.safetensors", + "model.layers.62.self_attn.v_proj.bias": "model-00067-of-00101.safetensors", + "model.layers.62.self_attn.o_proj.weight": "model-00067-of-00101.safetensors", + "model.layers.62.self_attn.q_norm.weight": "model-00067-of-00101.safetensors", + "model.layers.62.self_attn.k_norm.weight": "model-00067-of-00101.safetensors", + "model.layers.62.mlp.experts.0.gate_proj.weight": "model-00067-of-00101.safetensors", + "model.layers.62.mlp.experts.0.up_proj.weight": "model-00067-of-00101.safetensors", + "model.layers.62.mlp.experts.0.down_proj.weight": "model-00067-of-00101.safetensors", + "model.layers.62.mlp.experts.1.gate_proj.weight": "model-00067-of-00101.safetensors", + "model.layers.62.mlp.experts.1.up_proj.weight": "model-00067-of-00101.safetensors", + "model.layers.62.mlp.experts.1.down_proj.weight": "model-00067-of-00101.safetensors", + "model.layers.62.mlp.experts.2.gate_proj.weight": "model-00067-of-00101.safetensors", + "model.layers.62.mlp.experts.2.up_proj.weight": "model-00067-of-00101.safetensors", + "model.layers.62.mlp.experts.2.down_proj.weight": "model-00067-of-00101.safetensors", + "model.layers.62.mlp.experts.3.gate_proj.weight": "model-00067-of-00101.safetensors", + "model.layers.62.mlp.experts.3.up_proj.weight": "model-00067-of-00101.safetensors", + "model.layers.62.mlp.experts.3.down_proj.weight": "model-00067-of-00101.safetensors", + "model.layers.62.mlp.experts.4.gate_proj.weight": "model-00067-of-00101.safetensors", + "model.layers.62.mlp.experts.4.up_proj.weight": "model-00067-of-00101.safetensors", + "model.layers.62.mlp.experts.4.down_proj.weight": "model-00067-of-00101.safetensors", + "model.layers.62.mlp.experts.5.gate_proj.weight": "model-00067-of-00101.safetensors", + "model.layers.62.mlp.experts.5.up_proj.weight": "model-00067-of-00101.safetensors", + "model.layers.62.mlp.experts.5.down_proj.weight": "model-00067-of-00101.safetensors", + "model.layers.62.mlp.experts.6.gate_proj.weight": "model-00067-of-00101.safetensors", + "model.layers.62.mlp.experts.6.up_proj.weight": "model-00067-of-00101.safetensors", + "model.layers.62.mlp.experts.6.down_proj.weight": "model-00067-of-00101.safetensors", + "model.layers.62.mlp.experts.7.gate_proj.weight": "model-00067-of-00101.safetensors", + "model.layers.62.mlp.experts.7.up_proj.weight": "model-00067-of-00101.safetensors", + "model.layers.62.mlp.experts.7.down_proj.weight": "model-00067-of-00101.safetensors", + "model.layers.62.mlp.experts.8.gate_proj.weight": "model-00067-of-00101.safetensors", + "model.layers.62.mlp.experts.8.up_proj.weight": "model-00067-of-00101.safetensors", + "model.layers.62.mlp.experts.8.down_proj.weight": "model-00067-of-00101.safetensors", + "model.layers.62.mlp.experts.9.gate_proj.weight": "model-00067-of-00101.safetensors", + "model.layers.62.mlp.experts.9.up_proj.weight": "model-00067-of-00101.safetensors", + "model.layers.62.mlp.experts.9.down_proj.weight": "model-00067-of-00101.safetensors", + "model.layers.62.mlp.experts.10.gate_proj.weight": "model-00067-of-00101.safetensors", + "model.layers.62.mlp.experts.10.up_proj.weight": "model-00067-of-00101.safetensors", + "model.layers.62.mlp.experts.10.down_proj.weight": "model-00067-of-00101.safetensors", + "model.layers.62.mlp.experts.11.gate_proj.weight": "model-00067-of-00101.safetensors", + "model.layers.62.mlp.experts.11.up_proj.weight": "model-00067-of-00101.safetensors", + "model.layers.62.mlp.experts.11.down_proj.weight": "model-00067-of-00101.safetensors", + "model.layers.62.mlp.experts.12.gate_proj.weight": "model-00067-of-00101.safetensors", + "model.layers.62.mlp.experts.12.up_proj.weight": "model-00067-of-00101.safetensors", + "model.layers.62.mlp.experts.12.down_proj.weight": "model-00067-of-00101.safetensors", + "model.layers.62.mlp.experts.13.gate_proj.weight": "model-00067-of-00101.safetensors", + "model.layers.62.mlp.experts.13.up_proj.weight": "model-00067-of-00101.safetensors", + "model.layers.62.mlp.experts.13.down_proj.weight": "model-00067-of-00101.safetensors", + "model.layers.62.mlp.experts.14.gate_proj.weight": "model-00067-of-00101.safetensors", + "model.layers.62.mlp.experts.14.up_proj.weight": "model-00067-of-00101.safetensors", + "model.layers.62.mlp.experts.14.down_proj.weight": "model-00067-of-00101.safetensors", + "model.layers.62.mlp.experts.15.gate_proj.weight": "model-00067-of-00101.safetensors", + "model.layers.62.mlp.experts.15.up_proj.weight": "model-00067-of-00101.safetensors", + "model.layers.62.mlp.experts.15.down_proj.weight": "model-00067-of-00101.safetensors", + "model.layers.62.mlp.experts.16.gate_proj.weight": "model-00067-of-00101.safetensors", + "model.layers.62.mlp.experts.16.up_proj.weight": "model-00067-of-00101.safetensors", + "model.layers.62.mlp.experts.16.down_proj.weight": "model-00067-of-00101.safetensors", + "model.layers.62.mlp.experts.17.gate_proj.weight": "model-00067-of-00101.safetensors", + "model.layers.62.mlp.experts.17.up_proj.weight": "model-00067-of-00101.safetensors", + "model.layers.62.mlp.experts.17.down_proj.weight": "model-00067-of-00101.safetensors", + "model.layers.62.mlp.experts.18.gate_proj.weight": "model-00067-of-00101.safetensors", + "model.layers.62.mlp.experts.18.up_proj.weight": "model-00067-of-00101.safetensors", + "model.layers.62.mlp.experts.18.down_proj.weight": "model-00067-of-00101.safetensors", + "model.layers.62.mlp.experts.19.gate_proj.weight": "model-00067-of-00101.safetensors", + "model.layers.62.mlp.experts.19.up_proj.weight": "model-00067-of-00101.safetensors", + "model.layers.62.mlp.experts.19.down_proj.weight": "model-00067-of-00101.safetensors", + "model.layers.62.mlp.experts.20.gate_proj.weight": "model-00067-of-00101.safetensors", + "model.layers.62.mlp.experts.20.up_proj.weight": "model-00067-of-00101.safetensors", + "model.layers.62.mlp.experts.20.down_proj.weight": "model-00067-of-00101.safetensors", + "model.layers.62.mlp.experts.21.gate_proj.weight": "model-00067-of-00101.safetensors", + "model.layers.62.mlp.experts.21.up_proj.weight": "model-00067-of-00101.safetensors", + "model.layers.62.mlp.experts.21.down_proj.weight": "model-00067-of-00101.safetensors", + "model.layers.62.mlp.experts.22.gate_proj.weight": "model-00067-of-00101.safetensors", + "model.layers.62.mlp.experts.22.up_proj.weight": "model-00067-of-00101.safetensors", + "model.layers.62.mlp.experts.22.down_proj.weight": "model-00067-of-00101.safetensors", + "model.layers.62.mlp.experts.23.gate_proj.weight": "model-00067-of-00101.safetensors", + "model.layers.62.mlp.experts.23.up_proj.weight": "model-00067-of-00101.safetensors", + "model.layers.62.mlp.experts.23.down_proj.weight": "model-00067-of-00101.safetensors", + "model.layers.62.mlp.experts.24.gate_proj.weight": "model-00067-of-00101.safetensors", + "model.layers.62.mlp.experts.24.up_proj.weight": "model-00067-of-00101.safetensors", + "model.layers.62.mlp.experts.24.down_proj.weight": "model-00067-of-00101.safetensors", + "model.layers.62.mlp.experts.25.gate_proj.weight": "model-00067-of-00101.safetensors", + "model.layers.62.mlp.experts.25.up_proj.weight": "model-00067-of-00101.safetensors", + "model.layers.62.mlp.experts.25.down_proj.weight": "model-00067-of-00101.safetensors", + "model.layers.62.mlp.experts.26.gate_proj.weight": "model-00067-of-00101.safetensors", + "model.layers.62.mlp.experts.26.up_proj.weight": "model-00067-of-00101.safetensors", + "model.layers.62.mlp.experts.26.down_proj.weight": "model-00067-of-00101.safetensors", + "model.layers.62.mlp.experts.27.gate_proj.weight": "model-00067-of-00101.safetensors", + "model.layers.62.mlp.experts.27.up_proj.weight": "model-00067-of-00101.safetensors", + "model.layers.62.mlp.experts.27.down_proj.weight": "model-00067-of-00101.safetensors", + "model.layers.62.mlp.experts.28.gate_proj.weight": "model-00067-of-00101.safetensors", + "model.layers.62.mlp.experts.28.up_proj.weight": "model-00067-of-00101.safetensors", + "model.layers.62.mlp.experts.28.down_proj.weight": "model-00067-of-00101.safetensors", + "model.layers.62.mlp.experts.29.gate_proj.weight": "model-00067-of-00101.safetensors", + "model.layers.62.mlp.experts.29.up_proj.weight": "model-00067-of-00101.safetensors", + "model.layers.62.mlp.experts.29.down_proj.weight": "model-00067-of-00101.safetensors", + "model.layers.62.mlp.experts.30.gate_proj.weight": "model-00067-of-00101.safetensors", + "model.layers.62.mlp.experts.30.up_proj.weight": "model-00067-of-00101.safetensors", + "model.layers.62.mlp.experts.30.down_proj.weight": "model-00067-of-00101.safetensors", + "model.layers.62.mlp.experts.31.gate_proj.weight": "model-00067-of-00101.safetensors", + "model.layers.62.mlp.experts.31.up_proj.weight": "model-00067-of-00101.safetensors", + "model.layers.62.mlp.experts.31.down_proj.weight": "model-00067-of-00101.safetensors", + "model.layers.62.mlp.experts.32.gate_proj.weight": "model-00067-of-00101.safetensors", + "model.layers.62.mlp.experts.32.up_proj.weight": "model-00067-of-00101.safetensors", + "model.layers.62.mlp.experts.32.down_proj.weight": "model-00067-of-00101.safetensors", + "model.layers.62.mlp.experts.33.gate_proj.weight": "model-00067-of-00101.safetensors", + "model.layers.62.mlp.experts.33.up_proj.weight": "model-00067-of-00101.safetensors", + "model.layers.62.mlp.experts.33.down_proj.weight": "model-00067-of-00101.safetensors", + "model.layers.62.mlp.experts.34.gate_proj.weight": "model-00067-of-00101.safetensors", + "model.layers.62.mlp.experts.34.up_proj.weight": "model-00067-of-00101.safetensors", + "model.layers.62.mlp.experts.34.down_proj.weight": "model-00067-of-00101.safetensors", + "model.layers.62.mlp.experts.35.gate_proj.weight": "model-00067-of-00101.safetensors", + "model.layers.62.mlp.experts.35.up_proj.weight": "model-00067-of-00101.safetensors", + "model.layers.62.mlp.experts.35.down_proj.weight": "model-00067-of-00101.safetensors", + "model.layers.62.mlp.experts.36.gate_proj.weight": "model-00067-of-00101.safetensors", + "model.layers.62.mlp.experts.36.up_proj.weight": "model-00067-of-00101.safetensors", + "model.layers.62.mlp.experts.36.down_proj.weight": "model-00067-of-00101.safetensors", + "model.layers.62.mlp.experts.37.gate_proj.weight": "model-00067-of-00101.safetensors", + "model.layers.62.mlp.experts.37.up_proj.weight": "model-00067-of-00101.safetensors", + "model.layers.62.mlp.experts.37.down_proj.weight": "model-00067-of-00101.safetensors", + "model.layers.62.mlp.experts.38.gate_proj.weight": "model-00067-of-00101.safetensors", + "model.layers.62.mlp.experts.38.up_proj.weight": "model-00067-of-00101.safetensors", + "model.layers.62.mlp.experts.38.down_proj.weight": "model-00067-of-00101.safetensors", + "model.layers.62.mlp.experts.39.gate_proj.weight": "model-00067-of-00101.safetensors", + "model.layers.62.mlp.experts.39.up_proj.weight": "model-00067-of-00101.safetensors", + "model.layers.62.mlp.experts.39.down_proj.weight": "model-00067-of-00101.safetensors", + "model.layers.62.mlp.experts.40.gate_proj.weight": "model-00067-of-00101.safetensors", + "model.layers.62.mlp.experts.40.up_proj.weight": "model-00067-of-00101.safetensors", + "model.layers.62.mlp.experts.40.down_proj.weight": "model-00067-of-00101.safetensors", + "model.layers.62.mlp.experts.41.gate_proj.weight": "model-00068-of-00101.safetensors", + "model.layers.62.mlp.experts.41.up_proj.weight": "model-00068-of-00101.safetensors", + "model.layers.62.mlp.experts.41.down_proj.weight": "model-00068-of-00101.safetensors", + "model.layers.62.mlp.experts.42.gate_proj.weight": "model-00068-of-00101.safetensors", + "model.layers.62.mlp.experts.42.up_proj.weight": "model-00068-of-00101.safetensors", + "model.layers.62.mlp.experts.42.down_proj.weight": "model-00068-of-00101.safetensors", + "model.layers.62.mlp.experts.43.gate_proj.weight": "model-00068-of-00101.safetensors", + "model.layers.62.mlp.experts.43.up_proj.weight": "model-00068-of-00101.safetensors", + "model.layers.62.mlp.experts.43.down_proj.weight": "model-00068-of-00101.safetensors", + "model.layers.62.mlp.experts.44.gate_proj.weight": "model-00068-of-00101.safetensors", + "model.layers.62.mlp.experts.44.up_proj.weight": "model-00068-of-00101.safetensors", + "model.layers.62.mlp.experts.44.down_proj.weight": "model-00068-of-00101.safetensors", + "model.layers.62.mlp.experts.45.gate_proj.weight": "model-00068-of-00101.safetensors", + "model.layers.62.mlp.experts.45.up_proj.weight": "model-00068-of-00101.safetensors", + "model.layers.62.mlp.experts.45.down_proj.weight": "model-00068-of-00101.safetensors", + "model.layers.62.mlp.experts.46.gate_proj.weight": "model-00068-of-00101.safetensors", + "model.layers.62.mlp.experts.46.up_proj.weight": "model-00068-of-00101.safetensors", + "model.layers.62.mlp.experts.46.down_proj.weight": "model-00068-of-00101.safetensors", + "model.layers.62.mlp.experts.47.gate_proj.weight": "model-00068-of-00101.safetensors", + "model.layers.62.mlp.experts.47.up_proj.weight": "model-00068-of-00101.safetensors", + "model.layers.62.mlp.experts.47.down_proj.weight": "model-00068-of-00101.safetensors", + "model.layers.62.mlp.experts.48.gate_proj.weight": "model-00068-of-00101.safetensors", + "model.layers.62.mlp.experts.48.up_proj.weight": "model-00068-of-00101.safetensors", + "model.layers.62.mlp.experts.48.down_proj.weight": "model-00068-of-00101.safetensors", + "model.layers.62.mlp.experts.49.gate_proj.weight": "model-00068-of-00101.safetensors", + "model.layers.62.mlp.experts.49.up_proj.weight": "model-00068-of-00101.safetensors", + "model.layers.62.mlp.experts.49.down_proj.weight": "model-00068-of-00101.safetensors", + "model.layers.62.mlp.experts.50.gate_proj.weight": "model-00068-of-00101.safetensors", + "model.layers.62.mlp.experts.50.up_proj.weight": "model-00068-of-00101.safetensors", + "model.layers.62.mlp.experts.50.down_proj.weight": "model-00068-of-00101.safetensors", + "model.layers.62.mlp.experts.51.gate_proj.weight": "model-00068-of-00101.safetensors", + "model.layers.62.mlp.experts.51.up_proj.weight": "model-00068-of-00101.safetensors", + "model.layers.62.mlp.experts.51.down_proj.weight": "model-00068-of-00101.safetensors", + "model.layers.62.mlp.experts.52.gate_proj.weight": "model-00068-of-00101.safetensors", + "model.layers.62.mlp.experts.52.up_proj.weight": "model-00068-of-00101.safetensors", + "model.layers.62.mlp.experts.52.down_proj.weight": "model-00068-of-00101.safetensors", + "model.layers.62.mlp.experts.53.gate_proj.weight": "model-00068-of-00101.safetensors", + "model.layers.62.mlp.experts.53.up_proj.weight": "model-00068-of-00101.safetensors", + "model.layers.62.mlp.experts.53.down_proj.weight": "model-00068-of-00101.safetensors", + "model.layers.62.mlp.experts.54.gate_proj.weight": "model-00068-of-00101.safetensors", + "model.layers.62.mlp.experts.54.up_proj.weight": "model-00068-of-00101.safetensors", + "model.layers.62.mlp.experts.54.down_proj.weight": "model-00068-of-00101.safetensors", + "model.layers.62.mlp.experts.55.gate_proj.weight": "model-00068-of-00101.safetensors", + "model.layers.62.mlp.experts.55.up_proj.weight": "model-00068-of-00101.safetensors", + "model.layers.62.mlp.experts.55.down_proj.weight": "model-00068-of-00101.safetensors", + "model.layers.62.mlp.experts.56.gate_proj.weight": "model-00068-of-00101.safetensors", + "model.layers.62.mlp.experts.56.up_proj.weight": "model-00068-of-00101.safetensors", + "model.layers.62.mlp.experts.56.down_proj.weight": "model-00068-of-00101.safetensors", + "model.layers.62.mlp.experts.57.gate_proj.weight": "model-00068-of-00101.safetensors", + "model.layers.62.mlp.experts.57.up_proj.weight": "model-00068-of-00101.safetensors", + "model.layers.62.mlp.experts.57.down_proj.weight": "model-00068-of-00101.safetensors", + "model.layers.62.mlp.experts.58.gate_proj.weight": "model-00068-of-00101.safetensors", + "model.layers.62.mlp.experts.58.up_proj.weight": "model-00068-of-00101.safetensors", + "model.layers.62.mlp.experts.58.down_proj.weight": "model-00068-of-00101.safetensors", + "model.layers.62.mlp.experts.59.gate_proj.weight": "model-00068-of-00101.safetensors", + "model.layers.62.mlp.experts.59.up_proj.weight": "model-00068-of-00101.safetensors", + "model.layers.62.mlp.experts.59.down_proj.weight": "model-00068-of-00101.safetensors", + "model.layers.62.mlp.experts.60.gate_proj.weight": "model-00068-of-00101.safetensors", + "model.layers.62.mlp.experts.60.up_proj.weight": "model-00068-of-00101.safetensors", + "model.layers.62.mlp.experts.60.down_proj.weight": "model-00068-of-00101.safetensors", + "model.layers.62.mlp.experts.61.gate_proj.weight": "model-00068-of-00101.safetensors", + "model.layers.62.mlp.experts.61.up_proj.weight": "model-00068-of-00101.safetensors", + "model.layers.62.mlp.experts.61.down_proj.weight": "model-00068-of-00101.safetensors", + "model.layers.62.mlp.experts.62.gate_proj.weight": "model-00068-of-00101.safetensors", + "model.layers.62.mlp.experts.62.up_proj.weight": "model-00068-of-00101.safetensors", + "model.layers.62.mlp.experts.62.down_proj.weight": "model-00068-of-00101.safetensors", + "model.layers.62.mlp.experts.63.gate_proj.weight": "model-00068-of-00101.safetensors", + "model.layers.62.mlp.experts.63.up_proj.weight": "model-00068-of-00101.safetensors", + "model.layers.62.mlp.experts.63.down_proj.weight": "model-00068-of-00101.safetensors", + "model.layers.62.mlp.experts.64.gate_proj.weight": "model-00068-of-00101.safetensors", + "model.layers.62.mlp.experts.64.up_proj.weight": "model-00068-of-00101.safetensors", + "model.layers.62.mlp.experts.64.down_proj.weight": "model-00068-of-00101.safetensors", + "model.layers.62.mlp.experts.65.gate_proj.weight": "model-00068-of-00101.safetensors", + "model.layers.62.mlp.experts.65.up_proj.weight": "model-00068-of-00101.safetensors", + "model.layers.62.mlp.experts.65.down_proj.weight": "model-00068-of-00101.safetensors", + "model.layers.62.mlp.experts.66.gate_proj.weight": "model-00068-of-00101.safetensors", + "model.layers.62.mlp.experts.66.up_proj.weight": "model-00068-of-00101.safetensors", + "model.layers.62.mlp.experts.66.down_proj.weight": "model-00068-of-00101.safetensors", + "model.layers.62.mlp.experts.67.gate_proj.weight": "model-00068-of-00101.safetensors", + "model.layers.62.mlp.experts.67.up_proj.weight": "model-00068-of-00101.safetensors", + "model.layers.62.mlp.experts.67.down_proj.weight": "model-00068-of-00101.safetensors", + "model.layers.62.mlp.experts.68.gate_proj.weight": "model-00068-of-00101.safetensors", + "model.layers.62.mlp.experts.68.up_proj.weight": "model-00068-of-00101.safetensors", + "model.layers.62.mlp.experts.68.down_proj.weight": "model-00068-of-00101.safetensors", + "model.layers.62.mlp.experts.69.gate_proj.weight": "model-00068-of-00101.safetensors", + "model.layers.62.mlp.experts.69.up_proj.weight": "model-00068-of-00101.safetensors", + "model.layers.62.mlp.experts.69.down_proj.weight": "model-00068-of-00101.safetensors", + "model.layers.62.mlp.experts.70.gate_proj.weight": "model-00068-of-00101.safetensors", + "model.layers.62.mlp.experts.70.up_proj.weight": "model-00068-of-00101.safetensors", + "model.layers.62.mlp.experts.70.down_proj.weight": "model-00068-of-00101.safetensors", + "model.layers.62.mlp.experts.71.gate_proj.weight": "model-00068-of-00101.safetensors", + "model.layers.62.mlp.experts.71.up_proj.weight": "model-00068-of-00101.safetensors", + "model.layers.62.mlp.experts.71.down_proj.weight": "model-00068-of-00101.safetensors", + "model.layers.62.mlp.experts.72.gate_proj.weight": "model-00068-of-00101.safetensors", + "model.layers.62.mlp.experts.72.up_proj.weight": "model-00068-of-00101.safetensors", + "model.layers.62.mlp.experts.72.down_proj.weight": "model-00068-of-00101.safetensors", + "model.layers.62.mlp.experts.73.gate_proj.weight": "model-00068-of-00101.safetensors", + "model.layers.62.mlp.experts.73.up_proj.weight": "model-00068-of-00101.safetensors", + "model.layers.62.mlp.experts.73.down_proj.weight": "model-00068-of-00101.safetensors", + "model.layers.62.mlp.experts.74.gate_proj.weight": "model-00068-of-00101.safetensors", + "model.layers.62.mlp.experts.74.up_proj.weight": "model-00068-of-00101.safetensors", + "model.layers.62.mlp.experts.74.down_proj.weight": "model-00068-of-00101.safetensors", + "model.layers.62.mlp.experts.75.gate_proj.weight": "model-00068-of-00101.safetensors", + "model.layers.62.mlp.experts.75.up_proj.weight": "model-00068-of-00101.safetensors", + "model.layers.62.mlp.experts.75.down_proj.weight": "model-00068-of-00101.safetensors", + "model.layers.62.mlp.experts.76.gate_proj.weight": "model-00068-of-00101.safetensors", + "model.layers.62.mlp.experts.76.up_proj.weight": "model-00068-of-00101.safetensors", + "model.layers.62.mlp.experts.76.down_proj.weight": "model-00068-of-00101.safetensors", + "model.layers.62.mlp.experts.77.gate_proj.weight": "model-00068-of-00101.safetensors", + "model.layers.62.mlp.experts.77.up_proj.weight": "model-00068-of-00101.safetensors", + "model.layers.62.mlp.experts.77.down_proj.weight": "model-00068-of-00101.safetensors", + "model.layers.62.mlp.experts.78.gate_proj.weight": "model-00068-of-00101.safetensors", + "model.layers.62.mlp.experts.78.up_proj.weight": "model-00068-of-00101.safetensors", + "model.layers.62.mlp.experts.78.down_proj.weight": "model-00068-of-00101.safetensors", + "model.layers.62.mlp.experts.79.gate_proj.weight": "model-00068-of-00101.safetensors", + "model.layers.62.mlp.experts.79.up_proj.weight": "model-00068-of-00101.safetensors", + "model.layers.62.mlp.experts.79.down_proj.weight": "model-00068-of-00101.safetensors", + "model.layers.62.mlp.experts.80.gate_proj.weight": "model-00068-of-00101.safetensors", + "model.layers.62.mlp.experts.80.up_proj.weight": "model-00068-of-00101.safetensors", + "model.layers.62.mlp.experts.80.down_proj.weight": "model-00068-of-00101.safetensors", + "model.layers.62.mlp.experts.81.gate_proj.weight": "model-00068-of-00101.safetensors", + "model.layers.62.mlp.experts.81.up_proj.weight": "model-00068-of-00101.safetensors", + "model.layers.62.mlp.experts.81.down_proj.weight": "model-00068-of-00101.safetensors", + "model.layers.62.mlp.experts.82.gate_proj.weight": "model-00068-of-00101.safetensors", + "model.layers.62.mlp.experts.82.up_proj.weight": "model-00068-of-00101.safetensors", + "model.layers.62.mlp.experts.82.down_proj.weight": "model-00068-of-00101.safetensors", + "model.layers.62.mlp.experts.83.gate_proj.weight": "model-00068-of-00101.safetensors", + "model.layers.62.mlp.experts.83.up_proj.weight": "model-00068-of-00101.safetensors", + "model.layers.62.mlp.experts.83.down_proj.weight": "model-00068-of-00101.safetensors", + "model.layers.62.mlp.experts.84.gate_proj.weight": "model-00068-of-00101.safetensors", + "model.layers.62.mlp.experts.84.up_proj.weight": "model-00068-of-00101.safetensors", + "model.layers.62.mlp.experts.84.down_proj.weight": "model-00068-of-00101.safetensors", + "model.layers.62.mlp.experts.85.gate_proj.weight": "model-00068-of-00101.safetensors", + "model.layers.62.mlp.experts.85.up_proj.weight": "model-00068-of-00101.safetensors", + "model.layers.62.mlp.experts.85.down_proj.weight": "model-00068-of-00101.safetensors", + "model.layers.62.mlp.experts.86.gate_proj.weight": "model-00068-of-00101.safetensors", + "model.layers.62.mlp.experts.86.up_proj.weight": "model-00068-of-00101.safetensors", + "model.layers.62.mlp.experts.86.down_proj.weight": "model-00068-of-00101.safetensors", + "model.layers.62.mlp.experts.87.gate_proj.weight": "model-00068-of-00101.safetensors", + "model.layers.62.mlp.experts.87.up_proj.weight": "model-00068-of-00101.safetensors", + "model.layers.62.mlp.experts.87.down_proj.weight": "model-00068-of-00101.safetensors", + "model.layers.62.mlp.experts.88.gate_proj.weight": "model-00068-of-00101.safetensors", + "model.layers.62.mlp.experts.88.up_proj.weight": "model-00068-of-00101.safetensors", + "model.layers.62.mlp.experts.88.down_proj.weight": "model-00068-of-00101.safetensors", + "model.layers.62.mlp.experts.89.gate_proj.weight": "model-00068-of-00101.safetensors", + "model.layers.62.mlp.experts.89.up_proj.weight": "model-00068-of-00101.safetensors", + "model.layers.62.mlp.experts.89.down_proj.weight": "model-00068-of-00101.safetensors", + "model.layers.62.mlp.experts.90.gate_proj.weight": "model-00068-of-00101.safetensors", + "model.layers.62.mlp.experts.90.up_proj.weight": "model-00068-of-00101.safetensors", + "model.layers.62.mlp.experts.90.down_proj.weight": "model-00068-of-00101.safetensors", + "model.layers.62.mlp.experts.91.gate_proj.weight": "model-00068-of-00101.safetensors", + "model.layers.62.mlp.experts.91.up_proj.weight": "model-00068-of-00101.safetensors", + "model.layers.62.mlp.experts.91.down_proj.weight": "model-00068-of-00101.safetensors", + "model.layers.62.mlp.experts.92.gate_proj.weight": "model-00068-of-00101.safetensors", + "model.layers.62.mlp.experts.92.up_proj.weight": "model-00068-of-00101.safetensors", + "model.layers.62.mlp.experts.92.down_proj.weight": "model-00068-of-00101.safetensors", + "model.layers.62.mlp.experts.93.gate_proj.weight": "model-00068-of-00101.safetensors", + "model.layers.62.mlp.experts.93.up_proj.weight": "model-00068-of-00101.safetensors", + "model.layers.62.mlp.experts.93.down_proj.weight": "model-00068-of-00101.safetensors", + "model.layers.62.mlp.experts.94.gate_proj.weight": "model-00068-of-00101.safetensors", + "model.layers.62.mlp.experts.94.up_proj.weight": "model-00068-of-00101.safetensors", + "model.layers.62.mlp.experts.94.down_proj.weight": "model-00068-of-00101.safetensors", + "model.layers.62.mlp.experts.95.gate_proj.weight": "model-00068-of-00101.safetensors", + "model.layers.62.mlp.experts.95.up_proj.weight": "model-00068-of-00101.safetensors", + "model.layers.62.mlp.experts.95.down_proj.weight": "model-00068-of-00101.safetensors", + "model.layers.62.mlp.experts.96.gate_proj.weight": "model-00068-of-00101.safetensors", + "model.layers.62.mlp.experts.96.up_proj.weight": "model-00068-of-00101.safetensors", + "model.layers.62.mlp.experts.96.down_proj.weight": "model-00068-of-00101.safetensors", + "model.layers.62.mlp.experts.97.gate_proj.weight": "model-00068-of-00101.safetensors", + "model.layers.62.mlp.experts.97.up_proj.weight": "model-00068-of-00101.safetensors", + "model.layers.62.mlp.experts.97.down_proj.weight": "model-00068-of-00101.safetensors", + "model.layers.62.mlp.experts.98.gate_proj.weight": "model-00068-of-00101.safetensors", + "model.layers.62.mlp.experts.98.up_proj.weight": "model-00068-of-00101.safetensors", + "model.layers.62.mlp.experts.98.down_proj.weight": "model-00068-of-00101.safetensors", + "model.layers.62.mlp.experts.99.gate_proj.weight": "model-00068-of-00101.safetensors", + "model.layers.62.mlp.experts.99.up_proj.weight": "model-00068-of-00101.safetensors", + "model.layers.62.mlp.experts.99.down_proj.weight": "model-00068-of-00101.safetensors", + "model.layers.62.mlp.experts.100.gate_proj.weight": "model-00068-of-00101.safetensors", + "model.layers.62.mlp.experts.100.up_proj.weight": "model-00068-of-00101.safetensors", + "model.layers.62.mlp.experts.100.down_proj.weight": "model-00068-of-00101.safetensors", + "model.layers.62.mlp.experts.101.gate_proj.weight": "model-00068-of-00101.safetensors", + "model.layers.62.mlp.experts.101.up_proj.weight": "model-00068-of-00101.safetensors", + "model.layers.62.mlp.experts.101.down_proj.weight": "model-00068-of-00101.safetensors", + "model.layers.62.mlp.experts.102.gate_proj.weight": "model-00068-of-00101.safetensors", + "model.layers.62.mlp.experts.102.up_proj.weight": "model-00068-of-00101.safetensors", + "model.layers.62.mlp.experts.102.down_proj.weight": "model-00068-of-00101.safetensors", + "model.layers.62.mlp.experts.103.gate_proj.weight": "model-00068-of-00101.safetensors", + "model.layers.62.mlp.experts.103.up_proj.weight": "model-00068-of-00101.safetensors", + "model.layers.62.mlp.experts.103.down_proj.weight": "model-00068-of-00101.safetensors", + "model.layers.62.mlp.experts.104.gate_proj.weight": "model-00068-of-00101.safetensors", + "model.layers.62.mlp.experts.104.up_proj.weight": "model-00068-of-00101.safetensors", + "model.layers.62.mlp.experts.104.down_proj.weight": "model-00068-of-00101.safetensors", + "model.layers.62.mlp.experts.105.gate_proj.weight": "model-00068-of-00101.safetensors", + "model.layers.62.mlp.experts.105.up_proj.weight": "model-00068-of-00101.safetensors", + "model.layers.62.mlp.experts.105.down_proj.weight": "model-00068-of-00101.safetensors", + "model.layers.62.mlp.experts.106.gate_proj.weight": "model-00068-of-00101.safetensors", + "model.layers.62.mlp.experts.106.up_proj.weight": "model-00068-of-00101.safetensors", + "model.layers.62.mlp.experts.106.down_proj.weight": "model-00068-of-00101.safetensors", + "model.layers.62.mlp.experts.107.gate_proj.weight": "model-00068-of-00101.safetensors", + "model.layers.62.mlp.experts.107.up_proj.weight": "model-00068-of-00101.safetensors", + "model.layers.62.mlp.experts.107.down_proj.weight": "model-00068-of-00101.safetensors", + "model.layers.62.mlp.experts.108.gate_proj.weight": "model-00068-of-00101.safetensors", + "model.layers.62.mlp.experts.108.up_proj.weight": "model-00068-of-00101.safetensors", + "model.layers.62.mlp.experts.108.down_proj.weight": "model-00068-of-00101.safetensors", + "model.layers.62.mlp.experts.109.gate_proj.weight": "model-00068-of-00101.safetensors", + "model.layers.62.mlp.experts.109.up_proj.weight": "model-00068-of-00101.safetensors", + "model.layers.62.mlp.experts.109.down_proj.weight": "model-00068-of-00101.safetensors", + "model.layers.62.mlp.experts.110.gate_proj.weight": "model-00068-of-00101.safetensors", + "model.layers.62.mlp.experts.110.up_proj.weight": "model-00068-of-00101.safetensors", + "model.layers.62.mlp.experts.110.down_proj.weight": "model-00068-of-00101.safetensors", + "model.layers.62.mlp.experts.111.gate_proj.weight": "model-00068-of-00101.safetensors", + "model.layers.62.mlp.experts.111.up_proj.weight": "model-00068-of-00101.safetensors", + "model.layers.62.mlp.experts.111.down_proj.weight": "model-00068-of-00101.safetensors", + "model.layers.62.mlp.experts.112.gate_proj.weight": "model-00068-of-00101.safetensors", + "model.layers.62.mlp.experts.112.up_proj.weight": "model-00068-of-00101.safetensors", + "model.layers.62.mlp.experts.112.down_proj.weight": "model-00068-of-00101.safetensors", + "model.layers.62.mlp.experts.113.gate_proj.weight": "model-00068-of-00101.safetensors", + "model.layers.62.mlp.experts.113.up_proj.weight": "model-00068-of-00101.safetensors", + "model.layers.62.mlp.experts.113.down_proj.weight": "model-00068-of-00101.safetensors", + "model.layers.62.mlp.experts.114.gate_proj.weight": "model-00068-of-00101.safetensors", + "model.layers.62.mlp.experts.114.up_proj.weight": "model-00068-of-00101.safetensors", + "model.layers.62.mlp.experts.114.down_proj.weight": "model-00068-of-00101.safetensors", + "model.layers.62.mlp.experts.115.gate_proj.weight": "model-00068-of-00101.safetensors", + "model.layers.62.mlp.experts.115.up_proj.weight": "model-00068-of-00101.safetensors", + "model.layers.62.mlp.experts.115.down_proj.weight": "model-00068-of-00101.safetensors", + "model.layers.62.mlp.experts.116.gate_proj.weight": "model-00068-of-00101.safetensors", + "model.layers.62.mlp.experts.116.up_proj.weight": "model-00068-of-00101.safetensors", + "model.layers.62.mlp.experts.116.down_proj.weight": "model-00068-of-00101.safetensors", + "model.layers.62.mlp.experts.117.gate_proj.weight": "model-00068-of-00101.safetensors", + "model.layers.62.mlp.experts.117.up_proj.weight": "model-00068-of-00101.safetensors", + "model.layers.62.mlp.experts.117.down_proj.weight": "model-00068-of-00101.safetensors", + "model.layers.62.mlp.experts.118.gate_proj.weight": "model-00068-of-00101.safetensors", + "model.layers.62.mlp.experts.118.up_proj.weight": "model-00068-of-00101.safetensors", + "model.layers.62.mlp.experts.118.down_proj.weight": "model-00068-of-00101.safetensors", + "model.layers.62.mlp.experts.119.gate_proj.weight": "model-00068-of-00101.safetensors", + "model.layers.62.mlp.experts.119.up_proj.weight": "model-00068-of-00101.safetensors", + "model.layers.62.mlp.experts.119.down_proj.weight": "model-00068-of-00101.safetensors", + "model.layers.62.mlp.gate.weight": "model-00068-of-00101.safetensors", + "model.layers.62.mlp.gate.e_score_correction_bias": "model-00068-of-00101.safetensors", + "model.layers.62.mlp.shared_experts.gate_proj.weight": "model-00068-of-00101.safetensors", + "model.layers.62.mlp.shared_experts.up_proj.weight": "model-00068-of-00101.safetensors", + "model.layers.62.mlp.shared_experts.down_proj.weight": "model-00068-of-00101.safetensors", + "model.layers.62.input_layernorm.weight": "model-00068-of-00101.safetensors", + "model.layers.62.post_attention_layernorm.weight": "model-00068-of-00101.safetensors", + "model.layers.63.self_attn.q_proj.weight": "model-00068-of-00101.safetensors", + "model.layers.63.self_attn.q_proj.bias": "model-00068-of-00101.safetensors", + "model.layers.63.self_attn.k_proj.weight": "model-00068-of-00101.safetensors", + "model.layers.63.self_attn.k_proj.bias": "model-00068-of-00101.safetensors", + "model.layers.63.self_attn.v_proj.weight": "model-00068-of-00101.safetensors", + "model.layers.63.self_attn.v_proj.bias": "model-00068-of-00101.safetensors", + "model.layers.63.self_attn.o_proj.weight": "model-00068-of-00101.safetensors", + "model.layers.63.self_attn.q_norm.weight": "model-00068-of-00101.safetensors", + "model.layers.63.self_attn.k_norm.weight": "model-00068-of-00101.safetensors", + "model.layers.63.mlp.experts.0.gate_proj.weight": "model-00068-of-00101.safetensors", + "model.layers.63.mlp.experts.0.up_proj.weight": "model-00068-of-00101.safetensors", + "model.layers.63.mlp.experts.0.down_proj.weight": "model-00068-of-00101.safetensors", + "model.layers.63.mlp.experts.1.gate_proj.weight": "model-00068-of-00101.safetensors", + "model.layers.63.mlp.experts.1.up_proj.weight": "model-00068-of-00101.safetensors", + "model.layers.63.mlp.experts.1.down_proj.weight": "model-00068-of-00101.safetensors", + "model.layers.63.mlp.experts.2.gate_proj.weight": "model-00068-of-00101.safetensors", + "model.layers.63.mlp.experts.2.up_proj.weight": "model-00068-of-00101.safetensors", + "model.layers.63.mlp.experts.2.down_proj.weight": "model-00068-of-00101.safetensors", + "model.layers.63.mlp.experts.3.gate_proj.weight": "model-00068-of-00101.safetensors", + "model.layers.63.mlp.experts.3.up_proj.weight": "model-00068-of-00101.safetensors", + "model.layers.63.mlp.experts.3.down_proj.weight": "model-00068-of-00101.safetensors", + "model.layers.63.mlp.experts.4.gate_proj.weight": "model-00068-of-00101.safetensors", + "model.layers.63.mlp.experts.4.up_proj.weight": "model-00068-of-00101.safetensors", + "model.layers.63.mlp.experts.4.down_proj.weight": "model-00068-of-00101.safetensors", + "model.layers.63.mlp.experts.5.gate_proj.weight": "model-00068-of-00101.safetensors", + "model.layers.63.mlp.experts.5.up_proj.weight": "model-00068-of-00101.safetensors", + "model.layers.63.mlp.experts.5.down_proj.weight": "model-00068-of-00101.safetensors", + "model.layers.63.mlp.experts.6.gate_proj.weight": "model-00068-of-00101.safetensors", + "model.layers.63.mlp.experts.6.up_proj.weight": "model-00068-of-00101.safetensors", + "model.layers.63.mlp.experts.6.down_proj.weight": "model-00068-of-00101.safetensors", + "model.layers.63.mlp.experts.7.gate_proj.weight": "model-00068-of-00101.safetensors", + "model.layers.63.mlp.experts.7.up_proj.weight": "model-00068-of-00101.safetensors", + "model.layers.63.mlp.experts.7.down_proj.weight": "model-00068-of-00101.safetensors", + "model.layers.63.mlp.experts.8.gate_proj.weight": "model-00068-of-00101.safetensors", + "model.layers.63.mlp.experts.8.up_proj.weight": "model-00068-of-00101.safetensors", + "model.layers.63.mlp.experts.8.down_proj.weight": "model-00068-of-00101.safetensors", + "model.layers.63.mlp.experts.9.gate_proj.weight": "model-00068-of-00101.safetensors", + "model.layers.63.mlp.experts.9.up_proj.weight": "model-00068-of-00101.safetensors", + "model.layers.63.mlp.experts.9.down_proj.weight": "model-00068-of-00101.safetensors", + "model.layers.63.mlp.experts.10.gate_proj.weight": "model-00068-of-00101.safetensors", + "model.layers.63.mlp.experts.10.up_proj.weight": "model-00068-of-00101.safetensors", + "model.layers.63.mlp.experts.10.down_proj.weight": "model-00068-of-00101.safetensors", + "model.layers.63.mlp.experts.11.gate_proj.weight": "model-00068-of-00101.safetensors", + "model.layers.63.mlp.experts.11.up_proj.weight": "model-00068-of-00101.safetensors", + "model.layers.63.mlp.experts.11.down_proj.weight": "model-00068-of-00101.safetensors", + "model.layers.63.mlp.experts.12.gate_proj.weight": "model-00068-of-00101.safetensors", + "model.layers.63.mlp.experts.12.up_proj.weight": "model-00068-of-00101.safetensors", + "model.layers.63.mlp.experts.12.down_proj.weight": "model-00068-of-00101.safetensors", + "model.layers.63.mlp.experts.13.gate_proj.weight": "model-00068-of-00101.safetensors", + "model.layers.63.mlp.experts.13.up_proj.weight": "model-00068-of-00101.safetensors", + "model.layers.63.mlp.experts.13.down_proj.weight": "model-00068-of-00101.safetensors", + "model.layers.63.mlp.experts.14.gate_proj.weight": "model-00068-of-00101.safetensors", + "model.layers.63.mlp.experts.14.up_proj.weight": "model-00068-of-00101.safetensors", + "model.layers.63.mlp.experts.14.down_proj.weight": "model-00068-of-00101.safetensors", + "model.layers.63.mlp.experts.15.gate_proj.weight": "model-00068-of-00101.safetensors", + "model.layers.63.mlp.experts.15.up_proj.weight": "model-00068-of-00101.safetensors", + "model.layers.63.mlp.experts.15.down_proj.weight": "model-00068-of-00101.safetensors", + "model.layers.63.mlp.experts.16.gate_proj.weight": "model-00068-of-00101.safetensors", + "model.layers.63.mlp.experts.16.up_proj.weight": "model-00068-of-00101.safetensors", + "model.layers.63.mlp.experts.16.down_proj.weight": "model-00068-of-00101.safetensors", + "model.layers.63.mlp.experts.17.gate_proj.weight": "model-00068-of-00101.safetensors", + "model.layers.63.mlp.experts.17.up_proj.weight": "model-00068-of-00101.safetensors", + "model.layers.63.mlp.experts.17.down_proj.weight": "model-00068-of-00101.safetensors", + "model.layers.63.mlp.experts.18.gate_proj.weight": "model-00068-of-00101.safetensors", + "model.layers.63.mlp.experts.18.up_proj.weight": "model-00068-of-00101.safetensors", + "model.layers.63.mlp.experts.18.down_proj.weight": "model-00068-of-00101.safetensors", + "model.layers.63.mlp.experts.19.gate_proj.weight": "model-00068-of-00101.safetensors", + "model.layers.63.mlp.experts.19.up_proj.weight": "model-00068-of-00101.safetensors", + "model.layers.63.mlp.experts.19.down_proj.weight": "model-00068-of-00101.safetensors", + "model.layers.63.mlp.experts.20.gate_proj.weight": "model-00068-of-00101.safetensors", + "model.layers.63.mlp.experts.20.up_proj.weight": "model-00068-of-00101.safetensors", + "model.layers.63.mlp.experts.20.down_proj.weight": "model-00068-of-00101.safetensors", + "model.layers.63.mlp.experts.21.gate_proj.weight": "model-00068-of-00101.safetensors", + "model.layers.63.mlp.experts.21.up_proj.weight": "model-00068-of-00101.safetensors", + "model.layers.63.mlp.experts.21.down_proj.weight": "model-00068-of-00101.safetensors", + "model.layers.63.mlp.experts.22.gate_proj.weight": "model-00068-of-00101.safetensors", + "model.layers.63.mlp.experts.22.up_proj.weight": "model-00068-of-00101.safetensors", + "model.layers.63.mlp.experts.22.down_proj.weight": "model-00068-of-00101.safetensors", + "model.layers.63.mlp.experts.23.gate_proj.weight": "model-00068-of-00101.safetensors", + "model.layers.63.mlp.experts.23.up_proj.weight": "model-00068-of-00101.safetensors", + "model.layers.63.mlp.experts.23.down_proj.weight": "model-00068-of-00101.safetensors", + "model.layers.63.mlp.experts.24.gate_proj.weight": "model-00068-of-00101.safetensors", + "model.layers.63.mlp.experts.24.up_proj.weight": "model-00068-of-00101.safetensors", + "model.layers.63.mlp.experts.24.down_proj.weight": "model-00068-of-00101.safetensors", + "model.layers.63.mlp.experts.25.gate_proj.weight": "model-00068-of-00101.safetensors", + "model.layers.63.mlp.experts.25.up_proj.weight": "model-00068-of-00101.safetensors", + "model.layers.63.mlp.experts.25.down_proj.weight": "model-00068-of-00101.safetensors", + "model.layers.63.mlp.experts.26.gate_proj.weight": "model-00068-of-00101.safetensors", + "model.layers.63.mlp.experts.26.up_proj.weight": "model-00068-of-00101.safetensors", + "model.layers.63.mlp.experts.26.down_proj.weight": "model-00068-of-00101.safetensors", + "model.layers.63.mlp.experts.27.gate_proj.weight": "model-00068-of-00101.safetensors", + "model.layers.63.mlp.experts.27.up_proj.weight": "model-00068-of-00101.safetensors", + "model.layers.63.mlp.experts.27.down_proj.weight": "model-00069-of-00101.safetensors", + "model.layers.63.mlp.experts.28.gate_proj.weight": "model-00069-of-00101.safetensors", + "model.layers.63.mlp.experts.28.up_proj.weight": "model-00069-of-00101.safetensors", + "model.layers.63.mlp.experts.28.down_proj.weight": "model-00069-of-00101.safetensors", + "model.layers.63.mlp.experts.29.gate_proj.weight": "model-00069-of-00101.safetensors", + "model.layers.63.mlp.experts.29.up_proj.weight": "model-00069-of-00101.safetensors", + "model.layers.63.mlp.experts.29.down_proj.weight": "model-00069-of-00101.safetensors", + "model.layers.63.mlp.experts.30.gate_proj.weight": "model-00069-of-00101.safetensors", + "model.layers.63.mlp.experts.30.up_proj.weight": "model-00069-of-00101.safetensors", + "model.layers.63.mlp.experts.30.down_proj.weight": "model-00069-of-00101.safetensors", + "model.layers.63.mlp.experts.31.gate_proj.weight": "model-00069-of-00101.safetensors", + "model.layers.63.mlp.experts.31.up_proj.weight": "model-00069-of-00101.safetensors", + "model.layers.63.mlp.experts.31.down_proj.weight": "model-00069-of-00101.safetensors", + "model.layers.63.mlp.experts.32.gate_proj.weight": "model-00069-of-00101.safetensors", + "model.layers.63.mlp.experts.32.up_proj.weight": "model-00069-of-00101.safetensors", + "model.layers.63.mlp.experts.32.down_proj.weight": "model-00069-of-00101.safetensors", + "model.layers.63.mlp.experts.33.gate_proj.weight": "model-00069-of-00101.safetensors", + "model.layers.63.mlp.experts.33.up_proj.weight": "model-00069-of-00101.safetensors", + "model.layers.63.mlp.experts.33.down_proj.weight": "model-00069-of-00101.safetensors", + "model.layers.63.mlp.experts.34.gate_proj.weight": "model-00069-of-00101.safetensors", + "model.layers.63.mlp.experts.34.up_proj.weight": "model-00069-of-00101.safetensors", + "model.layers.63.mlp.experts.34.down_proj.weight": "model-00069-of-00101.safetensors", + "model.layers.63.mlp.experts.35.gate_proj.weight": "model-00069-of-00101.safetensors", + "model.layers.63.mlp.experts.35.up_proj.weight": "model-00069-of-00101.safetensors", + "model.layers.63.mlp.experts.35.down_proj.weight": "model-00069-of-00101.safetensors", + "model.layers.63.mlp.experts.36.gate_proj.weight": "model-00069-of-00101.safetensors", + "model.layers.63.mlp.experts.36.up_proj.weight": "model-00069-of-00101.safetensors", + "model.layers.63.mlp.experts.36.down_proj.weight": "model-00069-of-00101.safetensors", + "model.layers.63.mlp.experts.37.gate_proj.weight": "model-00069-of-00101.safetensors", + "model.layers.63.mlp.experts.37.up_proj.weight": "model-00069-of-00101.safetensors", + "model.layers.63.mlp.experts.37.down_proj.weight": "model-00069-of-00101.safetensors", + "model.layers.63.mlp.experts.38.gate_proj.weight": "model-00069-of-00101.safetensors", + "model.layers.63.mlp.experts.38.up_proj.weight": "model-00069-of-00101.safetensors", + "model.layers.63.mlp.experts.38.down_proj.weight": "model-00069-of-00101.safetensors", + "model.layers.63.mlp.experts.39.gate_proj.weight": "model-00069-of-00101.safetensors", + "model.layers.63.mlp.experts.39.up_proj.weight": "model-00069-of-00101.safetensors", + "model.layers.63.mlp.experts.39.down_proj.weight": "model-00069-of-00101.safetensors", + "model.layers.63.mlp.experts.40.gate_proj.weight": "model-00069-of-00101.safetensors", + "model.layers.63.mlp.experts.40.up_proj.weight": "model-00069-of-00101.safetensors", + "model.layers.63.mlp.experts.40.down_proj.weight": "model-00069-of-00101.safetensors", + "model.layers.63.mlp.experts.41.gate_proj.weight": "model-00069-of-00101.safetensors", + "model.layers.63.mlp.experts.41.up_proj.weight": "model-00069-of-00101.safetensors", + "model.layers.63.mlp.experts.41.down_proj.weight": "model-00069-of-00101.safetensors", + "model.layers.63.mlp.experts.42.gate_proj.weight": "model-00069-of-00101.safetensors", + "model.layers.63.mlp.experts.42.up_proj.weight": "model-00069-of-00101.safetensors", + "model.layers.63.mlp.experts.42.down_proj.weight": "model-00069-of-00101.safetensors", + "model.layers.63.mlp.experts.43.gate_proj.weight": "model-00069-of-00101.safetensors", + "model.layers.63.mlp.experts.43.up_proj.weight": "model-00069-of-00101.safetensors", + "model.layers.63.mlp.experts.43.down_proj.weight": "model-00069-of-00101.safetensors", + "model.layers.63.mlp.experts.44.gate_proj.weight": "model-00069-of-00101.safetensors", + "model.layers.63.mlp.experts.44.up_proj.weight": "model-00069-of-00101.safetensors", + "model.layers.63.mlp.experts.44.down_proj.weight": "model-00069-of-00101.safetensors", + "model.layers.63.mlp.experts.45.gate_proj.weight": "model-00069-of-00101.safetensors", + "model.layers.63.mlp.experts.45.up_proj.weight": "model-00069-of-00101.safetensors", + "model.layers.63.mlp.experts.45.down_proj.weight": "model-00069-of-00101.safetensors", + "model.layers.63.mlp.experts.46.gate_proj.weight": "model-00069-of-00101.safetensors", + "model.layers.63.mlp.experts.46.up_proj.weight": "model-00069-of-00101.safetensors", + "model.layers.63.mlp.experts.46.down_proj.weight": "model-00069-of-00101.safetensors", + "model.layers.63.mlp.experts.47.gate_proj.weight": "model-00069-of-00101.safetensors", + "model.layers.63.mlp.experts.47.up_proj.weight": "model-00069-of-00101.safetensors", + "model.layers.63.mlp.experts.47.down_proj.weight": "model-00069-of-00101.safetensors", + "model.layers.63.mlp.experts.48.gate_proj.weight": "model-00069-of-00101.safetensors", + "model.layers.63.mlp.experts.48.up_proj.weight": "model-00069-of-00101.safetensors", + "model.layers.63.mlp.experts.48.down_proj.weight": "model-00069-of-00101.safetensors", + "model.layers.63.mlp.experts.49.gate_proj.weight": "model-00069-of-00101.safetensors", + "model.layers.63.mlp.experts.49.up_proj.weight": "model-00069-of-00101.safetensors", + "model.layers.63.mlp.experts.49.down_proj.weight": "model-00069-of-00101.safetensors", + "model.layers.63.mlp.experts.50.gate_proj.weight": "model-00069-of-00101.safetensors", + "model.layers.63.mlp.experts.50.up_proj.weight": "model-00069-of-00101.safetensors", + "model.layers.63.mlp.experts.50.down_proj.weight": "model-00069-of-00101.safetensors", + "model.layers.63.mlp.experts.51.gate_proj.weight": "model-00069-of-00101.safetensors", + "model.layers.63.mlp.experts.51.up_proj.weight": "model-00069-of-00101.safetensors", + "model.layers.63.mlp.experts.51.down_proj.weight": "model-00069-of-00101.safetensors", + "model.layers.63.mlp.experts.52.gate_proj.weight": "model-00069-of-00101.safetensors", + "model.layers.63.mlp.experts.52.up_proj.weight": "model-00069-of-00101.safetensors", + "model.layers.63.mlp.experts.52.down_proj.weight": "model-00069-of-00101.safetensors", + "model.layers.63.mlp.experts.53.gate_proj.weight": "model-00069-of-00101.safetensors", + "model.layers.63.mlp.experts.53.up_proj.weight": "model-00069-of-00101.safetensors", + "model.layers.63.mlp.experts.53.down_proj.weight": "model-00069-of-00101.safetensors", + "model.layers.63.mlp.experts.54.gate_proj.weight": "model-00069-of-00101.safetensors", + "model.layers.63.mlp.experts.54.up_proj.weight": "model-00069-of-00101.safetensors", + "model.layers.63.mlp.experts.54.down_proj.weight": "model-00069-of-00101.safetensors", + "model.layers.63.mlp.experts.55.gate_proj.weight": "model-00069-of-00101.safetensors", + "model.layers.63.mlp.experts.55.up_proj.weight": "model-00069-of-00101.safetensors", + "model.layers.63.mlp.experts.55.down_proj.weight": "model-00069-of-00101.safetensors", + "model.layers.63.mlp.experts.56.gate_proj.weight": "model-00069-of-00101.safetensors", + "model.layers.63.mlp.experts.56.up_proj.weight": "model-00069-of-00101.safetensors", + "model.layers.63.mlp.experts.56.down_proj.weight": "model-00069-of-00101.safetensors", + "model.layers.63.mlp.experts.57.gate_proj.weight": "model-00069-of-00101.safetensors", + "model.layers.63.mlp.experts.57.up_proj.weight": "model-00069-of-00101.safetensors", + "model.layers.63.mlp.experts.57.down_proj.weight": "model-00069-of-00101.safetensors", + "model.layers.63.mlp.experts.58.gate_proj.weight": "model-00069-of-00101.safetensors", + "model.layers.63.mlp.experts.58.up_proj.weight": "model-00069-of-00101.safetensors", + "model.layers.63.mlp.experts.58.down_proj.weight": "model-00069-of-00101.safetensors", + "model.layers.63.mlp.experts.59.gate_proj.weight": "model-00069-of-00101.safetensors", + "model.layers.63.mlp.experts.59.up_proj.weight": "model-00069-of-00101.safetensors", + "model.layers.63.mlp.experts.59.down_proj.weight": "model-00069-of-00101.safetensors", + "model.layers.63.mlp.experts.60.gate_proj.weight": "model-00069-of-00101.safetensors", + "model.layers.63.mlp.experts.60.up_proj.weight": "model-00069-of-00101.safetensors", + "model.layers.63.mlp.experts.60.down_proj.weight": "model-00069-of-00101.safetensors", + "model.layers.63.mlp.experts.61.gate_proj.weight": "model-00069-of-00101.safetensors", + "model.layers.63.mlp.experts.61.up_proj.weight": "model-00069-of-00101.safetensors", + "model.layers.63.mlp.experts.61.down_proj.weight": "model-00069-of-00101.safetensors", + "model.layers.63.mlp.experts.62.gate_proj.weight": "model-00069-of-00101.safetensors", + "model.layers.63.mlp.experts.62.up_proj.weight": "model-00069-of-00101.safetensors", + "model.layers.63.mlp.experts.62.down_proj.weight": "model-00069-of-00101.safetensors", + "model.layers.63.mlp.experts.63.gate_proj.weight": "model-00069-of-00101.safetensors", + "model.layers.63.mlp.experts.63.up_proj.weight": "model-00069-of-00101.safetensors", + "model.layers.63.mlp.experts.63.down_proj.weight": "model-00069-of-00101.safetensors", + "model.layers.63.mlp.experts.64.gate_proj.weight": "model-00069-of-00101.safetensors", + "model.layers.63.mlp.experts.64.up_proj.weight": "model-00069-of-00101.safetensors", + "model.layers.63.mlp.experts.64.down_proj.weight": "model-00069-of-00101.safetensors", + "model.layers.63.mlp.experts.65.gate_proj.weight": "model-00069-of-00101.safetensors", + "model.layers.63.mlp.experts.65.up_proj.weight": "model-00069-of-00101.safetensors", + "model.layers.63.mlp.experts.65.down_proj.weight": "model-00069-of-00101.safetensors", + "model.layers.63.mlp.experts.66.gate_proj.weight": "model-00069-of-00101.safetensors", + "model.layers.63.mlp.experts.66.up_proj.weight": "model-00069-of-00101.safetensors", + "model.layers.63.mlp.experts.66.down_proj.weight": "model-00069-of-00101.safetensors", + "model.layers.63.mlp.experts.67.gate_proj.weight": "model-00069-of-00101.safetensors", + "model.layers.63.mlp.experts.67.up_proj.weight": "model-00069-of-00101.safetensors", + "model.layers.63.mlp.experts.67.down_proj.weight": "model-00069-of-00101.safetensors", + "model.layers.63.mlp.experts.68.gate_proj.weight": "model-00069-of-00101.safetensors", + "model.layers.63.mlp.experts.68.up_proj.weight": "model-00069-of-00101.safetensors", + "model.layers.63.mlp.experts.68.down_proj.weight": "model-00069-of-00101.safetensors", + "model.layers.63.mlp.experts.69.gate_proj.weight": "model-00069-of-00101.safetensors", + "model.layers.63.mlp.experts.69.up_proj.weight": "model-00069-of-00101.safetensors", + "model.layers.63.mlp.experts.69.down_proj.weight": "model-00069-of-00101.safetensors", + "model.layers.63.mlp.experts.70.gate_proj.weight": "model-00069-of-00101.safetensors", + "model.layers.63.mlp.experts.70.up_proj.weight": "model-00069-of-00101.safetensors", + "model.layers.63.mlp.experts.70.down_proj.weight": "model-00069-of-00101.safetensors", + "model.layers.63.mlp.experts.71.gate_proj.weight": "model-00069-of-00101.safetensors", + "model.layers.63.mlp.experts.71.up_proj.weight": "model-00069-of-00101.safetensors", + "model.layers.63.mlp.experts.71.down_proj.weight": "model-00069-of-00101.safetensors", + "model.layers.63.mlp.experts.72.gate_proj.weight": "model-00069-of-00101.safetensors", + "model.layers.63.mlp.experts.72.up_proj.weight": "model-00069-of-00101.safetensors", + "model.layers.63.mlp.experts.72.down_proj.weight": "model-00069-of-00101.safetensors", + "model.layers.63.mlp.experts.73.gate_proj.weight": "model-00069-of-00101.safetensors", + "model.layers.63.mlp.experts.73.up_proj.weight": "model-00069-of-00101.safetensors", + "model.layers.63.mlp.experts.73.down_proj.weight": "model-00069-of-00101.safetensors", + "model.layers.63.mlp.experts.74.gate_proj.weight": "model-00069-of-00101.safetensors", + "model.layers.63.mlp.experts.74.up_proj.weight": "model-00069-of-00101.safetensors", + "model.layers.63.mlp.experts.74.down_proj.weight": "model-00069-of-00101.safetensors", + "model.layers.63.mlp.experts.75.gate_proj.weight": "model-00069-of-00101.safetensors", + "model.layers.63.mlp.experts.75.up_proj.weight": "model-00069-of-00101.safetensors", + "model.layers.63.mlp.experts.75.down_proj.weight": "model-00069-of-00101.safetensors", + "model.layers.63.mlp.experts.76.gate_proj.weight": "model-00069-of-00101.safetensors", + "model.layers.63.mlp.experts.76.up_proj.weight": "model-00069-of-00101.safetensors", + "model.layers.63.mlp.experts.76.down_proj.weight": "model-00069-of-00101.safetensors", + "model.layers.63.mlp.experts.77.gate_proj.weight": "model-00069-of-00101.safetensors", + "model.layers.63.mlp.experts.77.up_proj.weight": "model-00069-of-00101.safetensors", + "model.layers.63.mlp.experts.77.down_proj.weight": "model-00069-of-00101.safetensors", + "model.layers.63.mlp.experts.78.gate_proj.weight": "model-00069-of-00101.safetensors", + "model.layers.63.mlp.experts.78.up_proj.weight": "model-00069-of-00101.safetensors", + "model.layers.63.mlp.experts.78.down_proj.weight": "model-00069-of-00101.safetensors", + "model.layers.63.mlp.experts.79.gate_proj.weight": "model-00069-of-00101.safetensors", + "model.layers.63.mlp.experts.79.up_proj.weight": "model-00069-of-00101.safetensors", + "model.layers.63.mlp.experts.79.down_proj.weight": "model-00069-of-00101.safetensors", + "model.layers.63.mlp.experts.80.gate_proj.weight": "model-00069-of-00101.safetensors", + "model.layers.63.mlp.experts.80.up_proj.weight": "model-00069-of-00101.safetensors", + "model.layers.63.mlp.experts.80.down_proj.weight": "model-00069-of-00101.safetensors", + "model.layers.63.mlp.experts.81.gate_proj.weight": "model-00069-of-00101.safetensors", + "model.layers.63.mlp.experts.81.up_proj.weight": "model-00069-of-00101.safetensors", + "model.layers.63.mlp.experts.81.down_proj.weight": "model-00069-of-00101.safetensors", + "model.layers.63.mlp.experts.82.gate_proj.weight": "model-00069-of-00101.safetensors", + "model.layers.63.mlp.experts.82.up_proj.weight": "model-00069-of-00101.safetensors", + "model.layers.63.mlp.experts.82.down_proj.weight": "model-00069-of-00101.safetensors", + "model.layers.63.mlp.experts.83.gate_proj.weight": "model-00069-of-00101.safetensors", + "model.layers.63.mlp.experts.83.up_proj.weight": "model-00069-of-00101.safetensors", + "model.layers.63.mlp.experts.83.down_proj.weight": "model-00069-of-00101.safetensors", + "model.layers.63.mlp.experts.84.gate_proj.weight": "model-00069-of-00101.safetensors", + "model.layers.63.mlp.experts.84.up_proj.weight": "model-00069-of-00101.safetensors", + "model.layers.63.mlp.experts.84.down_proj.weight": "model-00069-of-00101.safetensors", + "model.layers.63.mlp.experts.85.gate_proj.weight": "model-00069-of-00101.safetensors", + "model.layers.63.mlp.experts.85.up_proj.weight": "model-00069-of-00101.safetensors", + "model.layers.63.mlp.experts.85.down_proj.weight": "model-00069-of-00101.safetensors", + "model.layers.63.mlp.experts.86.gate_proj.weight": "model-00069-of-00101.safetensors", + "model.layers.63.mlp.experts.86.up_proj.weight": "model-00069-of-00101.safetensors", + "model.layers.63.mlp.experts.86.down_proj.weight": "model-00069-of-00101.safetensors", + "model.layers.63.mlp.experts.87.gate_proj.weight": "model-00069-of-00101.safetensors", + "model.layers.63.mlp.experts.87.up_proj.weight": "model-00069-of-00101.safetensors", + "model.layers.63.mlp.experts.87.down_proj.weight": "model-00069-of-00101.safetensors", + "model.layers.63.mlp.experts.88.gate_proj.weight": "model-00069-of-00101.safetensors", + "model.layers.63.mlp.experts.88.up_proj.weight": "model-00069-of-00101.safetensors", + "model.layers.63.mlp.experts.88.down_proj.weight": "model-00069-of-00101.safetensors", + "model.layers.63.mlp.experts.89.gate_proj.weight": "model-00069-of-00101.safetensors", + "model.layers.63.mlp.experts.89.up_proj.weight": "model-00069-of-00101.safetensors", + "model.layers.63.mlp.experts.89.down_proj.weight": "model-00069-of-00101.safetensors", + "model.layers.63.mlp.experts.90.gate_proj.weight": "model-00069-of-00101.safetensors", + "model.layers.63.mlp.experts.90.up_proj.weight": "model-00069-of-00101.safetensors", + "model.layers.63.mlp.experts.90.down_proj.weight": "model-00069-of-00101.safetensors", + "model.layers.63.mlp.experts.91.gate_proj.weight": "model-00069-of-00101.safetensors", + "model.layers.63.mlp.experts.91.up_proj.weight": "model-00069-of-00101.safetensors", + "model.layers.63.mlp.experts.91.down_proj.weight": "model-00069-of-00101.safetensors", + "model.layers.63.mlp.experts.92.gate_proj.weight": "model-00069-of-00101.safetensors", + "model.layers.63.mlp.experts.92.up_proj.weight": "model-00069-of-00101.safetensors", + "model.layers.63.mlp.experts.92.down_proj.weight": "model-00069-of-00101.safetensors", + "model.layers.63.mlp.experts.93.gate_proj.weight": "model-00069-of-00101.safetensors", + "model.layers.63.mlp.experts.93.up_proj.weight": "model-00069-of-00101.safetensors", + "model.layers.63.mlp.experts.93.down_proj.weight": "model-00069-of-00101.safetensors", + "model.layers.63.mlp.experts.94.gate_proj.weight": "model-00069-of-00101.safetensors", + "model.layers.63.mlp.experts.94.up_proj.weight": "model-00069-of-00101.safetensors", + "model.layers.63.mlp.experts.94.down_proj.weight": "model-00069-of-00101.safetensors", + "model.layers.63.mlp.experts.95.gate_proj.weight": "model-00069-of-00101.safetensors", + "model.layers.63.mlp.experts.95.up_proj.weight": "model-00069-of-00101.safetensors", + "model.layers.63.mlp.experts.95.down_proj.weight": "model-00069-of-00101.safetensors", + "model.layers.63.mlp.experts.96.gate_proj.weight": "model-00069-of-00101.safetensors", + "model.layers.63.mlp.experts.96.up_proj.weight": "model-00069-of-00101.safetensors", + "model.layers.63.mlp.experts.96.down_proj.weight": "model-00069-of-00101.safetensors", + "model.layers.63.mlp.experts.97.gate_proj.weight": "model-00069-of-00101.safetensors", + "model.layers.63.mlp.experts.97.up_proj.weight": "model-00069-of-00101.safetensors", + "model.layers.63.mlp.experts.97.down_proj.weight": "model-00069-of-00101.safetensors", + "model.layers.63.mlp.experts.98.gate_proj.weight": "model-00069-of-00101.safetensors", + "model.layers.63.mlp.experts.98.up_proj.weight": "model-00069-of-00101.safetensors", + "model.layers.63.mlp.experts.98.down_proj.weight": "model-00069-of-00101.safetensors", + "model.layers.63.mlp.experts.99.gate_proj.weight": "model-00069-of-00101.safetensors", + "model.layers.63.mlp.experts.99.up_proj.weight": "model-00069-of-00101.safetensors", + "model.layers.63.mlp.experts.99.down_proj.weight": "model-00069-of-00101.safetensors", + "model.layers.63.mlp.experts.100.gate_proj.weight": "model-00069-of-00101.safetensors", + "model.layers.63.mlp.experts.100.up_proj.weight": "model-00069-of-00101.safetensors", + "model.layers.63.mlp.experts.100.down_proj.weight": "model-00069-of-00101.safetensors", + "model.layers.63.mlp.experts.101.gate_proj.weight": "model-00069-of-00101.safetensors", + "model.layers.63.mlp.experts.101.up_proj.weight": "model-00069-of-00101.safetensors", + "model.layers.63.mlp.experts.101.down_proj.weight": "model-00069-of-00101.safetensors", + "model.layers.63.mlp.experts.102.gate_proj.weight": "model-00069-of-00101.safetensors", + "model.layers.63.mlp.experts.102.up_proj.weight": "model-00069-of-00101.safetensors", + "model.layers.63.mlp.experts.102.down_proj.weight": "model-00069-of-00101.safetensors", + "model.layers.63.mlp.experts.103.gate_proj.weight": "model-00069-of-00101.safetensors", + "model.layers.63.mlp.experts.103.up_proj.weight": "model-00069-of-00101.safetensors", + "model.layers.63.mlp.experts.103.down_proj.weight": "model-00069-of-00101.safetensors", + "model.layers.63.mlp.experts.104.gate_proj.weight": "model-00069-of-00101.safetensors", + "model.layers.63.mlp.experts.104.up_proj.weight": "model-00069-of-00101.safetensors", + "model.layers.63.mlp.experts.104.down_proj.weight": "model-00069-of-00101.safetensors", + "model.layers.63.mlp.experts.105.gate_proj.weight": "model-00069-of-00101.safetensors", + "model.layers.63.mlp.experts.105.up_proj.weight": "model-00069-of-00101.safetensors", + "model.layers.63.mlp.experts.105.down_proj.weight": "model-00069-of-00101.safetensors", + "model.layers.63.mlp.experts.106.gate_proj.weight": "model-00069-of-00101.safetensors", + "model.layers.63.mlp.experts.106.up_proj.weight": "model-00069-of-00101.safetensors", + "model.layers.63.mlp.experts.106.down_proj.weight": "model-00069-of-00101.safetensors", + "model.layers.63.mlp.experts.107.gate_proj.weight": "model-00069-of-00101.safetensors", + "model.layers.63.mlp.experts.107.up_proj.weight": "model-00069-of-00101.safetensors", + "model.layers.63.mlp.experts.107.down_proj.weight": "model-00069-of-00101.safetensors", + "model.layers.63.mlp.experts.108.gate_proj.weight": "model-00069-of-00101.safetensors", + "model.layers.63.mlp.experts.108.up_proj.weight": "model-00069-of-00101.safetensors", + "model.layers.63.mlp.experts.108.down_proj.weight": "model-00069-of-00101.safetensors", + "model.layers.63.mlp.experts.109.gate_proj.weight": "model-00069-of-00101.safetensors", + "model.layers.63.mlp.experts.109.up_proj.weight": "model-00069-of-00101.safetensors", + "model.layers.63.mlp.experts.109.down_proj.weight": "model-00069-of-00101.safetensors", + "model.layers.63.mlp.experts.110.gate_proj.weight": "model-00069-of-00101.safetensors", + "model.layers.63.mlp.experts.110.up_proj.weight": "model-00069-of-00101.safetensors", + "model.layers.63.mlp.experts.110.down_proj.weight": "model-00069-of-00101.safetensors", + "model.layers.63.mlp.experts.111.gate_proj.weight": "model-00069-of-00101.safetensors", + "model.layers.63.mlp.experts.111.up_proj.weight": "model-00069-of-00101.safetensors", + "model.layers.63.mlp.experts.111.down_proj.weight": "model-00069-of-00101.safetensors", + "model.layers.63.mlp.experts.112.gate_proj.weight": "model-00069-of-00101.safetensors", + "model.layers.63.mlp.experts.112.up_proj.weight": "model-00069-of-00101.safetensors", + "model.layers.63.mlp.experts.112.down_proj.weight": "model-00069-of-00101.safetensors", + "model.layers.63.mlp.experts.113.gate_proj.weight": "model-00069-of-00101.safetensors", + "model.layers.63.mlp.experts.113.up_proj.weight": "model-00069-of-00101.safetensors", + "model.layers.63.mlp.experts.113.down_proj.weight": "model-00069-of-00101.safetensors", + "model.layers.63.mlp.experts.114.gate_proj.weight": "model-00069-of-00101.safetensors", + "model.layers.63.mlp.experts.114.up_proj.weight": "model-00069-of-00101.safetensors", + "model.layers.63.mlp.experts.114.down_proj.weight": "model-00069-of-00101.safetensors", + "model.layers.63.mlp.experts.115.gate_proj.weight": "model-00069-of-00101.safetensors", + "model.layers.63.mlp.experts.115.up_proj.weight": "model-00069-of-00101.safetensors", + "model.layers.63.mlp.experts.115.down_proj.weight": "model-00069-of-00101.safetensors", + "model.layers.63.mlp.experts.116.gate_proj.weight": "model-00069-of-00101.safetensors", + "model.layers.63.mlp.experts.116.up_proj.weight": "model-00069-of-00101.safetensors", + "model.layers.63.mlp.experts.116.down_proj.weight": "model-00069-of-00101.safetensors", + "model.layers.63.mlp.experts.117.gate_proj.weight": "model-00069-of-00101.safetensors", + "model.layers.63.mlp.experts.117.up_proj.weight": "model-00069-of-00101.safetensors", + "model.layers.63.mlp.experts.117.down_proj.weight": "model-00069-of-00101.safetensors", + "model.layers.63.mlp.experts.118.gate_proj.weight": "model-00069-of-00101.safetensors", + "model.layers.63.mlp.experts.118.up_proj.weight": "model-00069-of-00101.safetensors", + "model.layers.63.mlp.experts.118.down_proj.weight": "model-00069-of-00101.safetensors", + "model.layers.63.mlp.experts.119.gate_proj.weight": "model-00069-of-00101.safetensors", + "model.layers.63.mlp.experts.119.up_proj.weight": "model-00069-of-00101.safetensors", + "model.layers.63.mlp.experts.119.down_proj.weight": "model-00069-of-00101.safetensors", + "model.layers.63.mlp.gate.weight": "model-00069-of-00101.safetensors", + "model.layers.63.mlp.gate.e_score_correction_bias": "model-00069-of-00101.safetensors", + "model.layers.63.mlp.shared_experts.gate_proj.weight": "model-00069-of-00101.safetensors", + "model.layers.63.mlp.shared_experts.up_proj.weight": "model-00069-of-00101.safetensors", + "model.layers.63.mlp.shared_experts.down_proj.weight": "model-00069-of-00101.safetensors", + "model.layers.63.input_layernorm.weight": "model-00069-of-00101.safetensors", + "model.layers.63.post_attention_layernorm.weight": "model-00069-of-00101.safetensors", + "model.layers.64.self_attn.q_proj.weight": "model-00069-of-00101.safetensors", + "model.layers.64.self_attn.q_proj.bias": "model-00069-of-00101.safetensors", + "model.layers.64.self_attn.k_proj.weight": "model-00069-of-00101.safetensors", + "model.layers.64.self_attn.k_proj.bias": "model-00069-of-00101.safetensors", + "model.layers.64.self_attn.v_proj.weight": "model-00069-of-00101.safetensors", + "model.layers.64.self_attn.v_proj.bias": "model-00069-of-00101.safetensors", + "model.layers.64.self_attn.o_proj.weight": "model-00069-of-00101.safetensors", + "model.layers.64.self_attn.q_norm.weight": "model-00069-of-00101.safetensors", + "model.layers.64.self_attn.k_norm.weight": "model-00069-of-00101.safetensors", + "model.layers.64.mlp.experts.0.gate_proj.weight": "model-00069-of-00101.safetensors", + "model.layers.64.mlp.experts.0.up_proj.weight": "model-00069-of-00101.safetensors", + "model.layers.64.mlp.experts.0.down_proj.weight": "model-00069-of-00101.safetensors", + "model.layers.64.mlp.experts.1.gate_proj.weight": "model-00069-of-00101.safetensors", + "model.layers.64.mlp.experts.1.up_proj.weight": "model-00069-of-00101.safetensors", + "model.layers.64.mlp.experts.1.down_proj.weight": "model-00069-of-00101.safetensors", + "model.layers.64.mlp.experts.2.gate_proj.weight": "model-00069-of-00101.safetensors", + "model.layers.64.mlp.experts.2.up_proj.weight": "model-00069-of-00101.safetensors", + "model.layers.64.mlp.experts.2.down_proj.weight": "model-00069-of-00101.safetensors", + "model.layers.64.mlp.experts.3.gate_proj.weight": "model-00069-of-00101.safetensors", + "model.layers.64.mlp.experts.3.up_proj.weight": "model-00069-of-00101.safetensors", + "model.layers.64.mlp.experts.3.down_proj.weight": "model-00069-of-00101.safetensors", + "model.layers.64.mlp.experts.4.gate_proj.weight": "model-00069-of-00101.safetensors", + "model.layers.64.mlp.experts.4.up_proj.weight": "model-00069-of-00101.safetensors", + "model.layers.64.mlp.experts.4.down_proj.weight": "model-00069-of-00101.safetensors", + "model.layers.64.mlp.experts.5.gate_proj.weight": "model-00069-of-00101.safetensors", + "model.layers.64.mlp.experts.5.up_proj.weight": "model-00069-of-00101.safetensors", + "model.layers.64.mlp.experts.5.down_proj.weight": "model-00069-of-00101.safetensors", + "model.layers.64.mlp.experts.6.gate_proj.weight": "model-00069-of-00101.safetensors", + "model.layers.64.mlp.experts.6.up_proj.weight": "model-00069-of-00101.safetensors", + "model.layers.64.mlp.experts.6.down_proj.weight": "model-00069-of-00101.safetensors", + "model.layers.64.mlp.experts.7.gate_proj.weight": "model-00069-of-00101.safetensors", + "model.layers.64.mlp.experts.7.up_proj.weight": "model-00069-of-00101.safetensors", + "model.layers.64.mlp.experts.7.down_proj.weight": "model-00069-of-00101.safetensors", + "model.layers.64.mlp.experts.8.gate_proj.weight": "model-00069-of-00101.safetensors", + "model.layers.64.mlp.experts.8.up_proj.weight": "model-00069-of-00101.safetensors", + "model.layers.64.mlp.experts.8.down_proj.weight": "model-00069-of-00101.safetensors", + "model.layers.64.mlp.experts.9.gate_proj.weight": "model-00069-of-00101.safetensors", + "model.layers.64.mlp.experts.9.up_proj.weight": "model-00069-of-00101.safetensors", + "model.layers.64.mlp.experts.9.down_proj.weight": "model-00069-of-00101.safetensors", + "model.layers.64.mlp.experts.10.gate_proj.weight": "model-00069-of-00101.safetensors", + "model.layers.64.mlp.experts.10.up_proj.weight": "model-00069-of-00101.safetensors", + "model.layers.64.mlp.experts.10.down_proj.weight": "model-00069-of-00101.safetensors", + "model.layers.64.mlp.experts.11.gate_proj.weight": "model-00069-of-00101.safetensors", + "model.layers.64.mlp.experts.11.up_proj.weight": "model-00069-of-00101.safetensors", + "model.layers.64.mlp.experts.11.down_proj.weight": "model-00069-of-00101.safetensors", + "model.layers.64.mlp.experts.12.gate_proj.weight": "model-00069-of-00101.safetensors", + "model.layers.64.mlp.experts.12.up_proj.weight": "model-00069-of-00101.safetensors", + "model.layers.64.mlp.experts.12.down_proj.weight": "model-00069-of-00101.safetensors", + "model.layers.64.mlp.experts.13.gate_proj.weight": "model-00069-of-00101.safetensors", + "model.layers.64.mlp.experts.13.up_proj.weight": "model-00069-of-00101.safetensors", + "model.layers.64.mlp.experts.13.down_proj.weight": "model-00069-of-00101.safetensors", + "model.layers.64.mlp.experts.14.gate_proj.weight": "model-00069-of-00101.safetensors", + "model.layers.64.mlp.experts.14.up_proj.weight": "model-00070-of-00101.safetensors", + "model.layers.64.mlp.experts.14.down_proj.weight": "model-00070-of-00101.safetensors", + "model.layers.64.mlp.experts.15.gate_proj.weight": "model-00070-of-00101.safetensors", + "model.layers.64.mlp.experts.15.up_proj.weight": "model-00070-of-00101.safetensors", + "model.layers.64.mlp.experts.15.down_proj.weight": "model-00070-of-00101.safetensors", + "model.layers.64.mlp.experts.16.gate_proj.weight": "model-00070-of-00101.safetensors", + "model.layers.64.mlp.experts.16.up_proj.weight": "model-00070-of-00101.safetensors", + "model.layers.64.mlp.experts.16.down_proj.weight": "model-00070-of-00101.safetensors", + "model.layers.64.mlp.experts.17.gate_proj.weight": "model-00070-of-00101.safetensors", + "model.layers.64.mlp.experts.17.up_proj.weight": "model-00070-of-00101.safetensors", + "model.layers.64.mlp.experts.17.down_proj.weight": "model-00070-of-00101.safetensors", + "model.layers.64.mlp.experts.18.gate_proj.weight": "model-00070-of-00101.safetensors", + "model.layers.64.mlp.experts.18.up_proj.weight": "model-00070-of-00101.safetensors", + "model.layers.64.mlp.experts.18.down_proj.weight": "model-00070-of-00101.safetensors", + "model.layers.64.mlp.experts.19.gate_proj.weight": "model-00070-of-00101.safetensors", + "model.layers.64.mlp.experts.19.up_proj.weight": "model-00070-of-00101.safetensors", + "model.layers.64.mlp.experts.19.down_proj.weight": "model-00070-of-00101.safetensors", + "model.layers.64.mlp.experts.20.gate_proj.weight": "model-00070-of-00101.safetensors", + "model.layers.64.mlp.experts.20.up_proj.weight": "model-00070-of-00101.safetensors", + "model.layers.64.mlp.experts.20.down_proj.weight": "model-00070-of-00101.safetensors", + "model.layers.64.mlp.experts.21.gate_proj.weight": "model-00070-of-00101.safetensors", + "model.layers.64.mlp.experts.21.up_proj.weight": "model-00070-of-00101.safetensors", + "model.layers.64.mlp.experts.21.down_proj.weight": "model-00070-of-00101.safetensors", + "model.layers.64.mlp.experts.22.gate_proj.weight": "model-00070-of-00101.safetensors", + "model.layers.64.mlp.experts.22.up_proj.weight": "model-00070-of-00101.safetensors", + "model.layers.64.mlp.experts.22.down_proj.weight": "model-00070-of-00101.safetensors", + "model.layers.64.mlp.experts.23.gate_proj.weight": "model-00070-of-00101.safetensors", + "model.layers.64.mlp.experts.23.up_proj.weight": "model-00070-of-00101.safetensors", + "model.layers.64.mlp.experts.23.down_proj.weight": "model-00070-of-00101.safetensors", + "model.layers.64.mlp.experts.24.gate_proj.weight": "model-00070-of-00101.safetensors", + "model.layers.64.mlp.experts.24.up_proj.weight": "model-00070-of-00101.safetensors", + "model.layers.64.mlp.experts.24.down_proj.weight": "model-00070-of-00101.safetensors", + "model.layers.64.mlp.experts.25.gate_proj.weight": "model-00070-of-00101.safetensors", + "model.layers.64.mlp.experts.25.up_proj.weight": "model-00070-of-00101.safetensors", + "model.layers.64.mlp.experts.25.down_proj.weight": "model-00070-of-00101.safetensors", + "model.layers.64.mlp.experts.26.gate_proj.weight": "model-00070-of-00101.safetensors", + "model.layers.64.mlp.experts.26.up_proj.weight": "model-00070-of-00101.safetensors", + "model.layers.64.mlp.experts.26.down_proj.weight": "model-00070-of-00101.safetensors", + "model.layers.64.mlp.experts.27.gate_proj.weight": "model-00070-of-00101.safetensors", + "model.layers.64.mlp.experts.27.up_proj.weight": "model-00070-of-00101.safetensors", + "model.layers.64.mlp.experts.27.down_proj.weight": "model-00070-of-00101.safetensors", + "model.layers.64.mlp.experts.28.gate_proj.weight": "model-00070-of-00101.safetensors", + "model.layers.64.mlp.experts.28.up_proj.weight": "model-00070-of-00101.safetensors", + "model.layers.64.mlp.experts.28.down_proj.weight": "model-00070-of-00101.safetensors", + "model.layers.64.mlp.experts.29.gate_proj.weight": "model-00070-of-00101.safetensors", + "model.layers.64.mlp.experts.29.up_proj.weight": "model-00070-of-00101.safetensors", + "model.layers.64.mlp.experts.29.down_proj.weight": "model-00070-of-00101.safetensors", + "model.layers.64.mlp.experts.30.gate_proj.weight": "model-00070-of-00101.safetensors", + "model.layers.64.mlp.experts.30.up_proj.weight": "model-00070-of-00101.safetensors", + "model.layers.64.mlp.experts.30.down_proj.weight": "model-00070-of-00101.safetensors", + "model.layers.64.mlp.experts.31.gate_proj.weight": "model-00070-of-00101.safetensors", + "model.layers.64.mlp.experts.31.up_proj.weight": "model-00070-of-00101.safetensors", + "model.layers.64.mlp.experts.31.down_proj.weight": "model-00070-of-00101.safetensors", + "model.layers.64.mlp.experts.32.gate_proj.weight": "model-00070-of-00101.safetensors", + "model.layers.64.mlp.experts.32.up_proj.weight": "model-00070-of-00101.safetensors", + "model.layers.64.mlp.experts.32.down_proj.weight": "model-00070-of-00101.safetensors", + "model.layers.64.mlp.experts.33.gate_proj.weight": "model-00070-of-00101.safetensors", + "model.layers.64.mlp.experts.33.up_proj.weight": "model-00070-of-00101.safetensors", + "model.layers.64.mlp.experts.33.down_proj.weight": "model-00070-of-00101.safetensors", + "model.layers.64.mlp.experts.34.gate_proj.weight": "model-00070-of-00101.safetensors", + "model.layers.64.mlp.experts.34.up_proj.weight": "model-00070-of-00101.safetensors", + "model.layers.64.mlp.experts.34.down_proj.weight": "model-00070-of-00101.safetensors", + "model.layers.64.mlp.experts.35.gate_proj.weight": "model-00070-of-00101.safetensors", + "model.layers.64.mlp.experts.35.up_proj.weight": "model-00070-of-00101.safetensors", + "model.layers.64.mlp.experts.35.down_proj.weight": "model-00070-of-00101.safetensors", + "model.layers.64.mlp.experts.36.gate_proj.weight": "model-00070-of-00101.safetensors", + "model.layers.64.mlp.experts.36.up_proj.weight": "model-00070-of-00101.safetensors", + "model.layers.64.mlp.experts.36.down_proj.weight": "model-00070-of-00101.safetensors", + "model.layers.64.mlp.experts.37.gate_proj.weight": "model-00070-of-00101.safetensors", + "model.layers.64.mlp.experts.37.up_proj.weight": "model-00070-of-00101.safetensors", + "model.layers.64.mlp.experts.37.down_proj.weight": "model-00070-of-00101.safetensors", + "model.layers.64.mlp.experts.38.gate_proj.weight": "model-00070-of-00101.safetensors", + "model.layers.64.mlp.experts.38.up_proj.weight": "model-00070-of-00101.safetensors", + "model.layers.64.mlp.experts.38.down_proj.weight": "model-00070-of-00101.safetensors", + "model.layers.64.mlp.experts.39.gate_proj.weight": "model-00070-of-00101.safetensors", + "model.layers.64.mlp.experts.39.up_proj.weight": "model-00070-of-00101.safetensors", + "model.layers.64.mlp.experts.39.down_proj.weight": "model-00070-of-00101.safetensors", + "model.layers.64.mlp.experts.40.gate_proj.weight": "model-00070-of-00101.safetensors", + "model.layers.64.mlp.experts.40.up_proj.weight": "model-00070-of-00101.safetensors", + "model.layers.64.mlp.experts.40.down_proj.weight": "model-00070-of-00101.safetensors", + "model.layers.64.mlp.experts.41.gate_proj.weight": "model-00070-of-00101.safetensors", + "model.layers.64.mlp.experts.41.up_proj.weight": "model-00070-of-00101.safetensors", + "model.layers.64.mlp.experts.41.down_proj.weight": "model-00070-of-00101.safetensors", + "model.layers.64.mlp.experts.42.gate_proj.weight": "model-00070-of-00101.safetensors", + "model.layers.64.mlp.experts.42.up_proj.weight": "model-00070-of-00101.safetensors", + "model.layers.64.mlp.experts.42.down_proj.weight": "model-00070-of-00101.safetensors", + "model.layers.64.mlp.experts.43.gate_proj.weight": "model-00070-of-00101.safetensors", + "model.layers.64.mlp.experts.43.up_proj.weight": "model-00070-of-00101.safetensors", + "model.layers.64.mlp.experts.43.down_proj.weight": "model-00070-of-00101.safetensors", + "model.layers.64.mlp.experts.44.gate_proj.weight": "model-00070-of-00101.safetensors", + "model.layers.64.mlp.experts.44.up_proj.weight": "model-00070-of-00101.safetensors", + "model.layers.64.mlp.experts.44.down_proj.weight": "model-00070-of-00101.safetensors", + "model.layers.64.mlp.experts.45.gate_proj.weight": "model-00070-of-00101.safetensors", + "model.layers.64.mlp.experts.45.up_proj.weight": "model-00070-of-00101.safetensors", + "model.layers.64.mlp.experts.45.down_proj.weight": "model-00070-of-00101.safetensors", + "model.layers.64.mlp.experts.46.gate_proj.weight": "model-00070-of-00101.safetensors", + "model.layers.64.mlp.experts.46.up_proj.weight": "model-00070-of-00101.safetensors", + "model.layers.64.mlp.experts.46.down_proj.weight": "model-00070-of-00101.safetensors", + "model.layers.64.mlp.experts.47.gate_proj.weight": "model-00070-of-00101.safetensors", + "model.layers.64.mlp.experts.47.up_proj.weight": "model-00070-of-00101.safetensors", + "model.layers.64.mlp.experts.47.down_proj.weight": "model-00070-of-00101.safetensors", + "model.layers.64.mlp.experts.48.gate_proj.weight": "model-00070-of-00101.safetensors", + "model.layers.64.mlp.experts.48.up_proj.weight": "model-00070-of-00101.safetensors", + "model.layers.64.mlp.experts.48.down_proj.weight": "model-00070-of-00101.safetensors", + "model.layers.64.mlp.experts.49.gate_proj.weight": "model-00070-of-00101.safetensors", + "model.layers.64.mlp.experts.49.up_proj.weight": "model-00070-of-00101.safetensors", + "model.layers.64.mlp.experts.49.down_proj.weight": "model-00070-of-00101.safetensors", + "model.layers.64.mlp.experts.50.gate_proj.weight": "model-00070-of-00101.safetensors", + "model.layers.64.mlp.experts.50.up_proj.weight": "model-00070-of-00101.safetensors", + "model.layers.64.mlp.experts.50.down_proj.weight": "model-00070-of-00101.safetensors", + "model.layers.64.mlp.experts.51.gate_proj.weight": "model-00070-of-00101.safetensors", + "model.layers.64.mlp.experts.51.up_proj.weight": "model-00070-of-00101.safetensors", + "model.layers.64.mlp.experts.51.down_proj.weight": "model-00070-of-00101.safetensors", + "model.layers.64.mlp.experts.52.gate_proj.weight": "model-00070-of-00101.safetensors", + "model.layers.64.mlp.experts.52.up_proj.weight": "model-00070-of-00101.safetensors", + "model.layers.64.mlp.experts.52.down_proj.weight": "model-00070-of-00101.safetensors", + "model.layers.64.mlp.experts.53.gate_proj.weight": "model-00070-of-00101.safetensors", + "model.layers.64.mlp.experts.53.up_proj.weight": "model-00070-of-00101.safetensors", + "model.layers.64.mlp.experts.53.down_proj.weight": "model-00070-of-00101.safetensors", + "model.layers.64.mlp.experts.54.gate_proj.weight": "model-00070-of-00101.safetensors", + "model.layers.64.mlp.experts.54.up_proj.weight": "model-00070-of-00101.safetensors", + "model.layers.64.mlp.experts.54.down_proj.weight": "model-00070-of-00101.safetensors", + "model.layers.64.mlp.experts.55.gate_proj.weight": "model-00070-of-00101.safetensors", + "model.layers.64.mlp.experts.55.up_proj.weight": "model-00070-of-00101.safetensors", + "model.layers.64.mlp.experts.55.down_proj.weight": "model-00070-of-00101.safetensors", + "model.layers.64.mlp.experts.56.gate_proj.weight": "model-00070-of-00101.safetensors", + "model.layers.64.mlp.experts.56.up_proj.weight": "model-00070-of-00101.safetensors", + "model.layers.64.mlp.experts.56.down_proj.weight": "model-00070-of-00101.safetensors", + "model.layers.64.mlp.experts.57.gate_proj.weight": "model-00070-of-00101.safetensors", + "model.layers.64.mlp.experts.57.up_proj.weight": "model-00070-of-00101.safetensors", + "model.layers.64.mlp.experts.57.down_proj.weight": "model-00070-of-00101.safetensors", + "model.layers.64.mlp.experts.58.gate_proj.weight": "model-00070-of-00101.safetensors", + "model.layers.64.mlp.experts.58.up_proj.weight": "model-00070-of-00101.safetensors", + "model.layers.64.mlp.experts.58.down_proj.weight": "model-00070-of-00101.safetensors", + "model.layers.64.mlp.experts.59.gate_proj.weight": "model-00070-of-00101.safetensors", + "model.layers.64.mlp.experts.59.up_proj.weight": "model-00070-of-00101.safetensors", + "model.layers.64.mlp.experts.59.down_proj.weight": "model-00070-of-00101.safetensors", + "model.layers.64.mlp.experts.60.gate_proj.weight": "model-00070-of-00101.safetensors", + "model.layers.64.mlp.experts.60.up_proj.weight": "model-00070-of-00101.safetensors", + "model.layers.64.mlp.experts.60.down_proj.weight": "model-00070-of-00101.safetensors", + "model.layers.64.mlp.experts.61.gate_proj.weight": "model-00070-of-00101.safetensors", + "model.layers.64.mlp.experts.61.up_proj.weight": "model-00070-of-00101.safetensors", + "model.layers.64.mlp.experts.61.down_proj.weight": "model-00070-of-00101.safetensors", + "model.layers.64.mlp.experts.62.gate_proj.weight": "model-00070-of-00101.safetensors", + "model.layers.64.mlp.experts.62.up_proj.weight": "model-00070-of-00101.safetensors", + "model.layers.64.mlp.experts.62.down_proj.weight": "model-00070-of-00101.safetensors", + "model.layers.64.mlp.experts.63.gate_proj.weight": "model-00070-of-00101.safetensors", + "model.layers.64.mlp.experts.63.up_proj.weight": "model-00070-of-00101.safetensors", + "model.layers.64.mlp.experts.63.down_proj.weight": "model-00070-of-00101.safetensors", + "model.layers.64.mlp.experts.64.gate_proj.weight": "model-00070-of-00101.safetensors", + "model.layers.64.mlp.experts.64.up_proj.weight": "model-00070-of-00101.safetensors", + "model.layers.64.mlp.experts.64.down_proj.weight": "model-00070-of-00101.safetensors", + "model.layers.64.mlp.experts.65.gate_proj.weight": "model-00070-of-00101.safetensors", + "model.layers.64.mlp.experts.65.up_proj.weight": "model-00070-of-00101.safetensors", + "model.layers.64.mlp.experts.65.down_proj.weight": "model-00070-of-00101.safetensors", + "model.layers.64.mlp.experts.66.gate_proj.weight": "model-00070-of-00101.safetensors", + "model.layers.64.mlp.experts.66.up_proj.weight": "model-00070-of-00101.safetensors", + "model.layers.64.mlp.experts.66.down_proj.weight": "model-00070-of-00101.safetensors", + "model.layers.64.mlp.experts.67.gate_proj.weight": "model-00070-of-00101.safetensors", + "model.layers.64.mlp.experts.67.up_proj.weight": "model-00070-of-00101.safetensors", + "model.layers.64.mlp.experts.67.down_proj.weight": "model-00070-of-00101.safetensors", + "model.layers.64.mlp.experts.68.gate_proj.weight": "model-00070-of-00101.safetensors", + "model.layers.64.mlp.experts.68.up_proj.weight": "model-00070-of-00101.safetensors", + "model.layers.64.mlp.experts.68.down_proj.weight": "model-00070-of-00101.safetensors", + "model.layers.64.mlp.experts.69.gate_proj.weight": "model-00070-of-00101.safetensors", + "model.layers.64.mlp.experts.69.up_proj.weight": "model-00070-of-00101.safetensors", + "model.layers.64.mlp.experts.69.down_proj.weight": "model-00070-of-00101.safetensors", + "model.layers.64.mlp.experts.70.gate_proj.weight": "model-00070-of-00101.safetensors", + "model.layers.64.mlp.experts.70.up_proj.weight": "model-00070-of-00101.safetensors", + "model.layers.64.mlp.experts.70.down_proj.weight": "model-00070-of-00101.safetensors", + "model.layers.64.mlp.experts.71.gate_proj.weight": "model-00070-of-00101.safetensors", + "model.layers.64.mlp.experts.71.up_proj.weight": "model-00070-of-00101.safetensors", + "model.layers.64.mlp.experts.71.down_proj.weight": "model-00070-of-00101.safetensors", + "model.layers.64.mlp.experts.72.gate_proj.weight": "model-00070-of-00101.safetensors", + "model.layers.64.mlp.experts.72.up_proj.weight": "model-00070-of-00101.safetensors", + "model.layers.64.mlp.experts.72.down_proj.weight": "model-00070-of-00101.safetensors", + "model.layers.64.mlp.experts.73.gate_proj.weight": "model-00070-of-00101.safetensors", + "model.layers.64.mlp.experts.73.up_proj.weight": "model-00070-of-00101.safetensors", + "model.layers.64.mlp.experts.73.down_proj.weight": "model-00070-of-00101.safetensors", + "model.layers.64.mlp.experts.74.gate_proj.weight": "model-00070-of-00101.safetensors", + "model.layers.64.mlp.experts.74.up_proj.weight": "model-00070-of-00101.safetensors", + "model.layers.64.mlp.experts.74.down_proj.weight": "model-00070-of-00101.safetensors", + "model.layers.64.mlp.experts.75.gate_proj.weight": "model-00070-of-00101.safetensors", + "model.layers.64.mlp.experts.75.up_proj.weight": "model-00070-of-00101.safetensors", + "model.layers.64.mlp.experts.75.down_proj.weight": "model-00070-of-00101.safetensors", + "model.layers.64.mlp.experts.76.gate_proj.weight": "model-00070-of-00101.safetensors", + "model.layers.64.mlp.experts.76.up_proj.weight": "model-00070-of-00101.safetensors", + "model.layers.64.mlp.experts.76.down_proj.weight": "model-00070-of-00101.safetensors", + "model.layers.64.mlp.experts.77.gate_proj.weight": "model-00070-of-00101.safetensors", + "model.layers.64.mlp.experts.77.up_proj.weight": "model-00070-of-00101.safetensors", + "model.layers.64.mlp.experts.77.down_proj.weight": "model-00070-of-00101.safetensors", + "model.layers.64.mlp.experts.78.gate_proj.weight": "model-00070-of-00101.safetensors", + "model.layers.64.mlp.experts.78.up_proj.weight": "model-00070-of-00101.safetensors", + "model.layers.64.mlp.experts.78.down_proj.weight": "model-00070-of-00101.safetensors", + "model.layers.64.mlp.experts.79.gate_proj.weight": "model-00070-of-00101.safetensors", + "model.layers.64.mlp.experts.79.up_proj.weight": "model-00070-of-00101.safetensors", + "model.layers.64.mlp.experts.79.down_proj.weight": "model-00070-of-00101.safetensors", + "model.layers.64.mlp.experts.80.gate_proj.weight": "model-00070-of-00101.safetensors", + "model.layers.64.mlp.experts.80.up_proj.weight": "model-00070-of-00101.safetensors", + "model.layers.64.mlp.experts.80.down_proj.weight": "model-00070-of-00101.safetensors", + "model.layers.64.mlp.experts.81.gate_proj.weight": "model-00070-of-00101.safetensors", + "model.layers.64.mlp.experts.81.up_proj.weight": "model-00070-of-00101.safetensors", + "model.layers.64.mlp.experts.81.down_proj.weight": "model-00070-of-00101.safetensors", + "model.layers.64.mlp.experts.82.gate_proj.weight": "model-00070-of-00101.safetensors", + "model.layers.64.mlp.experts.82.up_proj.weight": "model-00070-of-00101.safetensors", + "model.layers.64.mlp.experts.82.down_proj.weight": "model-00070-of-00101.safetensors", + "model.layers.64.mlp.experts.83.gate_proj.weight": "model-00070-of-00101.safetensors", + "model.layers.64.mlp.experts.83.up_proj.weight": "model-00070-of-00101.safetensors", + "model.layers.64.mlp.experts.83.down_proj.weight": "model-00070-of-00101.safetensors", + "model.layers.64.mlp.experts.84.gate_proj.weight": "model-00070-of-00101.safetensors", + "model.layers.64.mlp.experts.84.up_proj.weight": "model-00070-of-00101.safetensors", + "model.layers.64.mlp.experts.84.down_proj.weight": "model-00070-of-00101.safetensors", + "model.layers.64.mlp.experts.85.gate_proj.weight": "model-00070-of-00101.safetensors", + "model.layers.64.mlp.experts.85.up_proj.weight": "model-00070-of-00101.safetensors", + "model.layers.64.mlp.experts.85.down_proj.weight": "model-00070-of-00101.safetensors", + "model.layers.64.mlp.experts.86.gate_proj.weight": "model-00070-of-00101.safetensors", + "model.layers.64.mlp.experts.86.up_proj.weight": "model-00070-of-00101.safetensors", + "model.layers.64.mlp.experts.86.down_proj.weight": "model-00070-of-00101.safetensors", + "model.layers.64.mlp.experts.87.gate_proj.weight": "model-00070-of-00101.safetensors", + "model.layers.64.mlp.experts.87.up_proj.weight": "model-00070-of-00101.safetensors", + "model.layers.64.mlp.experts.87.down_proj.weight": "model-00070-of-00101.safetensors", + "model.layers.64.mlp.experts.88.gate_proj.weight": "model-00070-of-00101.safetensors", + "model.layers.64.mlp.experts.88.up_proj.weight": "model-00070-of-00101.safetensors", + "model.layers.64.mlp.experts.88.down_proj.weight": "model-00070-of-00101.safetensors", + "model.layers.64.mlp.experts.89.gate_proj.weight": "model-00070-of-00101.safetensors", + "model.layers.64.mlp.experts.89.up_proj.weight": "model-00070-of-00101.safetensors", + "model.layers.64.mlp.experts.89.down_proj.weight": "model-00070-of-00101.safetensors", + "model.layers.64.mlp.experts.90.gate_proj.weight": "model-00070-of-00101.safetensors", + "model.layers.64.mlp.experts.90.up_proj.weight": "model-00070-of-00101.safetensors", + "model.layers.64.mlp.experts.90.down_proj.weight": "model-00070-of-00101.safetensors", + "model.layers.64.mlp.experts.91.gate_proj.weight": "model-00070-of-00101.safetensors", + "model.layers.64.mlp.experts.91.up_proj.weight": "model-00070-of-00101.safetensors", + "model.layers.64.mlp.experts.91.down_proj.weight": "model-00070-of-00101.safetensors", + "model.layers.64.mlp.experts.92.gate_proj.weight": "model-00070-of-00101.safetensors", + "model.layers.64.mlp.experts.92.up_proj.weight": "model-00070-of-00101.safetensors", + "model.layers.64.mlp.experts.92.down_proj.weight": "model-00070-of-00101.safetensors", + "model.layers.64.mlp.experts.93.gate_proj.weight": "model-00070-of-00101.safetensors", + "model.layers.64.mlp.experts.93.up_proj.weight": "model-00070-of-00101.safetensors", + "model.layers.64.mlp.experts.93.down_proj.weight": "model-00070-of-00101.safetensors", + "model.layers.64.mlp.experts.94.gate_proj.weight": "model-00070-of-00101.safetensors", + "model.layers.64.mlp.experts.94.up_proj.weight": "model-00070-of-00101.safetensors", + "model.layers.64.mlp.experts.94.down_proj.weight": "model-00070-of-00101.safetensors", + "model.layers.64.mlp.experts.95.gate_proj.weight": "model-00070-of-00101.safetensors", + "model.layers.64.mlp.experts.95.up_proj.weight": "model-00070-of-00101.safetensors", + "model.layers.64.mlp.experts.95.down_proj.weight": "model-00070-of-00101.safetensors", + "model.layers.64.mlp.experts.96.gate_proj.weight": "model-00070-of-00101.safetensors", + "model.layers.64.mlp.experts.96.up_proj.weight": "model-00070-of-00101.safetensors", + "model.layers.64.mlp.experts.96.down_proj.weight": "model-00070-of-00101.safetensors", + "model.layers.64.mlp.experts.97.gate_proj.weight": "model-00070-of-00101.safetensors", + "model.layers.64.mlp.experts.97.up_proj.weight": "model-00070-of-00101.safetensors", + "model.layers.64.mlp.experts.97.down_proj.weight": "model-00070-of-00101.safetensors", + "model.layers.64.mlp.experts.98.gate_proj.weight": "model-00070-of-00101.safetensors", + "model.layers.64.mlp.experts.98.up_proj.weight": "model-00070-of-00101.safetensors", + "model.layers.64.mlp.experts.98.down_proj.weight": "model-00070-of-00101.safetensors", + "model.layers.64.mlp.experts.99.gate_proj.weight": "model-00070-of-00101.safetensors", + "model.layers.64.mlp.experts.99.up_proj.weight": "model-00070-of-00101.safetensors", + "model.layers.64.mlp.experts.99.down_proj.weight": "model-00070-of-00101.safetensors", + "model.layers.64.mlp.experts.100.gate_proj.weight": "model-00070-of-00101.safetensors", + "model.layers.64.mlp.experts.100.up_proj.weight": "model-00070-of-00101.safetensors", + "model.layers.64.mlp.experts.100.down_proj.weight": "model-00070-of-00101.safetensors", + "model.layers.64.mlp.experts.101.gate_proj.weight": "model-00070-of-00101.safetensors", + "model.layers.64.mlp.experts.101.up_proj.weight": "model-00070-of-00101.safetensors", + "model.layers.64.mlp.experts.101.down_proj.weight": "model-00070-of-00101.safetensors", + "model.layers.64.mlp.experts.102.gate_proj.weight": "model-00070-of-00101.safetensors", + "model.layers.64.mlp.experts.102.up_proj.weight": "model-00070-of-00101.safetensors", + "model.layers.64.mlp.experts.102.down_proj.weight": "model-00070-of-00101.safetensors", + "model.layers.64.mlp.experts.103.gate_proj.weight": "model-00070-of-00101.safetensors", + "model.layers.64.mlp.experts.103.up_proj.weight": "model-00070-of-00101.safetensors", + "model.layers.64.mlp.experts.103.down_proj.weight": "model-00070-of-00101.safetensors", + "model.layers.64.mlp.experts.104.gate_proj.weight": "model-00070-of-00101.safetensors", + "model.layers.64.mlp.experts.104.up_proj.weight": "model-00070-of-00101.safetensors", + "model.layers.64.mlp.experts.104.down_proj.weight": "model-00070-of-00101.safetensors", + "model.layers.64.mlp.experts.105.gate_proj.weight": "model-00070-of-00101.safetensors", + "model.layers.64.mlp.experts.105.up_proj.weight": "model-00070-of-00101.safetensors", + "model.layers.64.mlp.experts.105.down_proj.weight": "model-00070-of-00101.safetensors", + "model.layers.64.mlp.experts.106.gate_proj.weight": "model-00070-of-00101.safetensors", + "model.layers.64.mlp.experts.106.up_proj.weight": "model-00070-of-00101.safetensors", + "model.layers.64.mlp.experts.106.down_proj.weight": "model-00070-of-00101.safetensors", + "model.layers.64.mlp.experts.107.gate_proj.weight": "model-00070-of-00101.safetensors", + "model.layers.64.mlp.experts.107.up_proj.weight": "model-00070-of-00101.safetensors", + "model.layers.64.mlp.experts.107.down_proj.weight": "model-00070-of-00101.safetensors", + "model.layers.64.mlp.experts.108.gate_proj.weight": "model-00070-of-00101.safetensors", + "model.layers.64.mlp.experts.108.up_proj.weight": "model-00070-of-00101.safetensors", + "model.layers.64.mlp.experts.108.down_proj.weight": "model-00070-of-00101.safetensors", + "model.layers.64.mlp.experts.109.gate_proj.weight": "model-00070-of-00101.safetensors", + "model.layers.64.mlp.experts.109.up_proj.weight": "model-00070-of-00101.safetensors", + "model.layers.64.mlp.experts.109.down_proj.weight": "model-00070-of-00101.safetensors", + "model.layers.64.mlp.experts.110.gate_proj.weight": "model-00070-of-00101.safetensors", + "model.layers.64.mlp.experts.110.up_proj.weight": "model-00070-of-00101.safetensors", + "model.layers.64.mlp.experts.110.down_proj.weight": "model-00070-of-00101.safetensors", + "model.layers.64.mlp.experts.111.gate_proj.weight": "model-00070-of-00101.safetensors", + "model.layers.64.mlp.experts.111.up_proj.weight": "model-00070-of-00101.safetensors", + "model.layers.64.mlp.experts.111.down_proj.weight": "model-00070-of-00101.safetensors", + "model.layers.64.mlp.experts.112.gate_proj.weight": "model-00070-of-00101.safetensors", + "model.layers.64.mlp.experts.112.up_proj.weight": "model-00070-of-00101.safetensors", + "model.layers.64.mlp.experts.112.down_proj.weight": "model-00070-of-00101.safetensors", + "model.layers.64.mlp.experts.113.gate_proj.weight": "model-00070-of-00101.safetensors", + "model.layers.64.mlp.experts.113.up_proj.weight": "model-00070-of-00101.safetensors", + "model.layers.64.mlp.experts.113.down_proj.weight": "model-00070-of-00101.safetensors", + "model.layers.64.mlp.experts.114.gate_proj.weight": "model-00070-of-00101.safetensors", + "model.layers.64.mlp.experts.114.up_proj.weight": "model-00070-of-00101.safetensors", + "model.layers.64.mlp.experts.114.down_proj.weight": "model-00070-of-00101.safetensors", + "model.layers.64.mlp.experts.115.gate_proj.weight": "model-00070-of-00101.safetensors", + "model.layers.64.mlp.experts.115.up_proj.weight": "model-00070-of-00101.safetensors", + "model.layers.64.mlp.experts.115.down_proj.weight": "model-00070-of-00101.safetensors", + "model.layers.64.mlp.experts.116.gate_proj.weight": "model-00070-of-00101.safetensors", + "model.layers.64.mlp.experts.116.up_proj.weight": "model-00070-of-00101.safetensors", + "model.layers.64.mlp.experts.116.down_proj.weight": "model-00070-of-00101.safetensors", + "model.layers.64.mlp.experts.117.gate_proj.weight": "model-00070-of-00101.safetensors", + "model.layers.64.mlp.experts.117.up_proj.weight": "model-00070-of-00101.safetensors", + "model.layers.64.mlp.experts.117.down_proj.weight": "model-00070-of-00101.safetensors", + "model.layers.64.mlp.experts.118.gate_proj.weight": "model-00070-of-00101.safetensors", + "model.layers.64.mlp.experts.118.up_proj.weight": "model-00070-of-00101.safetensors", + "model.layers.64.mlp.experts.118.down_proj.weight": "model-00070-of-00101.safetensors", + "model.layers.64.mlp.experts.119.gate_proj.weight": "model-00070-of-00101.safetensors", + "model.layers.64.mlp.experts.119.up_proj.weight": "model-00070-of-00101.safetensors", + "model.layers.64.mlp.experts.119.down_proj.weight": "model-00070-of-00101.safetensors", + "model.layers.64.mlp.gate.weight": "model-00070-of-00101.safetensors", + "model.layers.64.mlp.gate.e_score_correction_bias": "model-00070-of-00101.safetensors", + "model.layers.64.mlp.shared_experts.gate_proj.weight": "model-00070-of-00101.safetensors", + "model.layers.64.mlp.shared_experts.up_proj.weight": "model-00070-of-00101.safetensors", + "model.layers.64.mlp.shared_experts.down_proj.weight": "model-00070-of-00101.safetensors", + "model.layers.64.input_layernorm.weight": "model-00070-of-00101.safetensors", + "model.layers.64.post_attention_layernorm.weight": "model-00070-of-00101.safetensors", + "model.layers.65.self_attn.q_proj.weight": "model-00070-of-00101.safetensors", + "model.layers.65.self_attn.q_proj.bias": "model-00070-of-00101.safetensors", + "model.layers.65.self_attn.k_proj.weight": "model-00070-of-00101.safetensors", + "model.layers.65.self_attn.k_proj.bias": "model-00070-of-00101.safetensors", + "model.layers.65.self_attn.v_proj.weight": "model-00070-of-00101.safetensors", + "model.layers.65.self_attn.v_proj.bias": "model-00070-of-00101.safetensors", + "model.layers.65.self_attn.o_proj.weight": "model-00070-of-00101.safetensors", + "model.layers.65.self_attn.q_norm.weight": "model-00070-of-00101.safetensors", + "model.layers.65.self_attn.k_norm.weight": "model-00070-of-00101.safetensors", + "model.layers.65.mlp.experts.0.gate_proj.weight": "model-00070-of-00101.safetensors", + "model.layers.65.mlp.experts.0.up_proj.weight": "model-00070-of-00101.safetensors", + "model.layers.65.mlp.experts.0.down_proj.weight": "model-00070-of-00101.safetensors", + "model.layers.65.mlp.experts.1.gate_proj.weight": "model-00071-of-00101.safetensors", + "model.layers.65.mlp.experts.1.up_proj.weight": "model-00071-of-00101.safetensors", + "model.layers.65.mlp.experts.1.down_proj.weight": "model-00071-of-00101.safetensors", + "model.layers.65.mlp.experts.2.gate_proj.weight": "model-00071-of-00101.safetensors", + "model.layers.65.mlp.experts.2.up_proj.weight": "model-00071-of-00101.safetensors", + "model.layers.65.mlp.experts.2.down_proj.weight": "model-00071-of-00101.safetensors", + "model.layers.65.mlp.experts.3.gate_proj.weight": "model-00071-of-00101.safetensors", + "model.layers.65.mlp.experts.3.up_proj.weight": "model-00071-of-00101.safetensors", + "model.layers.65.mlp.experts.3.down_proj.weight": "model-00071-of-00101.safetensors", + "model.layers.65.mlp.experts.4.gate_proj.weight": "model-00071-of-00101.safetensors", + "model.layers.65.mlp.experts.4.up_proj.weight": "model-00071-of-00101.safetensors", + "model.layers.65.mlp.experts.4.down_proj.weight": "model-00071-of-00101.safetensors", + "model.layers.65.mlp.experts.5.gate_proj.weight": "model-00071-of-00101.safetensors", + "model.layers.65.mlp.experts.5.up_proj.weight": "model-00071-of-00101.safetensors", + "model.layers.65.mlp.experts.5.down_proj.weight": "model-00071-of-00101.safetensors", + "model.layers.65.mlp.experts.6.gate_proj.weight": "model-00071-of-00101.safetensors", + "model.layers.65.mlp.experts.6.up_proj.weight": "model-00071-of-00101.safetensors", + "model.layers.65.mlp.experts.6.down_proj.weight": "model-00071-of-00101.safetensors", + "model.layers.65.mlp.experts.7.gate_proj.weight": "model-00071-of-00101.safetensors", + "model.layers.65.mlp.experts.7.up_proj.weight": "model-00071-of-00101.safetensors", + "model.layers.65.mlp.experts.7.down_proj.weight": "model-00071-of-00101.safetensors", + "model.layers.65.mlp.experts.8.gate_proj.weight": "model-00071-of-00101.safetensors", + "model.layers.65.mlp.experts.8.up_proj.weight": "model-00071-of-00101.safetensors", + "model.layers.65.mlp.experts.8.down_proj.weight": "model-00071-of-00101.safetensors", + "model.layers.65.mlp.experts.9.gate_proj.weight": "model-00071-of-00101.safetensors", + "model.layers.65.mlp.experts.9.up_proj.weight": "model-00071-of-00101.safetensors", + "model.layers.65.mlp.experts.9.down_proj.weight": "model-00071-of-00101.safetensors", + "model.layers.65.mlp.experts.10.gate_proj.weight": "model-00071-of-00101.safetensors", + "model.layers.65.mlp.experts.10.up_proj.weight": "model-00071-of-00101.safetensors", + "model.layers.65.mlp.experts.10.down_proj.weight": "model-00071-of-00101.safetensors", + "model.layers.65.mlp.experts.11.gate_proj.weight": "model-00071-of-00101.safetensors", + "model.layers.65.mlp.experts.11.up_proj.weight": "model-00071-of-00101.safetensors", + "model.layers.65.mlp.experts.11.down_proj.weight": "model-00071-of-00101.safetensors", + "model.layers.65.mlp.experts.12.gate_proj.weight": "model-00071-of-00101.safetensors", + "model.layers.65.mlp.experts.12.up_proj.weight": "model-00071-of-00101.safetensors", + "model.layers.65.mlp.experts.12.down_proj.weight": "model-00071-of-00101.safetensors", + "model.layers.65.mlp.experts.13.gate_proj.weight": "model-00071-of-00101.safetensors", + "model.layers.65.mlp.experts.13.up_proj.weight": "model-00071-of-00101.safetensors", + "model.layers.65.mlp.experts.13.down_proj.weight": "model-00071-of-00101.safetensors", + "model.layers.65.mlp.experts.14.gate_proj.weight": "model-00071-of-00101.safetensors", + "model.layers.65.mlp.experts.14.up_proj.weight": "model-00071-of-00101.safetensors", + "model.layers.65.mlp.experts.14.down_proj.weight": "model-00071-of-00101.safetensors", + "model.layers.65.mlp.experts.15.gate_proj.weight": "model-00071-of-00101.safetensors", + "model.layers.65.mlp.experts.15.up_proj.weight": "model-00071-of-00101.safetensors", + "model.layers.65.mlp.experts.15.down_proj.weight": "model-00071-of-00101.safetensors", + "model.layers.65.mlp.experts.16.gate_proj.weight": "model-00071-of-00101.safetensors", + "model.layers.65.mlp.experts.16.up_proj.weight": "model-00071-of-00101.safetensors", + "model.layers.65.mlp.experts.16.down_proj.weight": "model-00071-of-00101.safetensors", + "model.layers.65.mlp.experts.17.gate_proj.weight": "model-00071-of-00101.safetensors", + "model.layers.65.mlp.experts.17.up_proj.weight": "model-00071-of-00101.safetensors", + "model.layers.65.mlp.experts.17.down_proj.weight": "model-00071-of-00101.safetensors", + "model.layers.65.mlp.experts.18.gate_proj.weight": "model-00071-of-00101.safetensors", + "model.layers.65.mlp.experts.18.up_proj.weight": "model-00071-of-00101.safetensors", + "model.layers.65.mlp.experts.18.down_proj.weight": "model-00071-of-00101.safetensors", + "model.layers.65.mlp.experts.19.gate_proj.weight": "model-00071-of-00101.safetensors", + "model.layers.65.mlp.experts.19.up_proj.weight": "model-00071-of-00101.safetensors", + "model.layers.65.mlp.experts.19.down_proj.weight": "model-00071-of-00101.safetensors", + "model.layers.65.mlp.experts.20.gate_proj.weight": "model-00071-of-00101.safetensors", + "model.layers.65.mlp.experts.20.up_proj.weight": "model-00071-of-00101.safetensors", + "model.layers.65.mlp.experts.20.down_proj.weight": "model-00071-of-00101.safetensors", + "model.layers.65.mlp.experts.21.gate_proj.weight": "model-00071-of-00101.safetensors", + "model.layers.65.mlp.experts.21.up_proj.weight": "model-00071-of-00101.safetensors", + "model.layers.65.mlp.experts.21.down_proj.weight": "model-00071-of-00101.safetensors", + "model.layers.65.mlp.experts.22.gate_proj.weight": "model-00071-of-00101.safetensors", + "model.layers.65.mlp.experts.22.up_proj.weight": "model-00071-of-00101.safetensors", + "model.layers.65.mlp.experts.22.down_proj.weight": "model-00071-of-00101.safetensors", + "model.layers.65.mlp.experts.23.gate_proj.weight": "model-00071-of-00101.safetensors", + "model.layers.65.mlp.experts.23.up_proj.weight": "model-00071-of-00101.safetensors", + "model.layers.65.mlp.experts.23.down_proj.weight": "model-00071-of-00101.safetensors", + "model.layers.65.mlp.experts.24.gate_proj.weight": "model-00071-of-00101.safetensors", + "model.layers.65.mlp.experts.24.up_proj.weight": "model-00071-of-00101.safetensors", + "model.layers.65.mlp.experts.24.down_proj.weight": "model-00071-of-00101.safetensors", + "model.layers.65.mlp.experts.25.gate_proj.weight": "model-00071-of-00101.safetensors", + "model.layers.65.mlp.experts.25.up_proj.weight": "model-00071-of-00101.safetensors", + "model.layers.65.mlp.experts.25.down_proj.weight": "model-00071-of-00101.safetensors", + "model.layers.65.mlp.experts.26.gate_proj.weight": "model-00071-of-00101.safetensors", + "model.layers.65.mlp.experts.26.up_proj.weight": "model-00071-of-00101.safetensors", + "model.layers.65.mlp.experts.26.down_proj.weight": "model-00071-of-00101.safetensors", + "model.layers.65.mlp.experts.27.gate_proj.weight": "model-00071-of-00101.safetensors", + "model.layers.65.mlp.experts.27.up_proj.weight": "model-00071-of-00101.safetensors", + "model.layers.65.mlp.experts.27.down_proj.weight": "model-00071-of-00101.safetensors", + "model.layers.65.mlp.experts.28.gate_proj.weight": "model-00071-of-00101.safetensors", + "model.layers.65.mlp.experts.28.up_proj.weight": "model-00071-of-00101.safetensors", + "model.layers.65.mlp.experts.28.down_proj.weight": "model-00071-of-00101.safetensors", + "model.layers.65.mlp.experts.29.gate_proj.weight": "model-00071-of-00101.safetensors", + "model.layers.65.mlp.experts.29.up_proj.weight": "model-00071-of-00101.safetensors", + "model.layers.65.mlp.experts.29.down_proj.weight": "model-00071-of-00101.safetensors", + "model.layers.65.mlp.experts.30.gate_proj.weight": "model-00071-of-00101.safetensors", + "model.layers.65.mlp.experts.30.up_proj.weight": "model-00071-of-00101.safetensors", + "model.layers.65.mlp.experts.30.down_proj.weight": "model-00071-of-00101.safetensors", + "model.layers.65.mlp.experts.31.gate_proj.weight": "model-00071-of-00101.safetensors", + "model.layers.65.mlp.experts.31.up_proj.weight": "model-00071-of-00101.safetensors", + "model.layers.65.mlp.experts.31.down_proj.weight": "model-00071-of-00101.safetensors", + "model.layers.65.mlp.experts.32.gate_proj.weight": "model-00071-of-00101.safetensors", + "model.layers.65.mlp.experts.32.up_proj.weight": "model-00071-of-00101.safetensors", + "model.layers.65.mlp.experts.32.down_proj.weight": "model-00071-of-00101.safetensors", + "model.layers.65.mlp.experts.33.gate_proj.weight": "model-00071-of-00101.safetensors", + "model.layers.65.mlp.experts.33.up_proj.weight": "model-00071-of-00101.safetensors", + "model.layers.65.mlp.experts.33.down_proj.weight": "model-00071-of-00101.safetensors", + "model.layers.65.mlp.experts.34.gate_proj.weight": "model-00071-of-00101.safetensors", + "model.layers.65.mlp.experts.34.up_proj.weight": "model-00071-of-00101.safetensors", + "model.layers.65.mlp.experts.34.down_proj.weight": "model-00071-of-00101.safetensors", + "model.layers.65.mlp.experts.35.gate_proj.weight": "model-00071-of-00101.safetensors", + "model.layers.65.mlp.experts.35.up_proj.weight": "model-00071-of-00101.safetensors", + "model.layers.65.mlp.experts.35.down_proj.weight": "model-00071-of-00101.safetensors", + "model.layers.65.mlp.experts.36.gate_proj.weight": "model-00071-of-00101.safetensors", + "model.layers.65.mlp.experts.36.up_proj.weight": "model-00071-of-00101.safetensors", + "model.layers.65.mlp.experts.36.down_proj.weight": "model-00071-of-00101.safetensors", + "model.layers.65.mlp.experts.37.gate_proj.weight": "model-00071-of-00101.safetensors", + "model.layers.65.mlp.experts.37.up_proj.weight": "model-00071-of-00101.safetensors", + "model.layers.65.mlp.experts.37.down_proj.weight": "model-00071-of-00101.safetensors", + "model.layers.65.mlp.experts.38.gate_proj.weight": "model-00071-of-00101.safetensors", + "model.layers.65.mlp.experts.38.up_proj.weight": "model-00071-of-00101.safetensors", + "model.layers.65.mlp.experts.38.down_proj.weight": "model-00071-of-00101.safetensors", + "model.layers.65.mlp.experts.39.gate_proj.weight": "model-00071-of-00101.safetensors", + "model.layers.65.mlp.experts.39.up_proj.weight": "model-00071-of-00101.safetensors", + "model.layers.65.mlp.experts.39.down_proj.weight": "model-00071-of-00101.safetensors", + "model.layers.65.mlp.experts.40.gate_proj.weight": "model-00071-of-00101.safetensors", + "model.layers.65.mlp.experts.40.up_proj.weight": "model-00071-of-00101.safetensors", + "model.layers.65.mlp.experts.40.down_proj.weight": "model-00071-of-00101.safetensors", + "model.layers.65.mlp.experts.41.gate_proj.weight": "model-00071-of-00101.safetensors", + "model.layers.65.mlp.experts.41.up_proj.weight": "model-00071-of-00101.safetensors", + "model.layers.65.mlp.experts.41.down_proj.weight": "model-00071-of-00101.safetensors", + "model.layers.65.mlp.experts.42.gate_proj.weight": "model-00071-of-00101.safetensors", + "model.layers.65.mlp.experts.42.up_proj.weight": "model-00071-of-00101.safetensors", + "model.layers.65.mlp.experts.42.down_proj.weight": "model-00071-of-00101.safetensors", + "model.layers.65.mlp.experts.43.gate_proj.weight": "model-00071-of-00101.safetensors", + "model.layers.65.mlp.experts.43.up_proj.weight": "model-00071-of-00101.safetensors", + "model.layers.65.mlp.experts.43.down_proj.weight": "model-00071-of-00101.safetensors", + "model.layers.65.mlp.experts.44.gate_proj.weight": "model-00071-of-00101.safetensors", + "model.layers.65.mlp.experts.44.up_proj.weight": "model-00071-of-00101.safetensors", + "model.layers.65.mlp.experts.44.down_proj.weight": "model-00071-of-00101.safetensors", + "model.layers.65.mlp.experts.45.gate_proj.weight": "model-00071-of-00101.safetensors", + "model.layers.65.mlp.experts.45.up_proj.weight": "model-00071-of-00101.safetensors", + "model.layers.65.mlp.experts.45.down_proj.weight": "model-00071-of-00101.safetensors", + "model.layers.65.mlp.experts.46.gate_proj.weight": "model-00071-of-00101.safetensors", + "model.layers.65.mlp.experts.46.up_proj.weight": "model-00071-of-00101.safetensors", + "model.layers.65.mlp.experts.46.down_proj.weight": "model-00071-of-00101.safetensors", + "model.layers.65.mlp.experts.47.gate_proj.weight": "model-00071-of-00101.safetensors", + "model.layers.65.mlp.experts.47.up_proj.weight": "model-00071-of-00101.safetensors", + "model.layers.65.mlp.experts.47.down_proj.weight": "model-00071-of-00101.safetensors", + "model.layers.65.mlp.experts.48.gate_proj.weight": "model-00071-of-00101.safetensors", + "model.layers.65.mlp.experts.48.up_proj.weight": "model-00071-of-00101.safetensors", + "model.layers.65.mlp.experts.48.down_proj.weight": "model-00071-of-00101.safetensors", + "model.layers.65.mlp.experts.49.gate_proj.weight": "model-00071-of-00101.safetensors", + "model.layers.65.mlp.experts.49.up_proj.weight": "model-00071-of-00101.safetensors", + "model.layers.65.mlp.experts.49.down_proj.weight": "model-00071-of-00101.safetensors", + "model.layers.65.mlp.experts.50.gate_proj.weight": "model-00071-of-00101.safetensors", + "model.layers.65.mlp.experts.50.up_proj.weight": "model-00071-of-00101.safetensors", + "model.layers.65.mlp.experts.50.down_proj.weight": "model-00071-of-00101.safetensors", + "model.layers.65.mlp.experts.51.gate_proj.weight": "model-00071-of-00101.safetensors", + "model.layers.65.mlp.experts.51.up_proj.weight": "model-00071-of-00101.safetensors", + "model.layers.65.mlp.experts.51.down_proj.weight": "model-00071-of-00101.safetensors", + "model.layers.65.mlp.experts.52.gate_proj.weight": "model-00071-of-00101.safetensors", + "model.layers.65.mlp.experts.52.up_proj.weight": "model-00071-of-00101.safetensors", + "model.layers.65.mlp.experts.52.down_proj.weight": "model-00071-of-00101.safetensors", + "model.layers.65.mlp.experts.53.gate_proj.weight": "model-00071-of-00101.safetensors", + "model.layers.65.mlp.experts.53.up_proj.weight": "model-00071-of-00101.safetensors", + "model.layers.65.mlp.experts.53.down_proj.weight": "model-00071-of-00101.safetensors", + "model.layers.65.mlp.experts.54.gate_proj.weight": "model-00071-of-00101.safetensors", + "model.layers.65.mlp.experts.54.up_proj.weight": "model-00071-of-00101.safetensors", + "model.layers.65.mlp.experts.54.down_proj.weight": "model-00071-of-00101.safetensors", + "model.layers.65.mlp.experts.55.gate_proj.weight": "model-00071-of-00101.safetensors", + "model.layers.65.mlp.experts.55.up_proj.weight": "model-00071-of-00101.safetensors", + "model.layers.65.mlp.experts.55.down_proj.weight": "model-00071-of-00101.safetensors", + "model.layers.65.mlp.experts.56.gate_proj.weight": "model-00071-of-00101.safetensors", + "model.layers.65.mlp.experts.56.up_proj.weight": "model-00071-of-00101.safetensors", + "model.layers.65.mlp.experts.56.down_proj.weight": "model-00071-of-00101.safetensors", + "model.layers.65.mlp.experts.57.gate_proj.weight": "model-00071-of-00101.safetensors", + "model.layers.65.mlp.experts.57.up_proj.weight": "model-00071-of-00101.safetensors", + "model.layers.65.mlp.experts.57.down_proj.weight": "model-00071-of-00101.safetensors", + "model.layers.65.mlp.experts.58.gate_proj.weight": "model-00071-of-00101.safetensors", + "model.layers.65.mlp.experts.58.up_proj.weight": "model-00071-of-00101.safetensors", + "model.layers.65.mlp.experts.58.down_proj.weight": "model-00071-of-00101.safetensors", + "model.layers.65.mlp.experts.59.gate_proj.weight": "model-00071-of-00101.safetensors", + "model.layers.65.mlp.experts.59.up_proj.weight": "model-00071-of-00101.safetensors", + "model.layers.65.mlp.experts.59.down_proj.weight": "model-00071-of-00101.safetensors", + "model.layers.65.mlp.experts.60.gate_proj.weight": "model-00071-of-00101.safetensors", + "model.layers.65.mlp.experts.60.up_proj.weight": "model-00071-of-00101.safetensors", + "model.layers.65.mlp.experts.60.down_proj.weight": "model-00071-of-00101.safetensors", + "model.layers.65.mlp.experts.61.gate_proj.weight": "model-00071-of-00101.safetensors", + "model.layers.65.mlp.experts.61.up_proj.weight": "model-00071-of-00101.safetensors", + "model.layers.65.mlp.experts.61.down_proj.weight": "model-00071-of-00101.safetensors", + "model.layers.65.mlp.experts.62.gate_proj.weight": "model-00071-of-00101.safetensors", + "model.layers.65.mlp.experts.62.up_proj.weight": "model-00071-of-00101.safetensors", + "model.layers.65.mlp.experts.62.down_proj.weight": "model-00071-of-00101.safetensors", + "model.layers.65.mlp.experts.63.gate_proj.weight": "model-00071-of-00101.safetensors", + "model.layers.65.mlp.experts.63.up_proj.weight": "model-00071-of-00101.safetensors", + "model.layers.65.mlp.experts.63.down_proj.weight": "model-00071-of-00101.safetensors", + "model.layers.65.mlp.experts.64.gate_proj.weight": "model-00071-of-00101.safetensors", + "model.layers.65.mlp.experts.64.up_proj.weight": "model-00071-of-00101.safetensors", + "model.layers.65.mlp.experts.64.down_proj.weight": "model-00071-of-00101.safetensors", + "model.layers.65.mlp.experts.65.gate_proj.weight": "model-00071-of-00101.safetensors", + "model.layers.65.mlp.experts.65.up_proj.weight": "model-00071-of-00101.safetensors", + "model.layers.65.mlp.experts.65.down_proj.weight": "model-00071-of-00101.safetensors", + "model.layers.65.mlp.experts.66.gate_proj.weight": "model-00071-of-00101.safetensors", + "model.layers.65.mlp.experts.66.up_proj.weight": "model-00071-of-00101.safetensors", + "model.layers.65.mlp.experts.66.down_proj.weight": "model-00071-of-00101.safetensors", + "model.layers.65.mlp.experts.67.gate_proj.weight": "model-00071-of-00101.safetensors", + "model.layers.65.mlp.experts.67.up_proj.weight": "model-00071-of-00101.safetensors", + "model.layers.65.mlp.experts.67.down_proj.weight": "model-00071-of-00101.safetensors", + "model.layers.65.mlp.experts.68.gate_proj.weight": "model-00071-of-00101.safetensors", + "model.layers.65.mlp.experts.68.up_proj.weight": "model-00071-of-00101.safetensors", + "model.layers.65.mlp.experts.68.down_proj.weight": "model-00071-of-00101.safetensors", + "model.layers.65.mlp.experts.69.gate_proj.weight": "model-00071-of-00101.safetensors", + "model.layers.65.mlp.experts.69.up_proj.weight": "model-00071-of-00101.safetensors", + "model.layers.65.mlp.experts.69.down_proj.weight": "model-00071-of-00101.safetensors", + "model.layers.65.mlp.experts.70.gate_proj.weight": "model-00071-of-00101.safetensors", + "model.layers.65.mlp.experts.70.up_proj.weight": "model-00071-of-00101.safetensors", + "model.layers.65.mlp.experts.70.down_proj.weight": "model-00071-of-00101.safetensors", + "model.layers.65.mlp.experts.71.gate_proj.weight": "model-00071-of-00101.safetensors", + "model.layers.65.mlp.experts.71.up_proj.weight": "model-00071-of-00101.safetensors", + "model.layers.65.mlp.experts.71.down_proj.weight": "model-00071-of-00101.safetensors", + "model.layers.65.mlp.experts.72.gate_proj.weight": "model-00071-of-00101.safetensors", + "model.layers.65.mlp.experts.72.up_proj.weight": "model-00071-of-00101.safetensors", + "model.layers.65.mlp.experts.72.down_proj.weight": "model-00071-of-00101.safetensors", + "model.layers.65.mlp.experts.73.gate_proj.weight": "model-00071-of-00101.safetensors", + "model.layers.65.mlp.experts.73.up_proj.weight": "model-00071-of-00101.safetensors", + "model.layers.65.mlp.experts.73.down_proj.weight": "model-00071-of-00101.safetensors", + "model.layers.65.mlp.experts.74.gate_proj.weight": "model-00071-of-00101.safetensors", + "model.layers.65.mlp.experts.74.up_proj.weight": "model-00071-of-00101.safetensors", + "model.layers.65.mlp.experts.74.down_proj.weight": "model-00071-of-00101.safetensors", + "model.layers.65.mlp.experts.75.gate_proj.weight": "model-00071-of-00101.safetensors", + "model.layers.65.mlp.experts.75.up_proj.weight": "model-00071-of-00101.safetensors", + "model.layers.65.mlp.experts.75.down_proj.weight": "model-00071-of-00101.safetensors", + "model.layers.65.mlp.experts.76.gate_proj.weight": "model-00071-of-00101.safetensors", + "model.layers.65.mlp.experts.76.up_proj.weight": "model-00071-of-00101.safetensors", + "model.layers.65.mlp.experts.76.down_proj.weight": "model-00071-of-00101.safetensors", + "model.layers.65.mlp.experts.77.gate_proj.weight": "model-00071-of-00101.safetensors", + "model.layers.65.mlp.experts.77.up_proj.weight": "model-00071-of-00101.safetensors", + "model.layers.65.mlp.experts.77.down_proj.weight": "model-00071-of-00101.safetensors", + "model.layers.65.mlp.experts.78.gate_proj.weight": "model-00071-of-00101.safetensors", + "model.layers.65.mlp.experts.78.up_proj.weight": "model-00071-of-00101.safetensors", + "model.layers.65.mlp.experts.78.down_proj.weight": "model-00071-of-00101.safetensors", + "model.layers.65.mlp.experts.79.gate_proj.weight": "model-00071-of-00101.safetensors", + "model.layers.65.mlp.experts.79.up_proj.weight": "model-00071-of-00101.safetensors", + "model.layers.65.mlp.experts.79.down_proj.weight": "model-00071-of-00101.safetensors", + "model.layers.65.mlp.experts.80.gate_proj.weight": "model-00071-of-00101.safetensors", + "model.layers.65.mlp.experts.80.up_proj.weight": "model-00071-of-00101.safetensors", + "model.layers.65.mlp.experts.80.down_proj.weight": "model-00071-of-00101.safetensors", + "model.layers.65.mlp.experts.81.gate_proj.weight": "model-00071-of-00101.safetensors", + "model.layers.65.mlp.experts.81.up_proj.weight": "model-00071-of-00101.safetensors", + "model.layers.65.mlp.experts.81.down_proj.weight": "model-00071-of-00101.safetensors", + "model.layers.65.mlp.experts.82.gate_proj.weight": "model-00071-of-00101.safetensors", + "model.layers.65.mlp.experts.82.up_proj.weight": "model-00071-of-00101.safetensors", + "model.layers.65.mlp.experts.82.down_proj.weight": "model-00071-of-00101.safetensors", + "model.layers.65.mlp.experts.83.gate_proj.weight": "model-00071-of-00101.safetensors", + "model.layers.65.mlp.experts.83.up_proj.weight": "model-00071-of-00101.safetensors", + "model.layers.65.mlp.experts.83.down_proj.weight": "model-00071-of-00101.safetensors", + "model.layers.65.mlp.experts.84.gate_proj.weight": "model-00071-of-00101.safetensors", + "model.layers.65.mlp.experts.84.up_proj.weight": "model-00071-of-00101.safetensors", + "model.layers.65.mlp.experts.84.down_proj.weight": "model-00071-of-00101.safetensors", + "model.layers.65.mlp.experts.85.gate_proj.weight": "model-00071-of-00101.safetensors", + "model.layers.65.mlp.experts.85.up_proj.weight": "model-00071-of-00101.safetensors", + "model.layers.65.mlp.experts.85.down_proj.weight": "model-00071-of-00101.safetensors", + "model.layers.65.mlp.experts.86.gate_proj.weight": "model-00071-of-00101.safetensors", + "model.layers.65.mlp.experts.86.up_proj.weight": "model-00071-of-00101.safetensors", + "model.layers.65.mlp.experts.86.down_proj.weight": "model-00071-of-00101.safetensors", + "model.layers.65.mlp.experts.87.gate_proj.weight": "model-00071-of-00101.safetensors", + "model.layers.65.mlp.experts.87.up_proj.weight": "model-00071-of-00101.safetensors", + "model.layers.65.mlp.experts.87.down_proj.weight": "model-00071-of-00101.safetensors", + "model.layers.65.mlp.experts.88.gate_proj.weight": "model-00071-of-00101.safetensors", + "model.layers.65.mlp.experts.88.up_proj.weight": "model-00071-of-00101.safetensors", + "model.layers.65.mlp.experts.88.down_proj.weight": "model-00071-of-00101.safetensors", + "model.layers.65.mlp.experts.89.gate_proj.weight": "model-00071-of-00101.safetensors", + "model.layers.65.mlp.experts.89.up_proj.weight": "model-00071-of-00101.safetensors", + "model.layers.65.mlp.experts.89.down_proj.weight": "model-00071-of-00101.safetensors", + "model.layers.65.mlp.experts.90.gate_proj.weight": "model-00071-of-00101.safetensors", + "model.layers.65.mlp.experts.90.up_proj.weight": "model-00071-of-00101.safetensors", + "model.layers.65.mlp.experts.90.down_proj.weight": "model-00071-of-00101.safetensors", + "model.layers.65.mlp.experts.91.gate_proj.weight": "model-00071-of-00101.safetensors", + "model.layers.65.mlp.experts.91.up_proj.weight": "model-00071-of-00101.safetensors", + "model.layers.65.mlp.experts.91.down_proj.weight": "model-00071-of-00101.safetensors", + "model.layers.65.mlp.experts.92.gate_proj.weight": "model-00071-of-00101.safetensors", + "model.layers.65.mlp.experts.92.up_proj.weight": "model-00071-of-00101.safetensors", + "model.layers.65.mlp.experts.92.down_proj.weight": "model-00071-of-00101.safetensors", + "model.layers.65.mlp.experts.93.gate_proj.weight": "model-00071-of-00101.safetensors", + "model.layers.65.mlp.experts.93.up_proj.weight": "model-00071-of-00101.safetensors", + "model.layers.65.mlp.experts.93.down_proj.weight": "model-00071-of-00101.safetensors", + "model.layers.65.mlp.experts.94.gate_proj.weight": "model-00071-of-00101.safetensors", + "model.layers.65.mlp.experts.94.up_proj.weight": "model-00071-of-00101.safetensors", + "model.layers.65.mlp.experts.94.down_proj.weight": "model-00071-of-00101.safetensors", + "model.layers.65.mlp.experts.95.gate_proj.weight": "model-00071-of-00101.safetensors", + "model.layers.65.mlp.experts.95.up_proj.weight": "model-00071-of-00101.safetensors", + "model.layers.65.mlp.experts.95.down_proj.weight": "model-00071-of-00101.safetensors", + "model.layers.65.mlp.experts.96.gate_proj.weight": "model-00071-of-00101.safetensors", + "model.layers.65.mlp.experts.96.up_proj.weight": "model-00071-of-00101.safetensors", + "model.layers.65.mlp.experts.96.down_proj.weight": "model-00071-of-00101.safetensors", + "model.layers.65.mlp.experts.97.gate_proj.weight": "model-00071-of-00101.safetensors", + "model.layers.65.mlp.experts.97.up_proj.weight": "model-00071-of-00101.safetensors", + "model.layers.65.mlp.experts.97.down_proj.weight": "model-00071-of-00101.safetensors", + "model.layers.65.mlp.experts.98.gate_proj.weight": "model-00071-of-00101.safetensors", + "model.layers.65.mlp.experts.98.up_proj.weight": "model-00071-of-00101.safetensors", + "model.layers.65.mlp.experts.98.down_proj.weight": "model-00071-of-00101.safetensors", + "model.layers.65.mlp.experts.99.gate_proj.weight": "model-00071-of-00101.safetensors", + "model.layers.65.mlp.experts.99.up_proj.weight": "model-00071-of-00101.safetensors", + "model.layers.65.mlp.experts.99.down_proj.weight": "model-00071-of-00101.safetensors", + "model.layers.65.mlp.experts.100.gate_proj.weight": "model-00071-of-00101.safetensors", + "model.layers.65.mlp.experts.100.up_proj.weight": "model-00071-of-00101.safetensors", + "model.layers.65.mlp.experts.100.down_proj.weight": "model-00071-of-00101.safetensors", + "model.layers.65.mlp.experts.101.gate_proj.weight": "model-00071-of-00101.safetensors", + "model.layers.65.mlp.experts.101.up_proj.weight": "model-00071-of-00101.safetensors", + "model.layers.65.mlp.experts.101.down_proj.weight": "model-00071-of-00101.safetensors", + "model.layers.65.mlp.experts.102.gate_proj.weight": "model-00071-of-00101.safetensors", + "model.layers.65.mlp.experts.102.up_proj.weight": "model-00071-of-00101.safetensors", + "model.layers.65.mlp.experts.102.down_proj.weight": "model-00071-of-00101.safetensors", + "model.layers.65.mlp.experts.103.gate_proj.weight": "model-00071-of-00101.safetensors", + "model.layers.65.mlp.experts.103.up_proj.weight": "model-00071-of-00101.safetensors", + "model.layers.65.mlp.experts.103.down_proj.weight": "model-00071-of-00101.safetensors", + "model.layers.65.mlp.experts.104.gate_proj.weight": "model-00071-of-00101.safetensors", + "model.layers.65.mlp.experts.104.up_proj.weight": "model-00071-of-00101.safetensors", + "model.layers.65.mlp.experts.104.down_proj.weight": "model-00071-of-00101.safetensors", + "model.layers.65.mlp.experts.105.gate_proj.weight": "model-00071-of-00101.safetensors", + "model.layers.65.mlp.experts.105.up_proj.weight": "model-00071-of-00101.safetensors", + "model.layers.65.mlp.experts.105.down_proj.weight": "model-00071-of-00101.safetensors", + "model.layers.65.mlp.experts.106.gate_proj.weight": "model-00071-of-00101.safetensors", + "model.layers.65.mlp.experts.106.up_proj.weight": "model-00071-of-00101.safetensors", + "model.layers.65.mlp.experts.106.down_proj.weight": "model-00071-of-00101.safetensors", + "model.layers.65.mlp.experts.107.gate_proj.weight": "model-00071-of-00101.safetensors", + "model.layers.65.mlp.experts.107.up_proj.weight": "model-00071-of-00101.safetensors", + "model.layers.65.mlp.experts.107.down_proj.weight": "model-00071-of-00101.safetensors", + "model.layers.65.mlp.experts.108.gate_proj.weight": "model-00071-of-00101.safetensors", + "model.layers.65.mlp.experts.108.up_proj.weight": "model-00071-of-00101.safetensors", + "model.layers.65.mlp.experts.108.down_proj.weight": "model-00071-of-00101.safetensors", + "model.layers.65.mlp.experts.109.gate_proj.weight": "model-00071-of-00101.safetensors", + "model.layers.65.mlp.experts.109.up_proj.weight": "model-00071-of-00101.safetensors", + "model.layers.65.mlp.experts.109.down_proj.weight": "model-00071-of-00101.safetensors", + "model.layers.65.mlp.experts.110.gate_proj.weight": "model-00071-of-00101.safetensors", + "model.layers.65.mlp.experts.110.up_proj.weight": "model-00071-of-00101.safetensors", + "model.layers.65.mlp.experts.110.down_proj.weight": "model-00071-of-00101.safetensors", + "model.layers.65.mlp.experts.111.gate_proj.weight": "model-00071-of-00101.safetensors", + "model.layers.65.mlp.experts.111.up_proj.weight": "model-00071-of-00101.safetensors", + "model.layers.65.mlp.experts.111.down_proj.weight": "model-00071-of-00101.safetensors", + "model.layers.65.mlp.experts.112.gate_proj.weight": "model-00071-of-00101.safetensors", + "model.layers.65.mlp.experts.112.up_proj.weight": "model-00071-of-00101.safetensors", + "model.layers.65.mlp.experts.112.down_proj.weight": "model-00071-of-00101.safetensors", + "model.layers.65.mlp.experts.113.gate_proj.weight": "model-00071-of-00101.safetensors", + "model.layers.65.mlp.experts.113.up_proj.weight": "model-00071-of-00101.safetensors", + "model.layers.65.mlp.experts.113.down_proj.weight": "model-00071-of-00101.safetensors", + "model.layers.65.mlp.experts.114.gate_proj.weight": "model-00071-of-00101.safetensors", + "model.layers.65.mlp.experts.114.up_proj.weight": "model-00071-of-00101.safetensors", + "model.layers.65.mlp.experts.114.down_proj.weight": "model-00072-of-00101.safetensors", + "model.layers.65.mlp.experts.115.gate_proj.weight": "model-00072-of-00101.safetensors", + "model.layers.65.mlp.experts.115.up_proj.weight": "model-00072-of-00101.safetensors", + "model.layers.65.mlp.experts.115.down_proj.weight": "model-00072-of-00101.safetensors", + "model.layers.65.mlp.experts.116.gate_proj.weight": "model-00072-of-00101.safetensors", + "model.layers.65.mlp.experts.116.up_proj.weight": "model-00072-of-00101.safetensors", + "model.layers.65.mlp.experts.116.down_proj.weight": "model-00072-of-00101.safetensors", + "model.layers.65.mlp.experts.117.gate_proj.weight": "model-00072-of-00101.safetensors", + "model.layers.65.mlp.experts.117.up_proj.weight": "model-00072-of-00101.safetensors", + "model.layers.65.mlp.experts.117.down_proj.weight": "model-00072-of-00101.safetensors", + "model.layers.65.mlp.experts.118.gate_proj.weight": "model-00072-of-00101.safetensors", + "model.layers.65.mlp.experts.118.up_proj.weight": "model-00072-of-00101.safetensors", + "model.layers.65.mlp.experts.118.down_proj.weight": "model-00072-of-00101.safetensors", + "model.layers.65.mlp.experts.119.gate_proj.weight": "model-00072-of-00101.safetensors", + "model.layers.65.mlp.experts.119.up_proj.weight": "model-00072-of-00101.safetensors", + "model.layers.65.mlp.experts.119.down_proj.weight": "model-00072-of-00101.safetensors", + "model.layers.65.mlp.gate.weight": "model-00072-of-00101.safetensors", + "model.layers.65.mlp.gate.e_score_correction_bias": "model-00072-of-00101.safetensors", + "model.layers.65.mlp.shared_experts.gate_proj.weight": "model-00072-of-00101.safetensors", + "model.layers.65.mlp.shared_experts.up_proj.weight": "model-00072-of-00101.safetensors", + "model.layers.65.mlp.shared_experts.down_proj.weight": "model-00072-of-00101.safetensors", + "model.layers.65.input_layernorm.weight": "model-00072-of-00101.safetensors", + "model.layers.65.post_attention_layernorm.weight": "model-00072-of-00101.safetensors", + "model.layers.66.self_attn.q_proj.weight": "model-00072-of-00101.safetensors", + "model.layers.66.self_attn.q_proj.bias": "model-00072-of-00101.safetensors", + "model.layers.66.self_attn.k_proj.weight": "model-00072-of-00101.safetensors", + "model.layers.66.self_attn.k_proj.bias": "model-00072-of-00101.safetensors", + "model.layers.66.self_attn.v_proj.weight": "model-00072-of-00101.safetensors", + "model.layers.66.self_attn.v_proj.bias": "model-00072-of-00101.safetensors", + "model.layers.66.self_attn.o_proj.weight": "model-00072-of-00101.safetensors", + "model.layers.66.self_attn.q_norm.weight": "model-00072-of-00101.safetensors", + "model.layers.66.self_attn.k_norm.weight": "model-00072-of-00101.safetensors", + "model.layers.66.mlp.experts.0.gate_proj.weight": "model-00072-of-00101.safetensors", + "model.layers.66.mlp.experts.0.up_proj.weight": "model-00072-of-00101.safetensors", + "model.layers.66.mlp.experts.0.down_proj.weight": "model-00072-of-00101.safetensors", + "model.layers.66.mlp.experts.1.gate_proj.weight": "model-00072-of-00101.safetensors", + "model.layers.66.mlp.experts.1.up_proj.weight": "model-00072-of-00101.safetensors", + "model.layers.66.mlp.experts.1.down_proj.weight": "model-00072-of-00101.safetensors", + "model.layers.66.mlp.experts.2.gate_proj.weight": "model-00072-of-00101.safetensors", + "model.layers.66.mlp.experts.2.up_proj.weight": "model-00072-of-00101.safetensors", + "model.layers.66.mlp.experts.2.down_proj.weight": "model-00072-of-00101.safetensors", + "model.layers.66.mlp.experts.3.gate_proj.weight": "model-00072-of-00101.safetensors", + "model.layers.66.mlp.experts.3.up_proj.weight": "model-00072-of-00101.safetensors", + "model.layers.66.mlp.experts.3.down_proj.weight": "model-00072-of-00101.safetensors", + "model.layers.66.mlp.experts.4.gate_proj.weight": "model-00072-of-00101.safetensors", + "model.layers.66.mlp.experts.4.up_proj.weight": "model-00072-of-00101.safetensors", + "model.layers.66.mlp.experts.4.down_proj.weight": "model-00072-of-00101.safetensors", + "model.layers.66.mlp.experts.5.gate_proj.weight": "model-00072-of-00101.safetensors", + "model.layers.66.mlp.experts.5.up_proj.weight": "model-00072-of-00101.safetensors", + "model.layers.66.mlp.experts.5.down_proj.weight": "model-00072-of-00101.safetensors", + "model.layers.66.mlp.experts.6.gate_proj.weight": "model-00072-of-00101.safetensors", + "model.layers.66.mlp.experts.6.up_proj.weight": "model-00072-of-00101.safetensors", + "model.layers.66.mlp.experts.6.down_proj.weight": "model-00072-of-00101.safetensors", + "model.layers.66.mlp.experts.7.gate_proj.weight": "model-00072-of-00101.safetensors", + "model.layers.66.mlp.experts.7.up_proj.weight": "model-00072-of-00101.safetensors", + "model.layers.66.mlp.experts.7.down_proj.weight": "model-00072-of-00101.safetensors", + "model.layers.66.mlp.experts.8.gate_proj.weight": "model-00072-of-00101.safetensors", + "model.layers.66.mlp.experts.8.up_proj.weight": "model-00072-of-00101.safetensors", + "model.layers.66.mlp.experts.8.down_proj.weight": "model-00072-of-00101.safetensors", + "model.layers.66.mlp.experts.9.gate_proj.weight": "model-00072-of-00101.safetensors", + "model.layers.66.mlp.experts.9.up_proj.weight": "model-00072-of-00101.safetensors", + "model.layers.66.mlp.experts.9.down_proj.weight": "model-00072-of-00101.safetensors", + "model.layers.66.mlp.experts.10.gate_proj.weight": "model-00072-of-00101.safetensors", + "model.layers.66.mlp.experts.10.up_proj.weight": "model-00072-of-00101.safetensors", + "model.layers.66.mlp.experts.10.down_proj.weight": "model-00072-of-00101.safetensors", + "model.layers.66.mlp.experts.11.gate_proj.weight": "model-00072-of-00101.safetensors", + "model.layers.66.mlp.experts.11.up_proj.weight": "model-00072-of-00101.safetensors", + "model.layers.66.mlp.experts.11.down_proj.weight": "model-00072-of-00101.safetensors", + "model.layers.66.mlp.experts.12.gate_proj.weight": "model-00072-of-00101.safetensors", + "model.layers.66.mlp.experts.12.up_proj.weight": "model-00072-of-00101.safetensors", + "model.layers.66.mlp.experts.12.down_proj.weight": "model-00072-of-00101.safetensors", + "model.layers.66.mlp.experts.13.gate_proj.weight": "model-00072-of-00101.safetensors", + "model.layers.66.mlp.experts.13.up_proj.weight": "model-00072-of-00101.safetensors", + "model.layers.66.mlp.experts.13.down_proj.weight": "model-00072-of-00101.safetensors", + "model.layers.66.mlp.experts.14.gate_proj.weight": "model-00072-of-00101.safetensors", + "model.layers.66.mlp.experts.14.up_proj.weight": "model-00072-of-00101.safetensors", + "model.layers.66.mlp.experts.14.down_proj.weight": "model-00072-of-00101.safetensors", + "model.layers.66.mlp.experts.15.gate_proj.weight": "model-00072-of-00101.safetensors", + "model.layers.66.mlp.experts.15.up_proj.weight": "model-00072-of-00101.safetensors", + "model.layers.66.mlp.experts.15.down_proj.weight": "model-00072-of-00101.safetensors", + "model.layers.66.mlp.experts.16.gate_proj.weight": "model-00072-of-00101.safetensors", + "model.layers.66.mlp.experts.16.up_proj.weight": "model-00072-of-00101.safetensors", + "model.layers.66.mlp.experts.16.down_proj.weight": "model-00072-of-00101.safetensors", + "model.layers.66.mlp.experts.17.gate_proj.weight": "model-00072-of-00101.safetensors", + "model.layers.66.mlp.experts.17.up_proj.weight": "model-00072-of-00101.safetensors", + "model.layers.66.mlp.experts.17.down_proj.weight": "model-00072-of-00101.safetensors", + "model.layers.66.mlp.experts.18.gate_proj.weight": "model-00072-of-00101.safetensors", + "model.layers.66.mlp.experts.18.up_proj.weight": "model-00072-of-00101.safetensors", + "model.layers.66.mlp.experts.18.down_proj.weight": "model-00072-of-00101.safetensors", + "model.layers.66.mlp.experts.19.gate_proj.weight": "model-00072-of-00101.safetensors", + "model.layers.66.mlp.experts.19.up_proj.weight": "model-00072-of-00101.safetensors", + "model.layers.66.mlp.experts.19.down_proj.weight": "model-00072-of-00101.safetensors", + "model.layers.66.mlp.experts.20.gate_proj.weight": "model-00072-of-00101.safetensors", + "model.layers.66.mlp.experts.20.up_proj.weight": "model-00072-of-00101.safetensors", + "model.layers.66.mlp.experts.20.down_proj.weight": "model-00072-of-00101.safetensors", + "model.layers.66.mlp.experts.21.gate_proj.weight": "model-00072-of-00101.safetensors", + "model.layers.66.mlp.experts.21.up_proj.weight": "model-00072-of-00101.safetensors", + "model.layers.66.mlp.experts.21.down_proj.weight": "model-00072-of-00101.safetensors", + "model.layers.66.mlp.experts.22.gate_proj.weight": "model-00072-of-00101.safetensors", + "model.layers.66.mlp.experts.22.up_proj.weight": "model-00072-of-00101.safetensors", + "model.layers.66.mlp.experts.22.down_proj.weight": "model-00072-of-00101.safetensors", + "model.layers.66.mlp.experts.23.gate_proj.weight": "model-00072-of-00101.safetensors", + "model.layers.66.mlp.experts.23.up_proj.weight": "model-00072-of-00101.safetensors", + "model.layers.66.mlp.experts.23.down_proj.weight": "model-00072-of-00101.safetensors", + "model.layers.66.mlp.experts.24.gate_proj.weight": "model-00072-of-00101.safetensors", + "model.layers.66.mlp.experts.24.up_proj.weight": "model-00072-of-00101.safetensors", + "model.layers.66.mlp.experts.24.down_proj.weight": "model-00072-of-00101.safetensors", + "model.layers.66.mlp.experts.25.gate_proj.weight": "model-00072-of-00101.safetensors", + "model.layers.66.mlp.experts.25.up_proj.weight": "model-00072-of-00101.safetensors", + "model.layers.66.mlp.experts.25.down_proj.weight": "model-00072-of-00101.safetensors", + "model.layers.66.mlp.experts.26.gate_proj.weight": "model-00072-of-00101.safetensors", + "model.layers.66.mlp.experts.26.up_proj.weight": "model-00072-of-00101.safetensors", + "model.layers.66.mlp.experts.26.down_proj.weight": "model-00072-of-00101.safetensors", + "model.layers.66.mlp.experts.27.gate_proj.weight": "model-00072-of-00101.safetensors", + "model.layers.66.mlp.experts.27.up_proj.weight": "model-00072-of-00101.safetensors", + "model.layers.66.mlp.experts.27.down_proj.weight": "model-00072-of-00101.safetensors", + "model.layers.66.mlp.experts.28.gate_proj.weight": "model-00072-of-00101.safetensors", + "model.layers.66.mlp.experts.28.up_proj.weight": "model-00072-of-00101.safetensors", + "model.layers.66.mlp.experts.28.down_proj.weight": "model-00072-of-00101.safetensors", + "model.layers.66.mlp.experts.29.gate_proj.weight": "model-00072-of-00101.safetensors", + "model.layers.66.mlp.experts.29.up_proj.weight": "model-00072-of-00101.safetensors", + "model.layers.66.mlp.experts.29.down_proj.weight": "model-00072-of-00101.safetensors", + "model.layers.66.mlp.experts.30.gate_proj.weight": "model-00072-of-00101.safetensors", + "model.layers.66.mlp.experts.30.up_proj.weight": "model-00072-of-00101.safetensors", + "model.layers.66.mlp.experts.30.down_proj.weight": "model-00072-of-00101.safetensors", + "model.layers.66.mlp.experts.31.gate_proj.weight": "model-00072-of-00101.safetensors", + "model.layers.66.mlp.experts.31.up_proj.weight": "model-00072-of-00101.safetensors", + "model.layers.66.mlp.experts.31.down_proj.weight": "model-00072-of-00101.safetensors", + "model.layers.66.mlp.experts.32.gate_proj.weight": "model-00072-of-00101.safetensors", + "model.layers.66.mlp.experts.32.up_proj.weight": "model-00072-of-00101.safetensors", + "model.layers.66.mlp.experts.32.down_proj.weight": "model-00072-of-00101.safetensors", + "model.layers.66.mlp.experts.33.gate_proj.weight": "model-00072-of-00101.safetensors", + "model.layers.66.mlp.experts.33.up_proj.weight": "model-00072-of-00101.safetensors", + "model.layers.66.mlp.experts.33.down_proj.weight": "model-00072-of-00101.safetensors", + "model.layers.66.mlp.experts.34.gate_proj.weight": "model-00072-of-00101.safetensors", + "model.layers.66.mlp.experts.34.up_proj.weight": "model-00072-of-00101.safetensors", + "model.layers.66.mlp.experts.34.down_proj.weight": "model-00072-of-00101.safetensors", + "model.layers.66.mlp.experts.35.gate_proj.weight": "model-00072-of-00101.safetensors", + "model.layers.66.mlp.experts.35.up_proj.weight": "model-00072-of-00101.safetensors", + "model.layers.66.mlp.experts.35.down_proj.weight": "model-00072-of-00101.safetensors", + "model.layers.66.mlp.experts.36.gate_proj.weight": "model-00072-of-00101.safetensors", + "model.layers.66.mlp.experts.36.up_proj.weight": "model-00072-of-00101.safetensors", + "model.layers.66.mlp.experts.36.down_proj.weight": "model-00072-of-00101.safetensors", + "model.layers.66.mlp.experts.37.gate_proj.weight": "model-00072-of-00101.safetensors", + "model.layers.66.mlp.experts.37.up_proj.weight": "model-00072-of-00101.safetensors", + "model.layers.66.mlp.experts.37.down_proj.weight": "model-00072-of-00101.safetensors", + "model.layers.66.mlp.experts.38.gate_proj.weight": "model-00072-of-00101.safetensors", + "model.layers.66.mlp.experts.38.up_proj.weight": "model-00072-of-00101.safetensors", + "model.layers.66.mlp.experts.38.down_proj.weight": "model-00072-of-00101.safetensors", + "model.layers.66.mlp.experts.39.gate_proj.weight": "model-00072-of-00101.safetensors", + "model.layers.66.mlp.experts.39.up_proj.weight": "model-00072-of-00101.safetensors", + "model.layers.66.mlp.experts.39.down_proj.weight": "model-00072-of-00101.safetensors", + "model.layers.66.mlp.experts.40.gate_proj.weight": "model-00072-of-00101.safetensors", + "model.layers.66.mlp.experts.40.up_proj.weight": "model-00072-of-00101.safetensors", + "model.layers.66.mlp.experts.40.down_proj.weight": "model-00072-of-00101.safetensors", + "model.layers.66.mlp.experts.41.gate_proj.weight": "model-00072-of-00101.safetensors", + "model.layers.66.mlp.experts.41.up_proj.weight": "model-00072-of-00101.safetensors", + "model.layers.66.mlp.experts.41.down_proj.weight": "model-00072-of-00101.safetensors", + "model.layers.66.mlp.experts.42.gate_proj.weight": "model-00072-of-00101.safetensors", + "model.layers.66.mlp.experts.42.up_proj.weight": "model-00072-of-00101.safetensors", + "model.layers.66.mlp.experts.42.down_proj.weight": "model-00072-of-00101.safetensors", + "model.layers.66.mlp.experts.43.gate_proj.weight": "model-00072-of-00101.safetensors", + "model.layers.66.mlp.experts.43.up_proj.weight": "model-00072-of-00101.safetensors", + "model.layers.66.mlp.experts.43.down_proj.weight": "model-00072-of-00101.safetensors", + "model.layers.66.mlp.experts.44.gate_proj.weight": "model-00072-of-00101.safetensors", + "model.layers.66.mlp.experts.44.up_proj.weight": "model-00072-of-00101.safetensors", + "model.layers.66.mlp.experts.44.down_proj.weight": "model-00072-of-00101.safetensors", + "model.layers.66.mlp.experts.45.gate_proj.weight": "model-00072-of-00101.safetensors", + "model.layers.66.mlp.experts.45.up_proj.weight": "model-00072-of-00101.safetensors", + "model.layers.66.mlp.experts.45.down_proj.weight": "model-00072-of-00101.safetensors", + "model.layers.66.mlp.experts.46.gate_proj.weight": "model-00072-of-00101.safetensors", + "model.layers.66.mlp.experts.46.up_proj.weight": "model-00072-of-00101.safetensors", + "model.layers.66.mlp.experts.46.down_proj.weight": "model-00072-of-00101.safetensors", + "model.layers.66.mlp.experts.47.gate_proj.weight": "model-00072-of-00101.safetensors", + "model.layers.66.mlp.experts.47.up_proj.weight": "model-00072-of-00101.safetensors", + "model.layers.66.mlp.experts.47.down_proj.weight": "model-00072-of-00101.safetensors", + "model.layers.66.mlp.experts.48.gate_proj.weight": "model-00072-of-00101.safetensors", + "model.layers.66.mlp.experts.48.up_proj.weight": "model-00072-of-00101.safetensors", + "model.layers.66.mlp.experts.48.down_proj.weight": "model-00072-of-00101.safetensors", + "model.layers.66.mlp.experts.49.gate_proj.weight": "model-00072-of-00101.safetensors", + "model.layers.66.mlp.experts.49.up_proj.weight": "model-00072-of-00101.safetensors", + "model.layers.66.mlp.experts.49.down_proj.weight": "model-00072-of-00101.safetensors", + "model.layers.66.mlp.experts.50.gate_proj.weight": "model-00072-of-00101.safetensors", + "model.layers.66.mlp.experts.50.up_proj.weight": "model-00072-of-00101.safetensors", + "model.layers.66.mlp.experts.50.down_proj.weight": "model-00072-of-00101.safetensors", + "model.layers.66.mlp.experts.51.gate_proj.weight": "model-00072-of-00101.safetensors", + "model.layers.66.mlp.experts.51.up_proj.weight": "model-00072-of-00101.safetensors", + "model.layers.66.mlp.experts.51.down_proj.weight": "model-00072-of-00101.safetensors", + "model.layers.66.mlp.experts.52.gate_proj.weight": "model-00072-of-00101.safetensors", + "model.layers.66.mlp.experts.52.up_proj.weight": "model-00072-of-00101.safetensors", + "model.layers.66.mlp.experts.52.down_proj.weight": "model-00072-of-00101.safetensors", + "model.layers.66.mlp.experts.53.gate_proj.weight": "model-00072-of-00101.safetensors", + "model.layers.66.mlp.experts.53.up_proj.weight": "model-00072-of-00101.safetensors", + "model.layers.66.mlp.experts.53.down_proj.weight": "model-00072-of-00101.safetensors", + "model.layers.66.mlp.experts.54.gate_proj.weight": "model-00072-of-00101.safetensors", + "model.layers.66.mlp.experts.54.up_proj.weight": "model-00072-of-00101.safetensors", + "model.layers.66.mlp.experts.54.down_proj.weight": "model-00072-of-00101.safetensors", + "model.layers.66.mlp.experts.55.gate_proj.weight": "model-00072-of-00101.safetensors", + "model.layers.66.mlp.experts.55.up_proj.weight": "model-00072-of-00101.safetensors", + "model.layers.66.mlp.experts.55.down_proj.weight": "model-00072-of-00101.safetensors", + "model.layers.66.mlp.experts.56.gate_proj.weight": "model-00072-of-00101.safetensors", + "model.layers.66.mlp.experts.56.up_proj.weight": "model-00072-of-00101.safetensors", + "model.layers.66.mlp.experts.56.down_proj.weight": "model-00072-of-00101.safetensors", + "model.layers.66.mlp.experts.57.gate_proj.weight": "model-00072-of-00101.safetensors", + "model.layers.66.mlp.experts.57.up_proj.weight": "model-00072-of-00101.safetensors", + "model.layers.66.mlp.experts.57.down_proj.weight": "model-00072-of-00101.safetensors", + "model.layers.66.mlp.experts.58.gate_proj.weight": "model-00072-of-00101.safetensors", + "model.layers.66.mlp.experts.58.up_proj.weight": "model-00072-of-00101.safetensors", + "model.layers.66.mlp.experts.58.down_proj.weight": "model-00072-of-00101.safetensors", + "model.layers.66.mlp.experts.59.gate_proj.weight": "model-00072-of-00101.safetensors", + "model.layers.66.mlp.experts.59.up_proj.weight": "model-00072-of-00101.safetensors", + "model.layers.66.mlp.experts.59.down_proj.weight": "model-00072-of-00101.safetensors", + "model.layers.66.mlp.experts.60.gate_proj.weight": "model-00072-of-00101.safetensors", + "model.layers.66.mlp.experts.60.up_proj.weight": "model-00072-of-00101.safetensors", + "model.layers.66.mlp.experts.60.down_proj.weight": "model-00072-of-00101.safetensors", + "model.layers.66.mlp.experts.61.gate_proj.weight": "model-00072-of-00101.safetensors", + "model.layers.66.mlp.experts.61.up_proj.weight": "model-00072-of-00101.safetensors", + "model.layers.66.mlp.experts.61.down_proj.weight": "model-00072-of-00101.safetensors", + "model.layers.66.mlp.experts.62.gate_proj.weight": "model-00072-of-00101.safetensors", + "model.layers.66.mlp.experts.62.up_proj.weight": "model-00072-of-00101.safetensors", + "model.layers.66.mlp.experts.62.down_proj.weight": "model-00072-of-00101.safetensors", + "model.layers.66.mlp.experts.63.gate_proj.weight": "model-00072-of-00101.safetensors", + "model.layers.66.mlp.experts.63.up_proj.weight": "model-00072-of-00101.safetensors", + "model.layers.66.mlp.experts.63.down_proj.weight": "model-00072-of-00101.safetensors", + "model.layers.66.mlp.experts.64.gate_proj.weight": "model-00072-of-00101.safetensors", + "model.layers.66.mlp.experts.64.up_proj.weight": "model-00072-of-00101.safetensors", + "model.layers.66.mlp.experts.64.down_proj.weight": "model-00072-of-00101.safetensors", + "model.layers.66.mlp.experts.65.gate_proj.weight": "model-00072-of-00101.safetensors", + "model.layers.66.mlp.experts.65.up_proj.weight": "model-00072-of-00101.safetensors", + "model.layers.66.mlp.experts.65.down_proj.weight": "model-00072-of-00101.safetensors", + "model.layers.66.mlp.experts.66.gate_proj.weight": "model-00072-of-00101.safetensors", + "model.layers.66.mlp.experts.66.up_proj.weight": "model-00072-of-00101.safetensors", + "model.layers.66.mlp.experts.66.down_proj.weight": "model-00072-of-00101.safetensors", + "model.layers.66.mlp.experts.67.gate_proj.weight": "model-00072-of-00101.safetensors", + "model.layers.66.mlp.experts.67.up_proj.weight": "model-00072-of-00101.safetensors", + "model.layers.66.mlp.experts.67.down_proj.weight": "model-00072-of-00101.safetensors", + "model.layers.66.mlp.experts.68.gate_proj.weight": "model-00072-of-00101.safetensors", + "model.layers.66.mlp.experts.68.up_proj.weight": "model-00072-of-00101.safetensors", + "model.layers.66.mlp.experts.68.down_proj.weight": "model-00072-of-00101.safetensors", + "model.layers.66.mlp.experts.69.gate_proj.weight": "model-00072-of-00101.safetensors", + "model.layers.66.mlp.experts.69.up_proj.weight": "model-00072-of-00101.safetensors", + "model.layers.66.mlp.experts.69.down_proj.weight": "model-00072-of-00101.safetensors", + "model.layers.66.mlp.experts.70.gate_proj.weight": "model-00072-of-00101.safetensors", + "model.layers.66.mlp.experts.70.up_proj.weight": "model-00072-of-00101.safetensors", + "model.layers.66.mlp.experts.70.down_proj.weight": "model-00072-of-00101.safetensors", + "model.layers.66.mlp.experts.71.gate_proj.weight": "model-00072-of-00101.safetensors", + "model.layers.66.mlp.experts.71.up_proj.weight": "model-00072-of-00101.safetensors", + "model.layers.66.mlp.experts.71.down_proj.weight": "model-00072-of-00101.safetensors", + "model.layers.66.mlp.experts.72.gate_proj.weight": "model-00072-of-00101.safetensors", + "model.layers.66.mlp.experts.72.up_proj.weight": "model-00072-of-00101.safetensors", + "model.layers.66.mlp.experts.72.down_proj.weight": "model-00072-of-00101.safetensors", + "model.layers.66.mlp.experts.73.gate_proj.weight": "model-00072-of-00101.safetensors", + "model.layers.66.mlp.experts.73.up_proj.weight": "model-00072-of-00101.safetensors", + "model.layers.66.mlp.experts.73.down_proj.weight": "model-00072-of-00101.safetensors", + "model.layers.66.mlp.experts.74.gate_proj.weight": "model-00072-of-00101.safetensors", + "model.layers.66.mlp.experts.74.up_proj.weight": "model-00072-of-00101.safetensors", + "model.layers.66.mlp.experts.74.down_proj.weight": "model-00072-of-00101.safetensors", + "model.layers.66.mlp.experts.75.gate_proj.weight": "model-00072-of-00101.safetensors", + "model.layers.66.mlp.experts.75.up_proj.weight": "model-00072-of-00101.safetensors", + "model.layers.66.mlp.experts.75.down_proj.weight": "model-00072-of-00101.safetensors", + "model.layers.66.mlp.experts.76.gate_proj.weight": "model-00072-of-00101.safetensors", + "model.layers.66.mlp.experts.76.up_proj.weight": "model-00072-of-00101.safetensors", + "model.layers.66.mlp.experts.76.down_proj.weight": "model-00072-of-00101.safetensors", + "model.layers.66.mlp.experts.77.gate_proj.weight": "model-00072-of-00101.safetensors", + "model.layers.66.mlp.experts.77.up_proj.weight": "model-00072-of-00101.safetensors", + "model.layers.66.mlp.experts.77.down_proj.weight": "model-00072-of-00101.safetensors", + "model.layers.66.mlp.experts.78.gate_proj.weight": "model-00072-of-00101.safetensors", + "model.layers.66.mlp.experts.78.up_proj.weight": "model-00072-of-00101.safetensors", + "model.layers.66.mlp.experts.78.down_proj.weight": "model-00072-of-00101.safetensors", + "model.layers.66.mlp.experts.79.gate_proj.weight": "model-00072-of-00101.safetensors", + "model.layers.66.mlp.experts.79.up_proj.weight": "model-00072-of-00101.safetensors", + "model.layers.66.mlp.experts.79.down_proj.weight": "model-00072-of-00101.safetensors", + "model.layers.66.mlp.experts.80.gate_proj.weight": "model-00072-of-00101.safetensors", + "model.layers.66.mlp.experts.80.up_proj.weight": "model-00072-of-00101.safetensors", + "model.layers.66.mlp.experts.80.down_proj.weight": "model-00072-of-00101.safetensors", + "model.layers.66.mlp.experts.81.gate_proj.weight": "model-00072-of-00101.safetensors", + "model.layers.66.mlp.experts.81.up_proj.weight": "model-00072-of-00101.safetensors", + "model.layers.66.mlp.experts.81.down_proj.weight": "model-00072-of-00101.safetensors", + "model.layers.66.mlp.experts.82.gate_proj.weight": "model-00072-of-00101.safetensors", + "model.layers.66.mlp.experts.82.up_proj.weight": "model-00072-of-00101.safetensors", + "model.layers.66.mlp.experts.82.down_proj.weight": "model-00072-of-00101.safetensors", + "model.layers.66.mlp.experts.83.gate_proj.weight": "model-00072-of-00101.safetensors", + "model.layers.66.mlp.experts.83.up_proj.weight": "model-00072-of-00101.safetensors", + "model.layers.66.mlp.experts.83.down_proj.weight": "model-00072-of-00101.safetensors", + "model.layers.66.mlp.experts.84.gate_proj.weight": "model-00072-of-00101.safetensors", + "model.layers.66.mlp.experts.84.up_proj.weight": "model-00072-of-00101.safetensors", + "model.layers.66.mlp.experts.84.down_proj.weight": "model-00072-of-00101.safetensors", + "model.layers.66.mlp.experts.85.gate_proj.weight": "model-00072-of-00101.safetensors", + "model.layers.66.mlp.experts.85.up_proj.weight": "model-00072-of-00101.safetensors", + "model.layers.66.mlp.experts.85.down_proj.weight": "model-00072-of-00101.safetensors", + "model.layers.66.mlp.experts.86.gate_proj.weight": "model-00072-of-00101.safetensors", + "model.layers.66.mlp.experts.86.up_proj.weight": "model-00072-of-00101.safetensors", + "model.layers.66.mlp.experts.86.down_proj.weight": "model-00072-of-00101.safetensors", + "model.layers.66.mlp.experts.87.gate_proj.weight": "model-00072-of-00101.safetensors", + "model.layers.66.mlp.experts.87.up_proj.weight": "model-00072-of-00101.safetensors", + "model.layers.66.mlp.experts.87.down_proj.weight": "model-00072-of-00101.safetensors", + "model.layers.66.mlp.experts.88.gate_proj.weight": "model-00072-of-00101.safetensors", + "model.layers.66.mlp.experts.88.up_proj.weight": "model-00072-of-00101.safetensors", + "model.layers.66.mlp.experts.88.down_proj.weight": "model-00072-of-00101.safetensors", + "model.layers.66.mlp.experts.89.gate_proj.weight": "model-00072-of-00101.safetensors", + "model.layers.66.mlp.experts.89.up_proj.weight": "model-00072-of-00101.safetensors", + "model.layers.66.mlp.experts.89.down_proj.weight": "model-00072-of-00101.safetensors", + "model.layers.66.mlp.experts.90.gate_proj.weight": "model-00072-of-00101.safetensors", + "model.layers.66.mlp.experts.90.up_proj.weight": "model-00072-of-00101.safetensors", + "model.layers.66.mlp.experts.90.down_proj.weight": "model-00072-of-00101.safetensors", + "model.layers.66.mlp.experts.91.gate_proj.weight": "model-00072-of-00101.safetensors", + "model.layers.66.mlp.experts.91.up_proj.weight": "model-00072-of-00101.safetensors", + "model.layers.66.mlp.experts.91.down_proj.weight": "model-00072-of-00101.safetensors", + "model.layers.66.mlp.experts.92.gate_proj.weight": "model-00072-of-00101.safetensors", + "model.layers.66.mlp.experts.92.up_proj.weight": "model-00072-of-00101.safetensors", + "model.layers.66.mlp.experts.92.down_proj.weight": "model-00072-of-00101.safetensors", + "model.layers.66.mlp.experts.93.gate_proj.weight": "model-00072-of-00101.safetensors", + "model.layers.66.mlp.experts.93.up_proj.weight": "model-00072-of-00101.safetensors", + "model.layers.66.mlp.experts.93.down_proj.weight": "model-00072-of-00101.safetensors", + "model.layers.66.mlp.experts.94.gate_proj.weight": "model-00072-of-00101.safetensors", + "model.layers.66.mlp.experts.94.up_proj.weight": "model-00072-of-00101.safetensors", + "model.layers.66.mlp.experts.94.down_proj.weight": "model-00072-of-00101.safetensors", + "model.layers.66.mlp.experts.95.gate_proj.weight": "model-00072-of-00101.safetensors", + "model.layers.66.mlp.experts.95.up_proj.weight": "model-00072-of-00101.safetensors", + "model.layers.66.mlp.experts.95.down_proj.weight": "model-00072-of-00101.safetensors", + "model.layers.66.mlp.experts.96.gate_proj.weight": "model-00072-of-00101.safetensors", + "model.layers.66.mlp.experts.96.up_proj.weight": "model-00072-of-00101.safetensors", + "model.layers.66.mlp.experts.96.down_proj.weight": "model-00072-of-00101.safetensors", + "model.layers.66.mlp.experts.97.gate_proj.weight": "model-00072-of-00101.safetensors", + "model.layers.66.mlp.experts.97.up_proj.weight": "model-00072-of-00101.safetensors", + "model.layers.66.mlp.experts.97.down_proj.weight": "model-00072-of-00101.safetensors", + "model.layers.66.mlp.experts.98.gate_proj.weight": "model-00072-of-00101.safetensors", + "model.layers.66.mlp.experts.98.up_proj.weight": "model-00072-of-00101.safetensors", + "model.layers.66.mlp.experts.98.down_proj.weight": "model-00072-of-00101.safetensors", + "model.layers.66.mlp.experts.99.gate_proj.weight": "model-00072-of-00101.safetensors", + "model.layers.66.mlp.experts.99.up_proj.weight": "model-00072-of-00101.safetensors", + "model.layers.66.mlp.experts.99.down_proj.weight": "model-00072-of-00101.safetensors", + "model.layers.66.mlp.experts.100.gate_proj.weight": "model-00072-of-00101.safetensors", + "model.layers.66.mlp.experts.100.up_proj.weight": "model-00072-of-00101.safetensors", + "model.layers.66.mlp.experts.100.down_proj.weight": "model-00072-of-00101.safetensors", + "model.layers.66.mlp.experts.101.gate_proj.weight": "model-00072-of-00101.safetensors", + "model.layers.66.mlp.experts.101.up_proj.weight": "model-00073-of-00101.safetensors", + "model.layers.66.mlp.experts.101.down_proj.weight": "model-00073-of-00101.safetensors", + "model.layers.66.mlp.experts.102.gate_proj.weight": "model-00073-of-00101.safetensors", + "model.layers.66.mlp.experts.102.up_proj.weight": "model-00073-of-00101.safetensors", + "model.layers.66.mlp.experts.102.down_proj.weight": "model-00073-of-00101.safetensors", + "model.layers.66.mlp.experts.103.gate_proj.weight": "model-00073-of-00101.safetensors", + "model.layers.66.mlp.experts.103.up_proj.weight": "model-00073-of-00101.safetensors", + "model.layers.66.mlp.experts.103.down_proj.weight": "model-00073-of-00101.safetensors", + "model.layers.66.mlp.experts.104.gate_proj.weight": "model-00073-of-00101.safetensors", + "model.layers.66.mlp.experts.104.up_proj.weight": "model-00073-of-00101.safetensors", + "model.layers.66.mlp.experts.104.down_proj.weight": "model-00073-of-00101.safetensors", + "model.layers.66.mlp.experts.105.gate_proj.weight": "model-00073-of-00101.safetensors", + "model.layers.66.mlp.experts.105.up_proj.weight": "model-00073-of-00101.safetensors", + "model.layers.66.mlp.experts.105.down_proj.weight": "model-00073-of-00101.safetensors", + "model.layers.66.mlp.experts.106.gate_proj.weight": "model-00073-of-00101.safetensors", + "model.layers.66.mlp.experts.106.up_proj.weight": "model-00073-of-00101.safetensors", + "model.layers.66.mlp.experts.106.down_proj.weight": "model-00073-of-00101.safetensors", + "model.layers.66.mlp.experts.107.gate_proj.weight": "model-00073-of-00101.safetensors", + "model.layers.66.mlp.experts.107.up_proj.weight": "model-00073-of-00101.safetensors", + "model.layers.66.mlp.experts.107.down_proj.weight": "model-00073-of-00101.safetensors", + "model.layers.66.mlp.experts.108.gate_proj.weight": "model-00073-of-00101.safetensors", + "model.layers.66.mlp.experts.108.up_proj.weight": "model-00073-of-00101.safetensors", + "model.layers.66.mlp.experts.108.down_proj.weight": "model-00073-of-00101.safetensors", + "model.layers.66.mlp.experts.109.gate_proj.weight": "model-00073-of-00101.safetensors", + "model.layers.66.mlp.experts.109.up_proj.weight": "model-00073-of-00101.safetensors", + "model.layers.66.mlp.experts.109.down_proj.weight": "model-00073-of-00101.safetensors", + "model.layers.66.mlp.experts.110.gate_proj.weight": "model-00073-of-00101.safetensors", + "model.layers.66.mlp.experts.110.up_proj.weight": "model-00073-of-00101.safetensors", + "model.layers.66.mlp.experts.110.down_proj.weight": "model-00073-of-00101.safetensors", + "model.layers.66.mlp.experts.111.gate_proj.weight": "model-00073-of-00101.safetensors", + "model.layers.66.mlp.experts.111.up_proj.weight": "model-00073-of-00101.safetensors", + "model.layers.66.mlp.experts.111.down_proj.weight": "model-00073-of-00101.safetensors", + "model.layers.66.mlp.experts.112.gate_proj.weight": "model-00073-of-00101.safetensors", + "model.layers.66.mlp.experts.112.up_proj.weight": "model-00073-of-00101.safetensors", + "model.layers.66.mlp.experts.112.down_proj.weight": "model-00073-of-00101.safetensors", + "model.layers.66.mlp.experts.113.gate_proj.weight": "model-00073-of-00101.safetensors", + "model.layers.66.mlp.experts.113.up_proj.weight": "model-00073-of-00101.safetensors", + "model.layers.66.mlp.experts.113.down_proj.weight": "model-00073-of-00101.safetensors", + "model.layers.66.mlp.experts.114.gate_proj.weight": "model-00073-of-00101.safetensors", + "model.layers.66.mlp.experts.114.up_proj.weight": "model-00073-of-00101.safetensors", + "model.layers.66.mlp.experts.114.down_proj.weight": "model-00073-of-00101.safetensors", + "model.layers.66.mlp.experts.115.gate_proj.weight": "model-00073-of-00101.safetensors", + "model.layers.66.mlp.experts.115.up_proj.weight": "model-00073-of-00101.safetensors", + "model.layers.66.mlp.experts.115.down_proj.weight": "model-00073-of-00101.safetensors", + "model.layers.66.mlp.experts.116.gate_proj.weight": "model-00073-of-00101.safetensors", + "model.layers.66.mlp.experts.116.up_proj.weight": "model-00073-of-00101.safetensors", + "model.layers.66.mlp.experts.116.down_proj.weight": "model-00073-of-00101.safetensors", + "model.layers.66.mlp.experts.117.gate_proj.weight": "model-00073-of-00101.safetensors", + "model.layers.66.mlp.experts.117.up_proj.weight": "model-00073-of-00101.safetensors", + "model.layers.66.mlp.experts.117.down_proj.weight": "model-00073-of-00101.safetensors", + "model.layers.66.mlp.experts.118.gate_proj.weight": "model-00073-of-00101.safetensors", + "model.layers.66.mlp.experts.118.up_proj.weight": "model-00073-of-00101.safetensors", + "model.layers.66.mlp.experts.118.down_proj.weight": "model-00073-of-00101.safetensors", + "model.layers.66.mlp.experts.119.gate_proj.weight": "model-00073-of-00101.safetensors", + "model.layers.66.mlp.experts.119.up_proj.weight": "model-00073-of-00101.safetensors", + "model.layers.66.mlp.experts.119.down_proj.weight": "model-00073-of-00101.safetensors", + "model.layers.66.mlp.gate.weight": "model-00073-of-00101.safetensors", + "model.layers.66.mlp.gate.e_score_correction_bias": "model-00073-of-00101.safetensors", + "model.layers.66.mlp.shared_experts.gate_proj.weight": "model-00073-of-00101.safetensors", + "model.layers.66.mlp.shared_experts.up_proj.weight": "model-00073-of-00101.safetensors", + "model.layers.66.mlp.shared_experts.down_proj.weight": "model-00073-of-00101.safetensors", + "model.layers.66.input_layernorm.weight": "model-00073-of-00101.safetensors", + "model.layers.66.post_attention_layernorm.weight": "model-00073-of-00101.safetensors", + "model.layers.67.self_attn.q_proj.weight": "model-00073-of-00101.safetensors", + "model.layers.67.self_attn.q_proj.bias": "model-00073-of-00101.safetensors", + "model.layers.67.self_attn.k_proj.weight": "model-00073-of-00101.safetensors", + "model.layers.67.self_attn.k_proj.bias": "model-00073-of-00101.safetensors", + "model.layers.67.self_attn.v_proj.weight": "model-00073-of-00101.safetensors", + "model.layers.67.self_attn.v_proj.bias": "model-00073-of-00101.safetensors", + "model.layers.67.self_attn.o_proj.weight": "model-00073-of-00101.safetensors", + "model.layers.67.self_attn.q_norm.weight": "model-00073-of-00101.safetensors", + "model.layers.67.self_attn.k_norm.weight": "model-00073-of-00101.safetensors", + "model.layers.67.mlp.experts.0.gate_proj.weight": "model-00073-of-00101.safetensors", + "model.layers.67.mlp.experts.0.up_proj.weight": "model-00073-of-00101.safetensors", + "model.layers.67.mlp.experts.0.down_proj.weight": "model-00073-of-00101.safetensors", + "model.layers.67.mlp.experts.1.gate_proj.weight": "model-00073-of-00101.safetensors", + "model.layers.67.mlp.experts.1.up_proj.weight": "model-00073-of-00101.safetensors", + "model.layers.67.mlp.experts.1.down_proj.weight": "model-00073-of-00101.safetensors", + "model.layers.67.mlp.experts.2.gate_proj.weight": "model-00073-of-00101.safetensors", + "model.layers.67.mlp.experts.2.up_proj.weight": "model-00073-of-00101.safetensors", + "model.layers.67.mlp.experts.2.down_proj.weight": "model-00073-of-00101.safetensors", + "model.layers.67.mlp.experts.3.gate_proj.weight": "model-00073-of-00101.safetensors", + "model.layers.67.mlp.experts.3.up_proj.weight": "model-00073-of-00101.safetensors", + "model.layers.67.mlp.experts.3.down_proj.weight": "model-00073-of-00101.safetensors", + "model.layers.67.mlp.experts.4.gate_proj.weight": "model-00073-of-00101.safetensors", + "model.layers.67.mlp.experts.4.up_proj.weight": "model-00073-of-00101.safetensors", + "model.layers.67.mlp.experts.4.down_proj.weight": "model-00073-of-00101.safetensors", + "model.layers.67.mlp.experts.5.gate_proj.weight": "model-00073-of-00101.safetensors", + "model.layers.67.mlp.experts.5.up_proj.weight": "model-00073-of-00101.safetensors", + "model.layers.67.mlp.experts.5.down_proj.weight": "model-00073-of-00101.safetensors", + "model.layers.67.mlp.experts.6.gate_proj.weight": "model-00073-of-00101.safetensors", + "model.layers.67.mlp.experts.6.up_proj.weight": "model-00073-of-00101.safetensors", + "model.layers.67.mlp.experts.6.down_proj.weight": "model-00073-of-00101.safetensors", + "model.layers.67.mlp.experts.7.gate_proj.weight": "model-00073-of-00101.safetensors", + "model.layers.67.mlp.experts.7.up_proj.weight": "model-00073-of-00101.safetensors", + "model.layers.67.mlp.experts.7.down_proj.weight": "model-00073-of-00101.safetensors", + "model.layers.67.mlp.experts.8.gate_proj.weight": "model-00073-of-00101.safetensors", + "model.layers.67.mlp.experts.8.up_proj.weight": "model-00073-of-00101.safetensors", + "model.layers.67.mlp.experts.8.down_proj.weight": "model-00073-of-00101.safetensors", + "model.layers.67.mlp.experts.9.gate_proj.weight": "model-00073-of-00101.safetensors", + "model.layers.67.mlp.experts.9.up_proj.weight": "model-00073-of-00101.safetensors", + "model.layers.67.mlp.experts.9.down_proj.weight": "model-00073-of-00101.safetensors", + "model.layers.67.mlp.experts.10.gate_proj.weight": "model-00073-of-00101.safetensors", + "model.layers.67.mlp.experts.10.up_proj.weight": "model-00073-of-00101.safetensors", + "model.layers.67.mlp.experts.10.down_proj.weight": "model-00073-of-00101.safetensors", + "model.layers.67.mlp.experts.11.gate_proj.weight": "model-00073-of-00101.safetensors", + "model.layers.67.mlp.experts.11.up_proj.weight": "model-00073-of-00101.safetensors", + "model.layers.67.mlp.experts.11.down_proj.weight": "model-00073-of-00101.safetensors", + "model.layers.67.mlp.experts.12.gate_proj.weight": "model-00073-of-00101.safetensors", + "model.layers.67.mlp.experts.12.up_proj.weight": "model-00073-of-00101.safetensors", + "model.layers.67.mlp.experts.12.down_proj.weight": "model-00073-of-00101.safetensors", + "model.layers.67.mlp.experts.13.gate_proj.weight": "model-00073-of-00101.safetensors", + "model.layers.67.mlp.experts.13.up_proj.weight": "model-00073-of-00101.safetensors", + "model.layers.67.mlp.experts.13.down_proj.weight": "model-00073-of-00101.safetensors", + "model.layers.67.mlp.experts.14.gate_proj.weight": "model-00073-of-00101.safetensors", + "model.layers.67.mlp.experts.14.up_proj.weight": "model-00073-of-00101.safetensors", + "model.layers.67.mlp.experts.14.down_proj.weight": "model-00073-of-00101.safetensors", + "model.layers.67.mlp.experts.15.gate_proj.weight": "model-00073-of-00101.safetensors", + "model.layers.67.mlp.experts.15.up_proj.weight": "model-00073-of-00101.safetensors", + "model.layers.67.mlp.experts.15.down_proj.weight": "model-00073-of-00101.safetensors", + "model.layers.67.mlp.experts.16.gate_proj.weight": "model-00073-of-00101.safetensors", + "model.layers.67.mlp.experts.16.up_proj.weight": "model-00073-of-00101.safetensors", + "model.layers.67.mlp.experts.16.down_proj.weight": "model-00073-of-00101.safetensors", + "model.layers.67.mlp.experts.17.gate_proj.weight": "model-00073-of-00101.safetensors", + "model.layers.67.mlp.experts.17.up_proj.weight": "model-00073-of-00101.safetensors", + "model.layers.67.mlp.experts.17.down_proj.weight": "model-00073-of-00101.safetensors", + "model.layers.67.mlp.experts.18.gate_proj.weight": "model-00073-of-00101.safetensors", + "model.layers.67.mlp.experts.18.up_proj.weight": "model-00073-of-00101.safetensors", + "model.layers.67.mlp.experts.18.down_proj.weight": "model-00073-of-00101.safetensors", + "model.layers.67.mlp.experts.19.gate_proj.weight": "model-00073-of-00101.safetensors", + "model.layers.67.mlp.experts.19.up_proj.weight": "model-00073-of-00101.safetensors", + "model.layers.67.mlp.experts.19.down_proj.weight": "model-00073-of-00101.safetensors", + "model.layers.67.mlp.experts.20.gate_proj.weight": "model-00073-of-00101.safetensors", + "model.layers.67.mlp.experts.20.up_proj.weight": "model-00073-of-00101.safetensors", + "model.layers.67.mlp.experts.20.down_proj.weight": "model-00073-of-00101.safetensors", + "model.layers.67.mlp.experts.21.gate_proj.weight": "model-00073-of-00101.safetensors", + "model.layers.67.mlp.experts.21.up_proj.weight": "model-00073-of-00101.safetensors", + "model.layers.67.mlp.experts.21.down_proj.weight": "model-00073-of-00101.safetensors", + "model.layers.67.mlp.experts.22.gate_proj.weight": "model-00073-of-00101.safetensors", + "model.layers.67.mlp.experts.22.up_proj.weight": "model-00073-of-00101.safetensors", + "model.layers.67.mlp.experts.22.down_proj.weight": "model-00073-of-00101.safetensors", + "model.layers.67.mlp.experts.23.gate_proj.weight": "model-00073-of-00101.safetensors", + "model.layers.67.mlp.experts.23.up_proj.weight": "model-00073-of-00101.safetensors", + "model.layers.67.mlp.experts.23.down_proj.weight": "model-00073-of-00101.safetensors", + "model.layers.67.mlp.experts.24.gate_proj.weight": "model-00073-of-00101.safetensors", + "model.layers.67.mlp.experts.24.up_proj.weight": "model-00073-of-00101.safetensors", + "model.layers.67.mlp.experts.24.down_proj.weight": "model-00073-of-00101.safetensors", + "model.layers.67.mlp.experts.25.gate_proj.weight": "model-00073-of-00101.safetensors", + "model.layers.67.mlp.experts.25.up_proj.weight": "model-00073-of-00101.safetensors", + "model.layers.67.mlp.experts.25.down_proj.weight": "model-00073-of-00101.safetensors", + "model.layers.67.mlp.experts.26.gate_proj.weight": "model-00073-of-00101.safetensors", + "model.layers.67.mlp.experts.26.up_proj.weight": "model-00073-of-00101.safetensors", + "model.layers.67.mlp.experts.26.down_proj.weight": "model-00073-of-00101.safetensors", + "model.layers.67.mlp.experts.27.gate_proj.weight": "model-00073-of-00101.safetensors", + "model.layers.67.mlp.experts.27.up_proj.weight": "model-00073-of-00101.safetensors", + "model.layers.67.mlp.experts.27.down_proj.weight": "model-00073-of-00101.safetensors", + "model.layers.67.mlp.experts.28.gate_proj.weight": "model-00073-of-00101.safetensors", + "model.layers.67.mlp.experts.28.up_proj.weight": "model-00073-of-00101.safetensors", + "model.layers.67.mlp.experts.28.down_proj.weight": "model-00073-of-00101.safetensors", + "model.layers.67.mlp.experts.29.gate_proj.weight": "model-00073-of-00101.safetensors", + "model.layers.67.mlp.experts.29.up_proj.weight": "model-00073-of-00101.safetensors", + "model.layers.67.mlp.experts.29.down_proj.weight": "model-00073-of-00101.safetensors", + "model.layers.67.mlp.experts.30.gate_proj.weight": "model-00073-of-00101.safetensors", + "model.layers.67.mlp.experts.30.up_proj.weight": "model-00073-of-00101.safetensors", + "model.layers.67.mlp.experts.30.down_proj.weight": "model-00073-of-00101.safetensors", + "model.layers.67.mlp.experts.31.gate_proj.weight": "model-00073-of-00101.safetensors", + "model.layers.67.mlp.experts.31.up_proj.weight": "model-00073-of-00101.safetensors", + "model.layers.67.mlp.experts.31.down_proj.weight": "model-00073-of-00101.safetensors", + "model.layers.67.mlp.experts.32.gate_proj.weight": "model-00073-of-00101.safetensors", + "model.layers.67.mlp.experts.32.up_proj.weight": "model-00073-of-00101.safetensors", + "model.layers.67.mlp.experts.32.down_proj.weight": "model-00073-of-00101.safetensors", + "model.layers.67.mlp.experts.33.gate_proj.weight": "model-00073-of-00101.safetensors", + "model.layers.67.mlp.experts.33.up_proj.weight": "model-00073-of-00101.safetensors", + "model.layers.67.mlp.experts.33.down_proj.weight": "model-00073-of-00101.safetensors", + "model.layers.67.mlp.experts.34.gate_proj.weight": "model-00073-of-00101.safetensors", + "model.layers.67.mlp.experts.34.up_proj.weight": "model-00073-of-00101.safetensors", + "model.layers.67.mlp.experts.34.down_proj.weight": "model-00073-of-00101.safetensors", + "model.layers.67.mlp.experts.35.gate_proj.weight": "model-00073-of-00101.safetensors", + "model.layers.67.mlp.experts.35.up_proj.weight": "model-00073-of-00101.safetensors", + "model.layers.67.mlp.experts.35.down_proj.weight": "model-00073-of-00101.safetensors", + "model.layers.67.mlp.experts.36.gate_proj.weight": "model-00073-of-00101.safetensors", + "model.layers.67.mlp.experts.36.up_proj.weight": "model-00073-of-00101.safetensors", + "model.layers.67.mlp.experts.36.down_proj.weight": "model-00073-of-00101.safetensors", + "model.layers.67.mlp.experts.37.gate_proj.weight": "model-00073-of-00101.safetensors", + "model.layers.67.mlp.experts.37.up_proj.weight": "model-00073-of-00101.safetensors", + "model.layers.67.mlp.experts.37.down_proj.weight": "model-00073-of-00101.safetensors", + "model.layers.67.mlp.experts.38.gate_proj.weight": "model-00073-of-00101.safetensors", + "model.layers.67.mlp.experts.38.up_proj.weight": "model-00073-of-00101.safetensors", + "model.layers.67.mlp.experts.38.down_proj.weight": "model-00073-of-00101.safetensors", + "model.layers.67.mlp.experts.39.gate_proj.weight": "model-00073-of-00101.safetensors", + "model.layers.67.mlp.experts.39.up_proj.weight": "model-00073-of-00101.safetensors", + "model.layers.67.mlp.experts.39.down_proj.weight": "model-00073-of-00101.safetensors", + "model.layers.67.mlp.experts.40.gate_proj.weight": "model-00073-of-00101.safetensors", + "model.layers.67.mlp.experts.40.up_proj.weight": "model-00073-of-00101.safetensors", + "model.layers.67.mlp.experts.40.down_proj.weight": "model-00073-of-00101.safetensors", + "model.layers.67.mlp.experts.41.gate_proj.weight": "model-00073-of-00101.safetensors", + "model.layers.67.mlp.experts.41.up_proj.weight": "model-00073-of-00101.safetensors", + "model.layers.67.mlp.experts.41.down_proj.weight": "model-00073-of-00101.safetensors", + "model.layers.67.mlp.experts.42.gate_proj.weight": "model-00073-of-00101.safetensors", + "model.layers.67.mlp.experts.42.up_proj.weight": "model-00073-of-00101.safetensors", + "model.layers.67.mlp.experts.42.down_proj.weight": "model-00073-of-00101.safetensors", + "model.layers.67.mlp.experts.43.gate_proj.weight": "model-00073-of-00101.safetensors", + "model.layers.67.mlp.experts.43.up_proj.weight": "model-00073-of-00101.safetensors", + "model.layers.67.mlp.experts.43.down_proj.weight": "model-00073-of-00101.safetensors", + "model.layers.67.mlp.experts.44.gate_proj.weight": "model-00073-of-00101.safetensors", + "model.layers.67.mlp.experts.44.up_proj.weight": "model-00073-of-00101.safetensors", + "model.layers.67.mlp.experts.44.down_proj.weight": "model-00073-of-00101.safetensors", + "model.layers.67.mlp.experts.45.gate_proj.weight": "model-00073-of-00101.safetensors", + "model.layers.67.mlp.experts.45.up_proj.weight": "model-00073-of-00101.safetensors", + "model.layers.67.mlp.experts.45.down_proj.weight": "model-00073-of-00101.safetensors", + "model.layers.67.mlp.experts.46.gate_proj.weight": "model-00073-of-00101.safetensors", + "model.layers.67.mlp.experts.46.up_proj.weight": "model-00073-of-00101.safetensors", + "model.layers.67.mlp.experts.46.down_proj.weight": "model-00073-of-00101.safetensors", + "model.layers.67.mlp.experts.47.gate_proj.weight": "model-00073-of-00101.safetensors", + "model.layers.67.mlp.experts.47.up_proj.weight": "model-00073-of-00101.safetensors", + "model.layers.67.mlp.experts.47.down_proj.weight": "model-00073-of-00101.safetensors", + "model.layers.67.mlp.experts.48.gate_proj.weight": "model-00073-of-00101.safetensors", + "model.layers.67.mlp.experts.48.up_proj.weight": "model-00073-of-00101.safetensors", + "model.layers.67.mlp.experts.48.down_proj.weight": "model-00073-of-00101.safetensors", + "model.layers.67.mlp.experts.49.gate_proj.weight": "model-00073-of-00101.safetensors", + "model.layers.67.mlp.experts.49.up_proj.weight": "model-00073-of-00101.safetensors", + "model.layers.67.mlp.experts.49.down_proj.weight": "model-00073-of-00101.safetensors", + "model.layers.67.mlp.experts.50.gate_proj.weight": "model-00073-of-00101.safetensors", + "model.layers.67.mlp.experts.50.up_proj.weight": "model-00073-of-00101.safetensors", + "model.layers.67.mlp.experts.50.down_proj.weight": "model-00073-of-00101.safetensors", + "model.layers.67.mlp.experts.51.gate_proj.weight": "model-00073-of-00101.safetensors", + "model.layers.67.mlp.experts.51.up_proj.weight": "model-00073-of-00101.safetensors", + "model.layers.67.mlp.experts.51.down_proj.weight": "model-00073-of-00101.safetensors", + "model.layers.67.mlp.experts.52.gate_proj.weight": "model-00073-of-00101.safetensors", + "model.layers.67.mlp.experts.52.up_proj.weight": "model-00073-of-00101.safetensors", + "model.layers.67.mlp.experts.52.down_proj.weight": "model-00073-of-00101.safetensors", + "model.layers.67.mlp.experts.53.gate_proj.weight": "model-00073-of-00101.safetensors", + "model.layers.67.mlp.experts.53.up_proj.weight": "model-00073-of-00101.safetensors", + "model.layers.67.mlp.experts.53.down_proj.weight": "model-00073-of-00101.safetensors", + "model.layers.67.mlp.experts.54.gate_proj.weight": "model-00073-of-00101.safetensors", + "model.layers.67.mlp.experts.54.up_proj.weight": "model-00073-of-00101.safetensors", + "model.layers.67.mlp.experts.54.down_proj.weight": "model-00073-of-00101.safetensors", + "model.layers.67.mlp.experts.55.gate_proj.weight": "model-00073-of-00101.safetensors", + "model.layers.67.mlp.experts.55.up_proj.weight": "model-00073-of-00101.safetensors", + "model.layers.67.mlp.experts.55.down_proj.weight": "model-00073-of-00101.safetensors", + "model.layers.67.mlp.experts.56.gate_proj.weight": "model-00073-of-00101.safetensors", + "model.layers.67.mlp.experts.56.up_proj.weight": "model-00073-of-00101.safetensors", + "model.layers.67.mlp.experts.56.down_proj.weight": "model-00073-of-00101.safetensors", + "model.layers.67.mlp.experts.57.gate_proj.weight": "model-00073-of-00101.safetensors", + "model.layers.67.mlp.experts.57.up_proj.weight": "model-00073-of-00101.safetensors", + "model.layers.67.mlp.experts.57.down_proj.weight": "model-00073-of-00101.safetensors", + "model.layers.67.mlp.experts.58.gate_proj.weight": "model-00073-of-00101.safetensors", + "model.layers.67.mlp.experts.58.up_proj.weight": "model-00073-of-00101.safetensors", + "model.layers.67.mlp.experts.58.down_proj.weight": "model-00073-of-00101.safetensors", + "model.layers.67.mlp.experts.59.gate_proj.weight": "model-00073-of-00101.safetensors", + "model.layers.67.mlp.experts.59.up_proj.weight": "model-00073-of-00101.safetensors", + "model.layers.67.mlp.experts.59.down_proj.weight": "model-00073-of-00101.safetensors", + "model.layers.67.mlp.experts.60.gate_proj.weight": "model-00073-of-00101.safetensors", + "model.layers.67.mlp.experts.60.up_proj.weight": "model-00073-of-00101.safetensors", + "model.layers.67.mlp.experts.60.down_proj.weight": "model-00073-of-00101.safetensors", + "model.layers.67.mlp.experts.61.gate_proj.weight": "model-00073-of-00101.safetensors", + "model.layers.67.mlp.experts.61.up_proj.weight": "model-00073-of-00101.safetensors", + "model.layers.67.mlp.experts.61.down_proj.weight": "model-00073-of-00101.safetensors", + "model.layers.67.mlp.experts.62.gate_proj.weight": "model-00073-of-00101.safetensors", + "model.layers.67.mlp.experts.62.up_proj.weight": "model-00073-of-00101.safetensors", + "model.layers.67.mlp.experts.62.down_proj.weight": "model-00073-of-00101.safetensors", + "model.layers.67.mlp.experts.63.gate_proj.weight": "model-00073-of-00101.safetensors", + "model.layers.67.mlp.experts.63.up_proj.weight": "model-00073-of-00101.safetensors", + "model.layers.67.mlp.experts.63.down_proj.weight": "model-00073-of-00101.safetensors", + "model.layers.67.mlp.experts.64.gate_proj.weight": "model-00073-of-00101.safetensors", + "model.layers.67.mlp.experts.64.up_proj.weight": "model-00073-of-00101.safetensors", + "model.layers.67.mlp.experts.64.down_proj.weight": "model-00073-of-00101.safetensors", + "model.layers.67.mlp.experts.65.gate_proj.weight": "model-00073-of-00101.safetensors", + "model.layers.67.mlp.experts.65.up_proj.weight": "model-00073-of-00101.safetensors", + "model.layers.67.mlp.experts.65.down_proj.weight": "model-00073-of-00101.safetensors", + "model.layers.67.mlp.experts.66.gate_proj.weight": "model-00073-of-00101.safetensors", + "model.layers.67.mlp.experts.66.up_proj.weight": "model-00073-of-00101.safetensors", + "model.layers.67.mlp.experts.66.down_proj.weight": "model-00073-of-00101.safetensors", + "model.layers.67.mlp.experts.67.gate_proj.weight": "model-00073-of-00101.safetensors", + "model.layers.67.mlp.experts.67.up_proj.weight": "model-00073-of-00101.safetensors", + "model.layers.67.mlp.experts.67.down_proj.weight": "model-00073-of-00101.safetensors", + "model.layers.67.mlp.experts.68.gate_proj.weight": "model-00073-of-00101.safetensors", + "model.layers.67.mlp.experts.68.up_proj.weight": "model-00073-of-00101.safetensors", + "model.layers.67.mlp.experts.68.down_proj.weight": "model-00073-of-00101.safetensors", + "model.layers.67.mlp.experts.69.gate_proj.weight": "model-00073-of-00101.safetensors", + "model.layers.67.mlp.experts.69.up_proj.weight": "model-00073-of-00101.safetensors", + "model.layers.67.mlp.experts.69.down_proj.weight": "model-00073-of-00101.safetensors", + "model.layers.67.mlp.experts.70.gate_proj.weight": "model-00073-of-00101.safetensors", + "model.layers.67.mlp.experts.70.up_proj.weight": "model-00073-of-00101.safetensors", + "model.layers.67.mlp.experts.70.down_proj.weight": "model-00073-of-00101.safetensors", + "model.layers.67.mlp.experts.71.gate_proj.weight": "model-00073-of-00101.safetensors", + "model.layers.67.mlp.experts.71.up_proj.weight": "model-00073-of-00101.safetensors", + "model.layers.67.mlp.experts.71.down_proj.weight": "model-00073-of-00101.safetensors", + "model.layers.67.mlp.experts.72.gate_proj.weight": "model-00073-of-00101.safetensors", + "model.layers.67.mlp.experts.72.up_proj.weight": "model-00073-of-00101.safetensors", + "model.layers.67.mlp.experts.72.down_proj.weight": "model-00073-of-00101.safetensors", + "model.layers.67.mlp.experts.73.gate_proj.weight": "model-00073-of-00101.safetensors", + "model.layers.67.mlp.experts.73.up_proj.weight": "model-00073-of-00101.safetensors", + "model.layers.67.mlp.experts.73.down_proj.weight": "model-00073-of-00101.safetensors", + "model.layers.67.mlp.experts.74.gate_proj.weight": "model-00073-of-00101.safetensors", + "model.layers.67.mlp.experts.74.up_proj.weight": "model-00073-of-00101.safetensors", + "model.layers.67.mlp.experts.74.down_proj.weight": "model-00073-of-00101.safetensors", + "model.layers.67.mlp.experts.75.gate_proj.weight": "model-00073-of-00101.safetensors", + "model.layers.67.mlp.experts.75.up_proj.weight": "model-00073-of-00101.safetensors", + "model.layers.67.mlp.experts.75.down_proj.weight": "model-00073-of-00101.safetensors", + "model.layers.67.mlp.experts.76.gate_proj.weight": "model-00073-of-00101.safetensors", + "model.layers.67.mlp.experts.76.up_proj.weight": "model-00073-of-00101.safetensors", + "model.layers.67.mlp.experts.76.down_proj.weight": "model-00073-of-00101.safetensors", + "model.layers.67.mlp.experts.77.gate_proj.weight": "model-00073-of-00101.safetensors", + "model.layers.67.mlp.experts.77.up_proj.weight": "model-00073-of-00101.safetensors", + "model.layers.67.mlp.experts.77.down_proj.weight": "model-00073-of-00101.safetensors", + "model.layers.67.mlp.experts.78.gate_proj.weight": "model-00073-of-00101.safetensors", + "model.layers.67.mlp.experts.78.up_proj.weight": "model-00073-of-00101.safetensors", + "model.layers.67.mlp.experts.78.down_proj.weight": "model-00073-of-00101.safetensors", + "model.layers.67.mlp.experts.79.gate_proj.weight": "model-00073-of-00101.safetensors", + "model.layers.67.mlp.experts.79.up_proj.weight": "model-00073-of-00101.safetensors", + "model.layers.67.mlp.experts.79.down_proj.weight": "model-00073-of-00101.safetensors", + "model.layers.67.mlp.experts.80.gate_proj.weight": "model-00073-of-00101.safetensors", + "model.layers.67.mlp.experts.80.up_proj.weight": "model-00073-of-00101.safetensors", + "model.layers.67.mlp.experts.80.down_proj.weight": "model-00073-of-00101.safetensors", + "model.layers.67.mlp.experts.81.gate_proj.weight": "model-00073-of-00101.safetensors", + "model.layers.67.mlp.experts.81.up_proj.weight": "model-00073-of-00101.safetensors", + "model.layers.67.mlp.experts.81.down_proj.weight": "model-00073-of-00101.safetensors", + "model.layers.67.mlp.experts.82.gate_proj.weight": "model-00073-of-00101.safetensors", + "model.layers.67.mlp.experts.82.up_proj.weight": "model-00073-of-00101.safetensors", + "model.layers.67.mlp.experts.82.down_proj.weight": "model-00073-of-00101.safetensors", + "model.layers.67.mlp.experts.83.gate_proj.weight": "model-00073-of-00101.safetensors", + "model.layers.67.mlp.experts.83.up_proj.weight": "model-00073-of-00101.safetensors", + "model.layers.67.mlp.experts.83.down_proj.weight": "model-00073-of-00101.safetensors", + "model.layers.67.mlp.experts.84.gate_proj.weight": "model-00073-of-00101.safetensors", + "model.layers.67.mlp.experts.84.up_proj.weight": "model-00073-of-00101.safetensors", + "model.layers.67.mlp.experts.84.down_proj.weight": "model-00073-of-00101.safetensors", + "model.layers.67.mlp.experts.85.gate_proj.weight": "model-00073-of-00101.safetensors", + "model.layers.67.mlp.experts.85.up_proj.weight": "model-00073-of-00101.safetensors", + "model.layers.67.mlp.experts.85.down_proj.weight": "model-00073-of-00101.safetensors", + "model.layers.67.mlp.experts.86.gate_proj.weight": "model-00073-of-00101.safetensors", + "model.layers.67.mlp.experts.86.up_proj.weight": "model-00073-of-00101.safetensors", + "model.layers.67.mlp.experts.86.down_proj.weight": "model-00073-of-00101.safetensors", + "model.layers.67.mlp.experts.87.gate_proj.weight": "model-00073-of-00101.safetensors", + "model.layers.67.mlp.experts.87.up_proj.weight": "model-00073-of-00101.safetensors", + "model.layers.67.mlp.experts.87.down_proj.weight": "model-00073-of-00101.safetensors", + "model.layers.67.mlp.experts.88.gate_proj.weight": "model-00074-of-00101.safetensors", + "model.layers.67.mlp.experts.88.up_proj.weight": "model-00074-of-00101.safetensors", + "model.layers.67.mlp.experts.88.down_proj.weight": "model-00074-of-00101.safetensors", + "model.layers.67.mlp.experts.89.gate_proj.weight": "model-00074-of-00101.safetensors", + "model.layers.67.mlp.experts.89.up_proj.weight": "model-00074-of-00101.safetensors", + "model.layers.67.mlp.experts.89.down_proj.weight": "model-00074-of-00101.safetensors", + "model.layers.67.mlp.experts.90.gate_proj.weight": "model-00074-of-00101.safetensors", + "model.layers.67.mlp.experts.90.up_proj.weight": "model-00074-of-00101.safetensors", + "model.layers.67.mlp.experts.90.down_proj.weight": "model-00074-of-00101.safetensors", + "model.layers.67.mlp.experts.91.gate_proj.weight": "model-00074-of-00101.safetensors", + "model.layers.67.mlp.experts.91.up_proj.weight": "model-00074-of-00101.safetensors", + "model.layers.67.mlp.experts.91.down_proj.weight": "model-00074-of-00101.safetensors", + "model.layers.67.mlp.experts.92.gate_proj.weight": "model-00074-of-00101.safetensors", + "model.layers.67.mlp.experts.92.up_proj.weight": "model-00074-of-00101.safetensors", + "model.layers.67.mlp.experts.92.down_proj.weight": "model-00074-of-00101.safetensors", + "model.layers.67.mlp.experts.93.gate_proj.weight": "model-00074-of-00101.safetensors", + "model.layers.67.mlp.experts.93.up_proj.weight": "model-00074-of-00101.safetensors", + "model.layers.67.mlp.experts.93.down_proj.weight": "model-00074-of-00101.safetensors", + "model.layers.67.mlp.experts.94.gate_proj.weight": "model-00074-of-00101.safetensors", + "model.layers.67.mlp.experts.94.up_proj.weight": "model-00074-of-00101.safetensors", + "model.layers.67.mlp.experts.94.down_proj.weight": "model-00074-of-00101.safetensors", + "model.layers.67.mlp.experts.95.gate_proj.weight": "model-00074-of-00101.safetensors", + "model.layers.67.mlp.experts.95.up_proj.weight": "model-00074-of-00101.safetensors", + "model.layers.67.mlp.experts.95.down_proj.weight": "model-00074-of-00101.safetensors", + "model.layers.67.mlp.experts.96.gate_proj.weight": "model-00074-of-00101.safetensors", + "model.layers.67.mlp.experts.96.up_proj.weight": "model-00074-of-00101.safetensors", + "model.layers.67.mlp.experts.96.down_proj.weight": "model-00074-of-00101.safetensors", + "model.layers.67.mlp.experts.97.gate_proj.weight": "model-00074-of-00101.safetensors", + "model.layers.67.mlp.experts.97.up_proj.weight": "model-00074-of-00101.safetensors", + "model.layers.67.mlp.experts.97.down_proj.weight": "model-00074-of-00101.safetensors", + "model.layers.67.mlp.experts.98.gate_proj.weight": "model-00074-of-00101.safetensors", + "model.layers.67.mlp.experts.98.up_proj.weight": "model-00074-of-00101.safetensors", + "model.layers.67.mlp.experts.98.down_proj.weight": "model-00074-of-00101.safetensors", + "model.layers.67.mlp.experts.99.gate_proj.weight": "model-00074-of-00101.safetensors", + "model.layers.67.mlp.experts.99.up_proj.weight": "model-00074-of-00101.safetensors", + "model.layers.67.mlp.experts.99.down_proj.weight": "model-00074-of-00101.safetensors", + "model.layers.67.mlp.experts.100.gate_proj.weight": "model-00074-of-00101.safetensors", + "model.layers.67.mlp.experts.100.up_proj.weight": "model-00074-of-00101.safetensors", + "model.layers.67.mlp.experts.100.down_proj.weight": "model-00074-of-00101.safetensors", + "model.layers.67.mlp.experts.101.gate_proj.weight": "model-00074-of-00101.safetensors", + "model.layers.67.mlp.experts.101.up_proj.weight": "model-00074-of-00101.safetensors", + "model.layers.67.mlp.experts.101.down_proj.weight": "model-00074-of-00101.safetensors", + "model.layers.67.mlp.experts.102.gate_proj.weight": "model-00074-of-00101.safetensors", + "model.layers.67.mlp.experts.102.up_proj.weight": "model-00074-of-00101.safetensors", + "model.layers.67.mlp.experts.102.down_proj.weight": "model-00074-of-00101.safetensors", + "model.layers.67.mlp.experts.103.gate_proj.weight": "model-00074-of-00101.safetensors", + "model.layers.67.mlp.experts.103.up_proj.weight": "model-00074-of-00101.safetensors", + "model.layers.67.mlp.experts.103.down_proj.weight": "model-00074-of-00101.safetensors", + "model.layers.67.mlp.experts.104.gate_proj.weight": "model-00074-of-00101.safetensors", + "model.layers.67.mlp.experts.104.up_proj.weight": "model-00074-of-00101.safetensors", + "model.layers.67.mlp.experts.104.down_proj.weight": "model-00074-of-00101.safetensors", + "model.layers.67.mlp.experts.105.gate_proj.weight": "model-00074-of-00101.safetensors", + "model.layers.67.mlp.experts.105.up_proj.weight": "model-00074-of-00101.safetensors", + "model.layers.67.mlp.experts.105.down_proj.weight": "model-00074-of-00101.safetensors", + "model.layers.67.mlp.experts.106.gate_proj.weight": "model-00074-of-00101.safetensors", + "model.layers.67.mlp.experts.106.up_proj.weight": "model-00074-of-00101.safetensors", + "model.layers.67.mlp.experts.106.down_proj.weight": "model-00074-of-00101.safetensors", + "model.layers.67.mlp.experts.107.gate_proj.weight": "model-00074-of-00101.safetensors", + "model.layers.67.mlp.experts.107.up_proj.weight": "model-00074-of-00101.safetensors", + "model.layers.67.mlp.experts.107.down_proj.weight": "model-00074-of-00101.safetensors", + "model.layers.67.mlp.experts.108.gate_proj.weight": "model-00074-of-00101.safetensors", + "model.layers.67.mlp.experts.108.up_proj.weight": "model-00074-of-00101.safetensors", + "model.layers.67.mlp.experts.108.down_proj.weight": "model-00074-of-00101.safetensors", + "model.layers.67.mlp.experts.109.gate_proj.weight": "model-00074-of-00101.safetensors", + "model.layers.67.mlp.experts.109.up_proj.weight": "model-00074-of-00101.safetensors", + "model.layers.67.mlp.experts.109.down_proj.weight": "model-00074-of-00101.safetensors", + "model.layers.67.mlp.experts.110.gate_proj.weight": "model-00074-of-00101.safetensors", + "model.layers.67.mlp.experts.110.up_proj.weight": "model-00074-of-00101.safetensors", + "model.layers.67.mlp.experts.110.down_proj.weight": "model-00074-of-00101.safetensors", + "model.layers.67.mlp.experts.111.gate_proj.weight": "model-00074-of-00101.safetensors", + "model.layers.67.mlp.experts.111.up_proj.weight": "model-00074-of-00101.safetensors", + "model.layers.67.mlp.experts.111.down_proj.weight": "model-00074-of-00101.safetensors", + "model.layers.67.mlp.experts.112.gate_proj.weight": "model-00074-of-00101.safetensors", + "model.layers.67.mlp.experts.112.up_proj.weight": "model-00074-of-00101.safetensors", + "model.layers.67.mlp.experts.112.down_proj.weight": "model-00074-of-00101.safetensors", + "model.layers.67.mlp.experts.113.gate_proj.weight": "model-00074-of-00101.safetensors", + "model.layers.67.mlp.experts.113.up_proj.weight": "model-00074-of-00101.safetensors", + "model.layers.67.mlp.experts.113.down_proj.weight": "model-00074-of-00101.safetensors", + "model.layers.67.mlp.experts.114.gate_proj.weight": "model-00074-of-00101.safetensors", + "model.layers.67.mlp.experts.114.up_proj.weight": "model-00074-of-00101.safetensors", + "model.layers.67.mlp.experts.114.down_proj.weight": "model-00074-of-00101.safetensors", + "model.layers.67.mlp.experts.115.gate_proj.weight": "model-00074-of-00101.safetensors", + "model.layers.67.mlp.experts.115.up_proj.weight": "model-00074-of-00101.safetensors", + "model.layers.67.mlp.experts.115.down_proj.weight": "model-00074-of-00101.safetensors", + "model.layers.67.mlp.experts.116.gate_proj.weight": "model-00074-of-00101.safetensors", + "model.layers.67.mlp.experts.116.up_proj.weight": "model-00074-of-00101.safetensors", + "model.layers.67.mlp.experts.116.down_proj.weight": "model-00074-of-00101.safetensors", + "model.layers.67.mlp.experts.117.gate_proj.weight": "model-00074-of-00101.safetensors", + "model.layers.67.mlp.experts.117.up_proj.weight": "model-00074-of-00101.safetensors", + "model.layers.67.mlp.experts.117.down_proj.weight": "model-00074-of-00101.safetensors", + "model.layers.67.mlp.experts.118.gate_proj.weight": "model-00074-of-00101.safetensors", + "model.layers.67.mlp.experts.118.up_proj.weight": "model-00074-of-00101.safetensors", + "model.layers.67.mlp.experts.118.down_proj.weight": "model-00074-of-00101.safetensors", + "model.layers.67.mlp.experts.119.gate_proj.weight": "model-00074-of-00101.safetensors", + "model.layers.67.mlp.experts.119.up_proj.weight": "model-00074-of-00101.safetensors", + "model.layers.67.mlp.experts.119.down_proj.weight": "model-00074-of-00101.safetensors", + "model.layers.67.mlp.gate.weight": "model-00074-of-00101.safetensors", + "model.layers.67.mlp.gate.e_score_correction_bias": "model-00074-of-00101.safetensors", + "model.layers.67.mlp.shared_experts.gate_proj.weight": "model-00074-of-00101.safetensors", + "model.layers.67.mlp.shared_experts.up_proj.weight": "model-00074-of-00101.safetensors", + "model.layers.67.mlp.shared_experts.down_proj.weight": "model-00074-of-00101.safetensors", + "model.layers.67.input_layernorm.weight": "model-00074-of-00101.safetensors", + "model.layers.67.post_attention_layernorm.weight": "model-00074-of-00101.safetensors", + "model.layers.68.self_attn.q_proj.weight": "model-00074-of-00101.safetensors", + "model.layers.68.self_attn.q_proj.bias": "model-00074-of-00101.safetensors", + "model.layers.68.self_attn.k_proj.weight": "model-00074-of-00101.safetensors", + "model.layers.68.self_attn.k_proj.bias": "model-00074-of-00101.safetensors", + "model.layers.68.self_attn.v_proj.weight": "model-00074-of-00101.safetensors", + "model.layers.68.self_attn.v_proj.bias": "model-00074-of-00101.safetensors", + "model.layers.68.self_attn.o_proj.weight": "model-00074-of-00101.safetensors", + "model.layers.68.self_attn.q_norm.weight": "model-00074-of-00101.safetensors", + "model.layers.68.self_attn.k_norm.weight": "model-00074-of-00101.safetensors", + "model.layers.68.mlp.experts.0.gate_proj.weight": "model-00074-of-00101.safetensors", + "model.layers.68.mlp.experts.0.up_proj.weight": "model-00074-of-00101.safetensors", + "model.layers.68.mlp.experts.0.down_proj.weight": "model-00074-of-00101.safetensors", + "model.layers.68.mlp.experts.1.gate_proj.weight": "model-00074-of-00101.safetensors", + "model.layers.68.mlp.experts.1.up_proj.weight": "model-00074-of-00101.safetensors", + "model.layers.68.mlp.experts.1.down_proj.weight": "model-00074-of-00101.safetensors", + "model.layers.68.mlp.experts.2.gate_proj.weight": "model-00074-of-00101.safetensors", + "model.layers.68.mlp.experts.2.up_proj.weight": "model-00074-of-00101.safetensors", + "model.layers.68.mlp.experts.2.down_proj.weight": "model-00074-of-00101.safetensors", + "model.layers.68.mlp.experts.3.gate_proj.weight": "model-00074-of-00101.safetensors", + "model.layers.68.mlp.experts.3.up_proj.weight": "model-00074-of-00101.safetensors", + "model.layers.68.mlp.experts.3.down_proj.weight": "model-00074-of-00101.safetensors", + "model.layers.68.mlp.experts.4.gate_proj.weight": "model-00074-of-00101.safetensors", + "model.layers.68.mlp.experts.4.up_proj.weight": "model-00074-of-00101.safetensors", + "model.layers.68.mlp.experts.4.down_proj.weight": "model-00074-of-00101.safetensors", + "model.layers.68.mlp.experts.5.gate_proj.weight": "model-00074-of-00101.safetensors", + "model.layers.68.mlp.experts.5.up_proj.weight": "model-00074-of-00101.safetensors", + "model.layers.68.mlp.experts.5.down_proj.weight": "model-00074-of-00101.safetensors", + "model.layers.68.mlp.experts.6.gate_proj.weight": "model-00074-of-00101.safetensors", + "model.layers.68.mlp.experts.6.up_proj.weight": "model-00074-of-00101.safetensors", + "model.layers.68.mlp.experts.6.down_proj.weight": "model-00074-of-00101.safetensors", + "model.layers.68.mlp.experts.7.gate_proj.weight": "model-00074-of-00101.safetensors", + "model.layers.68.mlp.experts.7.up_proj.weight": "model-00074-of-00101.safetensors", + "model.layers.68.mlp.experts.7.down_proj.weight": "model-00074-of-00101.safetensors", + "model.layers.68.mlp.experts.8.gate_proj.weight": "model-00074-of-00101.safetensors", + "model.layers.68.mlp.experts.8.up_proj.weight": "model-00074-of-00101.safetensors", + "model.layers.68.mlp.experts.8.down_proj.weight": "model-00074-of-00101.safetensors", + "model.layers.68.mlp.experts.9.gate_proj.weight": "model-00074-of-00101.safetensors", + "model.layers.68.mlp.experts.9.up_proj.weight": "model-00074-of-00101.safetensors", + "model.layers.68.mlp.experts.9.down_proj.weight": "model-00074-of-00101.safetensors", + "model.layers.68.mlp.experts.10.gate_proj.weight": "model-00074-of-00101.safetensors", + "model.layers.68.mlp.experts.10.up_proj.weight": "model-00074-of-00101.safetensors", + "model.layers.68.mlp.experts.10.down_proj.weight": "model-00074-of-00101.safetensors", + "model.layers.68.mlp.experts.11.gate_proj.weight": "model-00074-of-00101.safetensors", + "model.layers.68.mlp.experts.11.up_proj.weight": "model-00074-of-00101.safetensors", + "model.layers.68.mlp.experts.11.down_proj.weight": "model-00074-of-00101.safetensors", + "model.layers.68.mlp.experts.12.gate_proj.weight": "model-00074-of-00101.safetensors", + "model.layers.68.mlp.experts.12.up_proj.weight": "model-00074-of-00101.safetensors", + "model.layers.68.mlp.experts.12.down_proj.weight": "model-00074-of-00101.safetensors", + "model.layers.68.mlp.experts.13.gate_proj.weight": "model-00074-of-00101.safetensors", + "model.layers.68.mlp.experts.13.up_proj.weight": "model-00074-of-00101.safetensors", + "model.layers.68.mlp.experts.13.down_proj.weight": "model-00074-of-00101.safetensors", + "model.layers.68.mlp.experts.14.gate_proj.weight": "model-00074-of-00101.safetensors", + "model.layers.68.mlp.experts.14.up_proj.weight": "model-00074-of-00101.safetensors", + "model.layers.68.mlp.experts.14.down_proj.weight": "model-00074-of-00101.safetensors", + "model.layers.68.mlp.experts.15.gate_proj.weight": "model-00074-of-00101.safetensors", + "model.layers.68.mlp.experts.15.up_proj.weight": "model-00074-of-00101.safetensors", + "model.layers.68.mlp.experts.15.down_proj.weight": "model-00074-of-00101.safetensors", + "model.layers.68.mlp.experts.16.gate_proj.weight": "model-00074-of-00101.safetensors", + "model.layers.68.mlp.experts.16.up_proj.weight": "model-00074-of-00101.safetensors", + "model.layers.68.mlp.experts.16.down_proj.weight": "model-00074-of-00101.safetensors", + "model.layers.68.mlp.experts.17.gate_proj.weight": "model-00074-of-00101.safetensors", + "model.layers.68.mlp.experts.17.up_proj.weight": "model-00074-of-00101.safetensors", + "model.layers.68.mlp.experts.17.down_proj.weight": "model-00074-of-00101.safetensors", + "model.layers.68.mlp.experts.18.gate_proj.weight": "model-00074-of-00101.safetensors", + "model.layers.68.mlp.experts.18.up_proj.weight": "model-00074-of-00101.safetensors", + "model.layers.68.mlp.experts.18.down_proj.weight": "model-00074-of-00101.safetensors", + "model.layers.68.mlp.experts.19.gate_proj.weight": "model-00074-of-00101.safetensors", + "model.layers.68.mlp.experts.19.up_proj.weight": "model-00074-of-00101.safetensors", + "model.layers.68.mlp.experts.19.down_proj.weight": "model-00074-of-00101.safetensors", + "model.layers.68.mlp.experts.20.gate_proj.weight": "model-00074-of-00101.safetensors", + "model.layers.68.mlp.experts.20.up_proj.weight": "model-00074-of-00101.safetensors", + "model.layers.68.mlp.experts.20.down_proj.weight": "model-00074-of-00101.safetensors", + "model.layers.68.mlp.experts.21.gate_proj.weight": "model-00074-of-00101.safetensors", + "model.layers.68.mlp.experts.21.up_proj.weight": "model-00074-of-00101.safetensors", + "model.layers.68.mlp.experts.21.down_proj.weight": "model-00074-of-00101.safetensors", + "model.layers.68.mlp.experts.22.gate_proj.weight": "model-00074-of-00101.safetensors", + "model.layers.68.mlp.experts.22.up_proj.weight": "model-00074-of-00101.safetensors", + "model.layers.68.mlp.experts.22.down_proj.weight": "model-00074-of-00101.safetensors", + "model.layers.68.mlp.experts.23.gate_proj.weight": "model-00074-of-00101.safetensors", + "model.layers.68.mlp.experts.23.up_proj.weight": "model-00074-of-00101.safetensors", + "model.layers.68.mlp.experts.23.down_proj.weight": "model-00074-of-00101.safetensors", + "model.layers.68.mlp.experts.24.gate_proj.weight": "model-00074-of-00101.safetensors", + "model.layers.68.mlp.experts.24.up_proj.weight": "model-00074-of-00101.safetensors", + "model.layers.68.mlp.experts.24.down_proj.weight": "model-00074-of-00101.safetensors", + "model.layers.68.mlp.experts.25.gate_proj.weight": "model-00074-of-00101.safetensors", + "model.layers.68.mlp.experts.25.up_proj.weight": "model-00074-of-00101.safetensors", + "model.layers.68.mlp.experts.25.down_proj.weight": "model-00074-of-00101.safetensors", + "model.layers.68.mlp.experts.26.gate_proj.weight": "model-00074-of-00101.safetensors", + "model.layers.68.mlp.experts.26.up_proj.weight": "model-00074-of-00101.safetensors", + "model.layers.68.mlp.experts.26.down_proj.weight": "model-00074-of-00101.safetensors", + "model.layers.68.mlp.experts.27.gate_proj.weight": "model-00074-of-00101.safetensors", + "model.layers.68.mlp.experts.27.up_proj.weight": "model-00074-of-00101.safetensors", + "model.layers.68.mlp.experts.27.down_proj.weight": "model-00074-of-00101.safetensors", + "model.layers.68.mlp.experts.28.gate_proj.weight": "model-00074-of-00101.safetensors", + "model.layers.68.mlp.experts.28.up_proj.weight": "model-00074-of-00101.safetensors", + "model.layers.68.mlp.experts.28.down_proj.weight": "model-00074-of-00101.safetensors", + "model.layers.68.mlp.experts.29.gate_proj.weight": "model-00074-of-00101.safetensors", + "model.layers.68.mlp.experts.29.up_proj.weight": "model-00074-of-00101.safetensors", + "model.layers.68.mlp.experts.29.down_proj.weight": "model-00074-of-00101.safetensors", + "model.layers.68.mlp.experts.30.gate_proj.weight": "model-00074-of-00101.safetensors", + "model.layers.68.mlp.experts.30.up_proj.weight": "model-00074-of-00101.safetensors", + "model.layers.68.mlp.experts.30.down_proj.weight": "model-00074-of-00101.safetensors", + "model.layers.68.mlp.experts.31.gate_proj.weight": "model-00074-of-00101.safetensors", + "model.layers.68.mlp.experts.31.up_proj.weight": "model-00074-of-00101.safetensors", + "model.layers.68.mlp.experts.31.down_proj.weight": "model-00074-of-00101.safetensors", + "model.layers.68.mlp.experts.32.gate_proj.weight": "model-00074-of-00101.safetensors", + "model.layers.68.mlp.experts.32.up_proj.weight": "model-00074-of-00101.safetensors", + "model.layers.68.mlp.experts.32.down_proj.weight": "model-00074-of-00101.safetensors", + "model.layers.68.mlp.experts.33.gate_proj.weight": "model-00074-of-00101.safetensors", + "model.layers.68.mlp.experts.33.up_proj.weight": "model-00074-of-00101.safetensors", + "model.layers.68.mlp.experts.33.down_proj.weight": "model-00074-of-00101.safetensors", + "model.layers.68.mlp.experts.34.gate_proj.weight": "model-00074-of-00101.safetensors", + "model.layers.68.mlp.experts.34.up_proj.weight": "model-00074-of-00101.safetensors", + "model.layers.68.mlp.experts.34.down_proj.weight": "model-00074-of-00101.safetensors", + "model.layers.68.mlp.experts.35.gate_proj.weight": "model-00074-of-00101.safetensors", + "model.layers.68.mlp.experts.35.up_proj.weight": "model-00074-of-00101.safetensors", + "model.layers.68.mlp.experts.35.down_proj.weight": "model-00074-of-00101.safetensors", + "model.layers.68.mlp.experts.36.gate_proj.weight": "model-00074-of-00101.safetensors", + "model.layers.68.mlp.experts.36.up_proj.weight": "model-00074-of-00101.safetensors", + "model.layers.68.mlp.experts.36.down_proj.weight": "model-00074-of-00101.safetensors", + "model.layers.68.mlp.experts.37.gate_proj.weight": "model-00074-of-00101.safetensors", + "model.layers.68.mlp.experts.37.up_proj.weight": "model-00074-of-00101.safetensors", + "model.layers.68.mlp.experts.37.down_proj.weight": "model-00074-of-00101.safetensors", + "model.layers.68.mlp.experts.38.gate_proj.weight": "model-00074-of-00101.safetensors", + "model.layers.68.mlp.experts.38.up_proj.weight": "model-00074-of-00101.safetensors", + "model.layers.68.mlp.experts.38.down_proj.weight": "model-00074-of-00101.safetensors", + "model.layers.68.mlp.experts.39.gate_proj.weight": "model-00074-of-00101.safetensors", + "model.layers.68.mlp.experts.39.up_proj.weight": "model-00074-of-00101.safetensors", + "model.layers.68.mlp.experts.39.down_proj.weight": "model-00074-of-00101.safetensors", + "model.layers.68.mlp.experts.40.gate_proj.weight": "model-00074-of-00101.safetensors", + "model.layers.68.mlp.experts.40.up_proj.weight": "model-00074-of-00101.safetensors", + "model.layers.68.mlp.experts.40.down_proj.weight": "model-00074-of-00101.safetensors", + "model.layers.68.mlp.experts.41.gate_proj.weight": "model-00074-of-00101.safetensors", + "model.layers.68.mlp.experts.41.up_proj.weight": "model-00074-of-00101.safetensors", + "model.layers.68.mlp.experts.41.down_proj.weight": "model-00074-of-00101.safetensors", + "model.layers.68.mlp.experts.42.gate_proj.weight": "model-00074-of-00101.safetensors", + "model.layers.68.mlp.experts.42.up_proj.weight": "model-00074-of-00101.safetensors", + "model.layers.68.mlp.experts.42.down_proj.weight": "model-00074-of-00101.safetensors", + "model.layers.68.mlp.experts.43.gate_proj.weight": "model-00074-of-00101.safetensors", + "model.layers.68.mlp.experts.43.up_proj.weight": "model-00074-of-00101.safetensors", + "model.layers.68.mlp.experts.43.down_proj.weight": "model-00074-of-00101.safetensors", + "model.layers.68.mlp.experts.44.gate_proj.weight": "model-00074-of-00101.safetensors", + "model.layers.68.mlp.experts.44.up_proj.weight": "model-00074-of-00101.safetensors", + "model.layers.68.mlp.experts.44.down_proj.weight": "model-00074-of-00101.safetensors", + "model.layers.68.mlp.experts.45.gate_proj.weight": "model-00074-of-00101.safetensors", + "model.layers.68.mlp.experts.45.up_proj.weight": "model-00074-of-00101.safetensors", + "model.layers.68.mlp.experts.45.down_proj.weight": "model-00074-of-00101.safetensors", + "model.layers.68.mlp.experts.46.gate_proj.weight": "model-00074-of-00101.safetensors", + "model.layers.68.mlp.experts.46.up_proj.weight": "model-00074-of-00101.safetensors", + "model.layers.68.mlp.experts.46.down_proj.weight": "model-00074-of-00101.safetensors", + "model.layers.68.mlp.experts.47.gate_proj.weight": "model-00074-of-00101.safetensors", + "model.layers.68.mlp.experts.47.up_proj.weight": "model-00074-of-00101.safetensors", + "model.layers.68.mlp.experts.47.down_proj.weight": "model-00074-of-00101.safetensors", + "model.layers.68.mlp.experts.48.gate_proj.weight": "model-00074-of-00101.safetensors", + "model.layers.68.mlp.experts.48.up_proj.weight": "model-00074-of-00101.safetensors", + "model.layers.68.mlp.experts.48.down_proj.weight": "model-00074-of-00101.safetensors", + "model.layers.68.mlp.experts.49.gate_proj.weight": "model-00074-of-00101.safetensors", + "model.layers.68.mlp.experts.49.up_proj.weight": "model-00074-of-00101.safetensors", + "model.layers.68.mlp.experts.49.down_proj.weight": "model-00074-of-00101.safetensors", + "model.layers.68.mlp.experts.50.gate_proj.weight": "model-00074-of-00101.safetensors", + "model.layers.68.mlp.experts.50.up_proj.weight": "model-00074-of-00101.safetensors", + "model.layers.68.mlp.experts.50.down_proj.weight": "model-00074-of-00101.safetensors", + "model.layers.68.mlp.experts.51.gate_proj.weight": "model-00074-of-00101.safetensors", + "model.layers.68.mlp.experts.51.up_proj.weight": "model-00074-of-00101.safetensors", + "model.layers.68.mlp.experts.51.down_proj.weight": "model-00074-of-00101.safetensors", + "model.layers.68.mlp.experts.52.gate_proj.weight": "model-00074-of-00101.safetensors", + "model.layers.68.mlp.experts.52.up_proj.weight": "model-00074-of-00101.safetensors", + "model.layers.68.mlp.experts.52.down_proj.weight": "model-00074-of-00101.safetensors", + "model.layers.68.mlp.experts.53.gate_proj.weight": "model-00074-of-00101.safetensors", + "model.layers.68.mlp.experts.53.up_proj.weight": "model-00074-of-00101.safetensors", + "model.layers.68.mlp.experts.53.down_proj.weight": "model-00074-of-00101.safetensors", + "model.layers.68.mlp.experts.54.gate_proj.weight": "model-00074-of-00101.safetensors", + "model.layers.68.mlp.experts.54.up_proj.weight": "model-00074-of-00101.safetensors", + "model.layers.68.mlp.experts.54.down_proj.weight": "model-00074-of-00101.safetensors", + "model.layers.68.mlp.experts.55.gate_proj.weight": "model-00074-of-00101.safetensors", + "model.layers.68.mlp.experts.55.up_proj.weight": "model-00074-of-00101.safetensors", + "model.layers.68.mlp.experts.55.down_proj.weight": "model-00074-of-00101.safetensors", + "model.layers.68.mlp.experts.56.gate_proj.weight": "model-00074-of-00101.safetensors", + "model.layers.68.mlp.experts.56.up_proj.weight": "model-00074-of-00101.safetensors", + "model.layers.68.mlp.experts.56.down_proj.weight": "model-00074-of-00101.safetensors", + "model.layers.68.mlp.experts.57.gate_proj.weight": "model-00074-of-00101.safetensors", + "model.layers.68.mlp.experts.57.up_proj.weight": "model-00074-of-00101.safetensors", + "model.layers.68.mlp.experts.57.down_proj.weight": "model-00074-of-00101.safetensors", + "model.layers.68.mlp.experts.58.gate_proj.weight": "model-00074-of-00101.safetensors", + "model.layers.68.mlp.experts.58.up_proj.weight": "model-00074-of-00101.safetensors", + "model.layers.68.mlp.experts.58.down_proj.weight": "model-00074-of-00101.safetensors", + "model.layers.68.mlp.experts.59.gate_proj.weight": "model-00074-of-00101.safetensors", + "model.layers.68.mlp.experts.59.up_proj.weight": "model-00074-of-00101.safetensors", + "model.layers.68.mlp.experts.59.down_proj.weight": "model-00074-of-00101.safetensors", + "model.layers.68.mlp.experts.60.gate_proj.weight": "model-00074-of-00101.safetensors", + "model.layers.68.mlp.experts.60.up_proj.weight": "model-00074-of-00101.safetensors", + "model.layers.68.mlp.experts.60.down_proj.weight": "model-00074-of-00101.safetensors", + "model.layers.68.mlp.experts.61.gate_proj.weight": "model-00074-of-00101.safetensors", + "model.layers.68.mlp.experts.61.up_proj.weight": "model-00074-of-00101.safetensors", + "model.layers.68.mlp.experts.61.down_proj.weight": "model-00074-of-00101.safetensors", + "model.layers.68.mlp.experts.62.gate_proj.weight": "model-00074-of-00101.safetensors", + "model.layers.68.mlp.experts.62.up_proj.weight": "model-00074-of-00101.safetensors", + "model.layers.68.mlp.experts.62.down_proj.weight": "model-00074-of-00101.safetensors", + "model.layers.68.mlp.experts.63.gate_proj.weight": "model-00074-of-00101.safetensors", + "model.layers.68.mlp.experts.63.up_proj.weight": "model-00074-of-00101.safetensors", + "model.layers.68.mlp.experts.63.down_proj.weight": "model-00074-of-00101.safetensors", + "model.layers.68.mlp.experts.64.gate_proj.weight": "model-00074-of-00101.safetensors", + "model.layers.68.mlp.experts.64.up_proj.weight": "model-00074-of-00101.safetensors", + "model.layers.68.mlp.experts.64.down_proj.weight": "model-00074-of-00101.safetensors", + "model.layers.68.mlp.experts.65.gate_proj.weight": "model-00074-of-00101.safetensors", + "model.layers.68.mlp.experts.65.up_proj.weight": "model-00074-of-00101.safetensors", + "model.layers.68.mlp.experts.65.down_proj.weight": "model-00074-of-00101.safetensors", + "model.layers.68.mlp.experts.66.gate_proj.weight": "model-00074-of-00101.safetensors", + "model.layers.68.mlp.experts.66.up_proj.weight": "model-00074-of-00101.safetensors", + "model.layers.68.mlp.experts.66.down_proj.weight": "model-00074-of-00101.safetensors", + "model.layers.68.mlp.experts.67.gate_proj.weight": "model-00074-of-00101.safetensors", + "model.layers.68.mlp.experts.67.up_proj.weight": "model-00074-of-00101.safetensors", + "model.layers.68.mlp.experts.67.down_proj.weight": "model-00074-of-00101.safetensors", + "model.layers.68.mlp.experts.68.gate_proj.weight": "model-00074-of-00101.safetensors", + "model.layers.68.mlp.experts.68.up_proj.weight": "model-00074-of-00101.safetensors", + "model.layers.68.mlp.experts.68.down_proj.weight": "model-00074-of-00101.safetensors", + "model.layers.68.mlp.experts.69.gate_proj.weight": "model-00074-of-00101.safetensors", + "model.layers.68.mlp.experts.69.up_proj.weight": "model-00074-of-00101.safetensors", + "model.layers.68.mlp.experts.69.down_proj.weight": "model-00074-of-00101.safetensors", + "model.layers.68.mlp.experts.70.gate_proj.weight": "model-00074-of-00101.safetensors", + "model.layers.68.mlp.experts.70.up_proj.weight": "model-00074-of-00101.safetensors", + "model.layers.68.mlp.experts.70.down_proj.weight": "model-00074-of-00101.safetensors", + "model.layers.68.mlp.experts.71.gate_proj.weight": "model-00074-of-00101.safetensors", + "model.layers.68.mlp.experts.71.up_proj.weight": "model-00074-of-00101.safetensors", + "model.layers.68.mlp.experts.71.down_proj.weight": "model-00074-of-00101.safetensors", + "model.layers.68.mlp.experts.72.gate_proj.weight": "model-00074-of-00101.safetensors", + "model.layers.68.mlp.experts.72.up_proj.weight": "model-00074-of-00101.safetensors", + "model.layers.68.mlp.experts.72.down_proj.weight": "model-00074-of-00101.safetensors", + "model.layers.68.mlp.experts.73.gate_proj.weight": "model-00074-of-00101.safetensors", + "model.layers.68.mlp.experts.73.up_proj.weight": "model-00074-of-00101.safetensors", + "model.layers.68.mlp.experts.73.down_proj.weight": "model-00074-of-00101.safetensors", + "model.layers.68.mlp.experts.74.gate_proj.weight": "model-00074-of-00101.safetensors", + "model.layers.68.mlp.experts.74.up_proj.weight": "model-00074-of-00101.safetensors", + "model.layers.68.mlp.experts.74.down_proj.weight": "model-00075-of-00101.safetensors", + "model.layers.68.mlp.experts.75.gate_proj.weight": "model-00075-of-00101.safetensors", + "model.layers.68.mlp.experts.75.up_proj.weight": "model-00075-of-00101.safetensors", + "model.layers.68.mlp.experts.75.down_proj.weight": "model-00075-of-00101.safetensors", + "model.layers.68.mlp.experts.76.gate_proj.weight": "model-00075-of-00101.safetensors", + "model.layers.68.mlp.experts.76.up_proj.weight": "model-00075-of-00101.safetensors", + "model.layers.68.mlp.experts.76.down_proj.weight": "model-00075-of-00101.safetensors", + "model.layers.68.mlp.experts.77.gate_proj.weight": "model-00075-of-00101.safetensors", + "model.layers.68.mlp.experts.77.up_proj.weight": "model-00075-of-00101.safetensors", + "model.layers.68.mlp.experts.77.down_proj.weight": "model-00075-of-00101.safetensors", + "model.layers.68.mlp.experts.78.gate_proj.weight": "model-00075-of-00101.safetensors", + "model.layers.68.mlp.experts.78.up_proj.weight": "model-00075-of-00101.safetensors", + "model.layers.68.mlp.experts.78.down_proj.weight": "model-00075-of-00101.safetensors", + "model.layers.68.mlp.experts.79.gate_proj.weight": "model-00075-of-00101.safetensors", + "model.layers.68.mlp.experts.79.up_proj.weight": "model-00075-of-00101.safetensors", + "model.layers.68.mlp.experts.79.down_proj.weight": "model-00075-of-00101.safetensors", + "model.layers.68.mlp.experts.80.gate_proj.weight": "model-00075-of-00101.safetensors", + "model.layers.68.mlp.experts.80.up_proj.weight": "model-00075-of-00101.safetensors", + "model.layers.68.mlp.experts.80.down_proj.weight": "model-00075-of-00101.safetensors", + "model.layers.68.mlp.experts.81.gate_proj.weight": "model-00075-of-00101.safetensors", + "model.layers.68.mlp.experts.81.up_proj.weight": "model-00075-of-00101.safetensors", + "model.layers.68.mlp.experts.81.down_proj.weight": "model-00075-of-00101.safetensors", + "model.layers.68.mlp.experts.82.gate_proj.weight": "model-00075-of-00101.safetensors", + "model.layers.68.mlp.experts.82.up_proj.weight": "model-00075-of-00101.safetensors", + "model.layers.68.mlp.experts.82.down_proj.weight": "model-00075-of-00101.safetensors", + "model.layers.68.mlp.experts.83.gate_proj.weight": "model-00075-of-00101.safetensors", + "model.layers.68.mlp.experts.83.up_proj.weight": "model-00075-of-00101.safetensors", + "model.layers.68.mlp.experts.83.down_proj.weight": "model-00075-of-00101.safetensors", + "model.layers.68.mlp.experts.84.gate_proj.weight": "model-00075-of-00101.safetensors", + "model.layers.68.mlp.experts.84.up_proj.weight": "model-00075-of-00101.safetensors", + "model.layers.68.mlp.experts.84.down_proj.weight": "model-00075-of-00101.safetensors", + "model.layers.68.mlp.experts.85.gate_proj.weight": "model-00075-of-00101.safetensors", + "model.layers.68.mlp.experts.85.up_proj.weight": "model-00075-of-00101.safetensors", + "model.layers.68.mlp.experts.85.down_proj.weight": "model-00075-of-00101.safetensors", + "model.layers.68.mlp.experts.86.gate_proj.weight": "model-00075-of-00101.safetensors", + "model.layers.68.mlp.experts.86.up_proj.weight": "model-00075-of-00101.safetensors", + "model.layers.68.mlp.experts.86.down_proj.weight": "model-00075-of-00101.safetensors", + "model.layers.68.mlp.experts.87.gate_proj.weight": "model-00075-of-00101.safetensors", + "model.layers.68.mlp.experts.87.up_proj.weight": "model-00075-of-00101.safetensors", + "model.layers.68.mlp.experts.87.down_proj.weight": "model-00075-of-00101.safetensors", + "model.layers.68.mlp.experts.88.gate_proj.weight": "model-00075-of-00101.safetensors", + "model.layers.68.mlp.experts.88.up_proj.weight": "model-00075-of-00101.safetensors", + "model.layers.68.mlp.experts.88.down_proj.weight": "model-00075-of-00101.safetensors", + "model.layers.68.mlp.experts.89.gate_proj.weight": "model-00075-of-00101.safetensors", + "model.layers.68.mlp.experts.89.up_proj.weight": "model-00075-of-00101.safetensors", + "model.layers.68.mlp.experts.89.down_proj.weight": "model-00075-of-00101.safetensors", + "model.layers.68.mlp.experts.90.gate_proj.weight": "model-00075-of-00101.safetensors", + "model.layers.68.mlp.experts.90.up_proj.weight": "model-00075-of-00101.safetensors", + "model.layers.68.mlp.experts.90.down_proj.weight": "model-00075-of-00101.safetensors", + "model.layers.68.mlp.experts.91.gate_proj.weight": "model-00075-of-00101.safetensors", + "model.layers.68.mlp.experts.91.up_proj.weight": "model-00075-of-00101.safetensors", + "model.layers.68.mlp.experts.91.down_proj.weight": "model-00075-of-00101.safetensors", + "model.layers.68.mlp.experts.92.gate_proj.weight": "model-00075-of-00101.safetensors", + "model.layers.68.mlp.experts.92.up_proj.weight": "model-00075-of-00101.safetensors", + "model.layers.68.mlp.experts.92.down_proj.weight": "model-00075-of-00101.safetensors", + "model.layers.68.mlp.experts.93.gate_proj.weight": "model-00075-of-00101.safetensors", + "model.layers.68.mlp.experts.93.up_proj.weight": "model-00075-of-00101.safetensors", + "model.layers.68.mlp.experts.93.down_proj.weight": "model-00075-of-00101.safetensors", + "model.layers.68.mlp.experts.94.gate_proj.weight": "model-00075-of-00101.safetensors", + "model.layers.68.mlp.experts.94.up_proj.weight": "model-00075-of-00101.safetensors", + "model.layers.68.mlp.experts.94.down_proj.weight": "model-00075-of-00101.safetensors", + "model.layers.68.mlp.experts.95.gate_proj.weight": "model-00075-of-00101.safetensors", + "model.layers.68.mlp.experts.95.up_proj.weight": "model-00075-of-00101.safetensors", + "model.layers.68.mlp.experts.95.down_proj.weight": "model-00075-of-00101.safetensors", + "model.layers.68.mlp.experts.96.gate_proj.weight": "model-00075-of-00101.safetensors", + "model.layers.68.mlp.experts.96.up_proj.weight": "model-00075-of-00101.safetensors", + "model.layers.68.mlp.experts.96.down_proj.weight": "model-00075-of-00101.safetensors", + "model.layers.68.mlp.experts.97.gate_proj.weight": "model-00075-of-00101.safetensors", + "model.layers.68.mlp.experts.97.up_proj.weight": "model-00075-of-00101.safetensors", + "model.layers.68.mlp.experts.97.down_proj.weight": "model-00075-of-00101.safetensors", + "model.layers.68.mlp.experts.98.gate_proj.weight": "model-00075-of-00101.safetensors", + "model.layers.68.mlp.experts.98.up_proj.weight": "model-00075-of-00101.safetensors", + "model.layers.68.mlp.experts.98.down_proj.weight": "model-00075-of-00101.safetensors", + "model.layers.68.mlp.experts.99.gate_proj.weight": "model-00075-of-00101.safetensors", + "model.layers.68.mlp.experts.99.up_proj.weight": "model-00075-of-00101.safetensors", + "model.layers.68.mlp.experts.99.down_proj.weight": "model-00075-of-00101.safetensors", + "model.layers.68.mlp.experts.100.gate_proj.weight": "model-00075-of-00101.safetensors", + "model.layers.68.mlp.experts.100.up_proj.weight": "model-00075-of-00101.safetensors", + "model.layers.68.mlp.experts.100.down_proj.weight": "model-00075-of-00101.safetensors", + "model.layers.68.mlp.experts.101.gate_proj.weight": "model-00075-of-00101.safetensors", + "model.layers.68.mlp.experts.101.up_proj.weight": "model-00075-of-00101.safetensors", + "model.layers.68.mlp.experts.101.down_proj.weight": "model-00075-of-00101.safetensors", + "model.layers.68.mlp.experts.102.gate_proj.weight": "model-00075-of-00101.safetensors", + "model.layers.68.mlp.experts.102.up_proj.weight": "model-00075-of-00101.safetensors", + "model.layers.68.mlp.experts.102.down_proj.weight": "model-00075-of-00101.safetensors", + "model.layers.68.mlp.experts.103.gate_proj.weight": "model-00075-of-00101.safetensors", + "model.layers.68.mlp.experts.103.up_proj.weight": "model-00075-of-00101.safetensors", + "model.layers.68.mlp.experts.103.down_proj.weight": "model-00075-of-00101.safetensors", + "model.layers.68.mlp.experts.104.gate_proj.weight": "model-00075-of-00101.safetensors", + "model.layers.68.mlp.experts.104.up_proj.weight": "model-00075-of-00101.safetensors", + "model.layers.68.mlp.experts.104.down_proj.weight": "model-00075-of-00101.safetensors", + "model.layers.68.mlp.experts.105.gate_proj.weight": "model-00075-of-00101.safetensors", + "model.layers.68.mlp.experts.105.up_proj.weight": "model-00075-of-00101.safetensors", + "model.layers.68.mlp.experts.105.down_proj.weight": "model-00075-of-00101.safetensors", + "model.layers.68.mlp.experts.106.gate_proj.weight": "model-00075-of-00101.safetensors", + "model.layers.68.mlp.experts.106.up_proj.weight": "model-00075-of-00101.safetensors", + "model.layers.68.mlp.experts.106.down_proj.weight": "model-00075-of-00101.safetensors", + "model.layers.68.mlp.experts.107.gate_proj.weight": "model-00075-of-00101.safetensors", + "model.layers.68.mlp.experts.107.up_proj.weight": "model-00075-of-00101.safetensors", + "model.layers.68.mlp.experts.107.down_proj.weight": "model-00075-of-00101.safetensors", + "model.layers.68.mlp.experts.108.gate_proj.weight": "model-00075-of-00101.safetensors", + "model.layers.68.mlp.experts.108.up_proj.weight": "model-00075-of-00101.safetensors", + "model.layers.68.mlp.experts.108.down_proj.weight": "model-00075-of-00101.safetensors", + "model.layers.68.mlp.experts.109.gate_proj.weight": "model-00075-of-00101.safetensors", + "model.layers.68.mlp.experts.109.up_proj.weight": "model-00075-of-00101.safetensors", + "model.layers.68.mlp.experts.109.down_proj.weight": "model-00075-of-00101.safetensors", + "model.layers.68.mlp.experts.110.gate_proj.weight": "model-00075-of-00101.safetensors", + "model.layers.68.mlp.experts.110.up_proj.weight": "model-00075-of-00101.safetensors", + "model.layers.68.mlp.experts.110.down_proj.weight": "model-00075-of-00101.safetensors", + "model.layers.68.mlp.experts.111.gate_proj.weight": "model-00075-of-00101.safetensors", + "model.layers.68.mlp.experts.111.up_proj.weight": "model-00075-of-00101.safetensors", + "model.layers.68.mlp.experts.111.down_proj.weight": "model-00075-of-00101.safetensors", + "model.layers.68.mlp.experts.112.gate_proj.weight": "model-00075-of-00101.safetensors", + "model.layers.68.mlp.experts.112.up_proj.weight": "model-00075-of-00101.safetensors", + "model.layers.68.mlp.experts.112.down_proj.weight": "model-00075-of-00101.safetensors", + "model.layers.68.mlp.experts.113.gate_proj.weight": "model-00075-of-00101.safetensors", + "model.layers.68.mlp.experts.113.up_proj.weight": "model-00075-of-00101.safetensors", + "model.layers.68.mlp.experts.113.down_proj.weight": "model-00075-of-00101.safetensors", + "model.layers.68.mlp.experts.114.gate_proj.weight": "model-00075-of-00101.safetensors", + "model.layers.68.mlp.experts.114.up_proj.weight": "model-00075-of-00101.safetensors", + "model.layers.68.mlp.experts.114.down_proj.weight": "model-00075-of-00101.safetensors", + "model.layers.68.mlp.experts.115.gate_proj.weight": "model-00075-of-00101.safetensors", + "model.layers.68.mlp.experts.115.up_proj.weight": "model-00075-of-00101.safetensors", + "model.layers.68.mlp.experts.115.down_proj.weight": "model-00075-of-00101.safetensors", + "model.layers.68.mlp.experts.116.gate_proj.weight": "model-00075-of-00101.safetensors", + "model.layers.68.mlp.experts.116.up_proj.weight": "model-00075-of-00101.safetensors", + "model.layers.68.mlp.experts.116.down_proj.weight": "model-00075-of-00101.safetensors", + "model.layers.68.mlp.experts.117.gate_proj.weight": "model-00075-of-00101.safetensors", + "model.layers.68.mlp.experts.117.up_proj.weight": "model-00075-of-00101.safetensors", + "model.layers.68.mlp.experts.117.down_proj.weight": "model-00075-of-00101.safetensors", + "model.layers.68.mlp.experts.118.gate_proj.weight": "model-00075-of-00101.safetensors", + "model.layers.68.mlp.experts.118.up_proj.weight": "model-00075-of-00101.safetensors", + "model.layers.68.mlp.experts.118.down_proj.weight": "model-00075-of-00101.safetensors", + "model.layers.68.mlp.experts.119.gate_proj.weight": "model-00075-of-00101.safetensors", + "model.layers.68.mlp.experts.119.up_proj.weight": "model-00075-of-00101.safetensors", + "model.layers.68.mlp.experts.119.down_proj.weight": "model-00075-of-00101.safetensors", + "model.layers.68.mlp.gate.weight": "model-00075-of-00101.safetensors", + "model.layers.68.mlp.gate.e_score_correction_bias": "model-00075-of-00101.safetensors", + "model.layers.68.mlp.shared_experts.gate_proj.weight": "model-00075-of-00101.safetensors", + "model.layers.68.mlp.shared_experts.up_proj.weight": "model-00075-of-00101.safetensors", + "model.layers.68.mlp.shared_experts.down_proj.weight": "model-00075-of-00101.safetensors", + "model.layers.68.input_layernorm.weight": "model-00075-of-00101.safetensors", + "model.layers.68.post_attention_layernorm.weight": "model-00075-of-00101.safetensors", + "model.layers.69.self_attn.q_proj.weight": "model-00075-of-00101.safetensors", + "model.layers.69.self_attn.q_proj.bias": "model-00075-of-00101.safetensors", + "model.layers.69.self_attn.k_proj.weight": "model-00075-of-00101.safetensors", + "model.layers.69.self_attn.k_proj.bias": "model-00075-of-00101.safetensors", + "model.layers.69.self_attn.v_proj.weight": "model-00075-of-00101.safetensors", + "model.layers.69.self_attn.v_proj.bias": "model-00075-of-00101.safetensors", + "model.layers.69.self_attn.o_proj.weight": "model-00075-of-00101.safetensors", + "model.layers.69.self_attn.q_norm.weight": "model-00075-of-00101.safetensors", + "model.layers.69.self_attn.k_norm.weight": "model-00075-of-00101.safetensors", + "model.layers.69.mlp.experts.0.gate_proj.weight": "model-00075-of-00101.safetensors", + "model.layers.69.mlp.experts.0.up_proj.weight": "model-00075-of-00101.safetensors", + "model.layers.69.mlp.experts.0.down_proj.weight": "model-00075-of-00101.safetensors", + "model.layers.69.mlp.experts.1.gate_proj.weight": "model-00075-of-00101.safetensors", + "model.layers.69.mlp.experts.1.up_proj.weight": "model-00075-of-00101.safetensors", + "model.layers.69.mlp.experts.1.down_proj.weight": "model-00075-of-00101.safetensors", + "model.layers.69.mlp.experts.2.gate_proj.weight": "model-00075-of-00101.safetensors", + "model.layers.69.mlp.experts.2.up_proj.weight": "model-00075-of-00101.safetensors", + "model.layers.69.mlp.experts.2.down_proj.weight": "model-00075-of-00101.safetensors", + "model.layers.69.mlp.experts.3.gate_proj.weight": "model-00075-of-00101.safetensors", + "model.layers.69.mlp.experts.3.up_proj.weight": "model-00075-of-00101.safetensors", + "model.layers.69.mlp.experts.3.down_proj.weight": "model-00075-of-00101.safetensors", + "model.layers.69.mlp.experts.4.gate_proj.weight": "model-00075-of-00101.safetensors", + "model.layers.69.mlp.experts.4.up_proj.weight": "model-00075-of-00101.safetensors", + "model.layers.69.mlp.experts.4.down_proj.weight": "model-00075-of-00101.safetensors", + "model.layers.69.mlp.experts.5.gate_proj.weight": "model-00075-of-00101.safetensors", + "model.layers.69.mlp.experts.5.up_proj.weight": "model-00075-of-00101.safetensors", + "model.layers.69.mlp.experts.5.down_proj.weight": "model-00075-of-00101.safetensors", + "model.layers.69.mlp.experts.6.gate_proj.weight": "model-00075-of-00101.safetensors", + "model.layers.69.mlp.experts.6.up_proj.weight": "model-00075-of-00101.safetensors", + "model.layers.69.mlp.experts.6.down_proj.weight": "model-00075-of-00101.safetensors", + "model.layers.69.mlp.experts.7.gate_proj.weight": "model-00075-of-00101.safetensors", + "model.layers.69.mlp.experts.7.up_proj.weight": "model-00075-of-00101.safetensors", + "model.layers.69.mlp.experts.7.down_proj.weight": "model-00075-of-00101.safetensors", + "model.layers.69.mlp.experts.8.gate_proj.weight": "model-00075-of-00101.safetensors", + "model.layers.69.mlp.experts.8.up_proj.weight": "model-00075-of-00101.safetensors", + "model.layers.69.mlp.experts.8.down_proj.weight": "model-00075-of-00101.safetensors", + "model.layers.69.mlp.experts.9.gate_proj.weight": "model-00075-of-00101.safetensors", + "model.layers.69.mlp.experts.9.up_proj.weight": "model-00075-of-00101.safetensors", + "model.layers.69.mlp.experts.9.down_proj.weight": "model-00075-of-00101.safetensors", + "model.layers.69.mlp.experts.10.gate_proj.weight": "model-00075-of-00101.safetensors", + "model.layers.69.mlp.experts.10.up_proj.weight": "model-00075-of-00101.safetensors", + "model.layers.69.mlp.experts.10.down_proj.weight": "model-00075-of-00101.safetensors", + "model.layers.69.mlp.experts.11.gate_proj.weight": "model-00075-of-00101.safetensors", + "model.layers.69.mlp.experts.11.up_proj.weight": "model-00075-of-00101.safetensors", + "model.layers.69.mlp.experts.11.down_proj.weight": "model-00075-of-00101.safetensors", + "model.layers.69.mlp.experts.12.gate_proj.weight": "model-00075-of-00101.safetensors", + "model.layers.69.mlp.experts.12.up_proj.weight": "model-00075-of-00101.safetensors", + "model.layers.69.mlp.experts.12.down_proj.weight": "model-00075-of-00101.safetensors", + "model.layers.69.mlp.experts.13.gate_proj.weight": "model-00075-of-00101.safetensors", + "model.layers.69.mlp.experts.13.up_proj.weight": "model-00075-of-00101.safetensors", + "model.layers.69.mlp.experts.13.down_proj.weight": "model-00075-of-00101.safetensors", + "model.layers.69.mlp.experts.14.gate_proj.weight": "model-00075-of-00101.safetensors", + "model.layers.69.mlp.experts.14.up_proj.weight": "model-00075-of-00101.safetensors", + "model.layers.69.mlp.experts.14.down_proj.weight": "model-00075-of-00101.safetensors", + "model.layers.69.mlp.experts.15.gate_proj.weight": "model-00075-of-00101.safetensors", + "model.layers.69.mlp.experts.15.up_proj.weight": "model-00075-of-00101.safetensors", + "model.layers.69.mlp.experts.15.down_proj.weight": "model-00075-of-00101.safetensors", + "model.layers.69.mlp.experts.16.gate_proj.weight": "model-00075-of-00101.safetensors", + "model.layers.69.mlp.experts.16.up_proj.weight": "model-00075-of-00101.safetensors", + "model.layers.69.mlp.experts.16.down_proj.weight": "model-00075-of-00101.safetensors", + "model.layers.69.mlp.experts.17.gate_proj.weight": "model-00075-of-00101.safetensors", + "model.layers.69.mlp.experts.17.up_proj.weight": "model-00075-of-00101.safetensors", + "model.layers.69.mlp.experts.17.down_proj.weight": "model-00075-of-00101.safetensors", + "model.layers.69.mlp.experts.18.gate_proj.weight": "model-00075-of-00101.safetensors", + "model.layers.69.mlp.experts.18.up_proj.weight": "model-00075-of-00101.safetensors", + "model.layers.69.mlp.experts.18.down_proj.weight": "model-00075-of-00101.safetensors", + "model.layers.69.mlp.experts.19.gate_proj.weight": "model-00075-of-00101.safetensors", + "model.layers.69.mlp.experts.19.up_proj.weight": "model-00075-of-00101.safetensors", + "model.layers.69.mlp.experts.19.down_proj.weight": "model-00075-of-00101.safetensors", + "model.layers.69.mlp.experts.20.gate_proj.weight": "model-00075-of-00101.safetensors", + "model.layers.69.mlp.experts.20.up_proj.weight": "model-00075-of-00101.safetensors", + "model.layers.69.mlp.experts.20.down_proj.weight": "model-00075-of-00101.safetensors", + "model.layers.69.mlp.experts.21.gate_proj.weight": "model-00075-of-00101.safetensors", + "model.layers.69.mlp.experts.21.up_proj.weight": "model-00075-of-00101.safetensors", + "model.layers.69.mlp.experts.21.down_proj.weight": "model-00075-of-00101.safetensors", + "model.layers.69.mlp.experts.22.gate_proj.weight": "model-00075-of-00101.safetensors", + "model.layers.69.mlp.experts.22.up_proj.weight": "model-00075-of-00101.safetensors", + "model.layers.69.mlp.experts.22.down_proj.weight": "model-00075-of-00101.safetensors", + "model.layers.69.mlp.experts.23.gate_proj.weight": "model-00075-of-00101.safetensors", + "model.layers.69.mlp.experts.23.up_proj.weight": "model-00075-of-00101.safetensors", + "model.layers.69.mlp.experts.23.down_proj.weight": "model-00075-of-00101.safetensors", + "model.layers.69.mlp.experts.24.gate_proj.weight": "model-00075-of-00101.safetensors", + "model.layers.69.mlp.experts.24.up_proj.weight": "model-00075-of-00101.safetensors", + "model.layers.69.mlp.experts.24.down_proj.weight": "model-00075-of-00101.safetensors", + "model.layers.69.mlp.experts.25.gate_proj.weight": "model-00075-of-00101.safetensors", + "model.layers.69.mlp.experts.25.up_proj.weight": "model-00075-of-00101.safetensors", + "model.layers.69.mlp.experts.25.down_proj.weight": "model-00075-of-00101.safetensors", + "model.layers.69.mlp.experts.26.gate_proj.weight": "model-00075-of-00101.safetensors", + "model.layers.69.mlp.experts.26.up_proj.weight": "model-00075-of-00101.safetensors", + "model.layers.69.mlp.experts.26.down_proj.weight": "model-00075-of-00101.safetensors", + "model.layers.69.mlp.experts.27.gate_proj.weight": "model-00075-of-00101.safetensors", + "model.layers.69.mlp.experts.27.up_proj.weight": "model-00075-of-00101.safetensors", + "model.layers.69.mlp.experts.27.down_proj.weight": "model-00075-of-00101.safetensors", + "model.layers.69.mlp.experts.28.gate_proj.weight": "model-00075-of-00101.safetensors", + "model.layers.69.mlp.experts.28.up_proj.weight": "model-00075-of-00101.safetensors", + "model.layers.69.mlp.experts.28.down_proj.weight": "model-00075-of-00101.safetensors", + "model.layers.69.mlp.experts.29.gate_proj.weight": "model-00075-of-00101.safetensors", + "model.layers.69.mlp.experts.29.up_proj.weight": "model-00075-of-00101.safetensors", + "model.layers.69.mlp.experts.29.down_proj.weight": "model-00075-of-00101.safetensors", + "model.layers.69.mlp.experts.30.gate_proj.weight": "model-00075-of-00101.safetensors", + "model.layers.69.mlp.experts.30.up_proj.weight": "model-00075-of-00101.safetensors", + "model.layers.69.mlp.experts.30.down_proj.weight": "model-00075-of-00101.safetensors", + "model.layers.69.mlp.experts.31.gate_proj.weight": "model-00075-of-00101.safetensors", + "model.layers.69.mlp.experts.31.up_proj.weight": "model-00075-of-00101.safetensors", + "model.layers.69.mlp.experts.31.down_proj.weight": "model-00075-of-00101.safetensors", + "model.layers.69.mlp.experts.32.gate_proj.weight": "model-00075-of-00101.safetensors", + "model.layers.69.mlp.experts.32.up_proj.weight": "model-00075-of-00101.safetensors", + "model.layers.69.mlp.experts.32.down_proj.weight": "model-00075-of-00101.safetensors", + "model.layers.69.mlp.experts.33.gate_proj.weight": "model-00075-of-00101.safetensors", + "model.layers.69.mlp.experts.33.up_proj.weight": "model-00075-of-00101.safetensors", + "model.layers.69.mlp.experts.33.down_proj.weight": "model-00075-of-00101.safetensors", + "model.layers.69.mlp.experts.34.gate_proj.weight": "model-00075-of-00101.safetensors", + "model.layers.69.mlp.experts.34.up_proj.weight": "model-00075-of-00101.safetensors", + "model.layers.69.mlp.experts.34.down_proj.weight": "model-00075-of-00101.safetensors", + "model.layers.69.mlp.experts.35.gate_proj.weight": "model-00075-of-00101.safetensors", + "model.layers.69.mlp.experts.35.up_proj.weight": "model-00075-of-00101.safetensors", + "model.layers.69.mlp.experts.35.down_proj.weight": "model-00075-of-00101.safetensors", + "model.layers.69.mlp.experts.36.gate_proj.weight": "model-00075-of-00101.safetensors", + "model.layers.69.mlp.experts.36.up_proj.weight": "model-00075-of-00101.safetensors", + "model.layers.69.mlp.experts.36.down_proj.weight": "model-00075-of-00101.safetensors", + "model.layers.69.mlp.experts.37.gate_proj.weight": "model-00075-of-00101.safetensors", + "model.layers.69.mlp.experts.37.up_proj.weight": "model-00075-of-00101.safetensors", + "model.layers.69.mlp.experts.37.down_proj.weight": "model-00075-of-00101.safetensors", + "model.layers.69.mlp.experts.38.gate_proj.weight": "model-00075-of-00101.safetensors", + "model.layers.69.mlp.experts.38.up_proj.weight": "model-00075-of-00101.safetensors", + "model.layers.69.mlp.experts.38.down_proj.weight": "model-00075-of-00101.safetensors", + "model.layers.69.mlp.experts.39.gate_proj.weight": "model-00075-of-00101.safetensors", + "model.layers.69.mlp.experts.39.up_proj.weight": "model-00075-of-00101.safetensors", + "model.layers.69.mlp.experts.39.down_proj.weight": "model-00075-of-00101.safetensors", + "model.layers.69.mlp.experts.40.gate_proj.weight": "model-00075-of-00101.safetensors", + "model.layers.69.mlp.experts.40.up_proj.weight": "model-00075-of-00101.safetensors", + "model.layers.69.mlp.experts.40.down_proj.weight": "model-00075-of-00101.safetensors", + "model.layers.69.mlp.experts.41.gate_proj.weight": "model-00075-of-00101.safetensors", + "model.layers.69.mlp.experts.41.up_proj.weight": "model-00075-of-00101.safetensors", + "model.layers.69.mlp.experts.41.down_proj.weight": "model-00075-of-00101.safetensors", + "model.layers.69.mlp.experts.42.gate_proj.weight": "model-00075-of-00101.safetensors", + "model.layers.69.mlp.experts.42.up_proj.weight": "model-00075-of-00101.safetensors", + "model.layers.69.mlp.experts.42.down_proj.weight": "model-00075-of-00101.safetensors", + "model.layers.69.mlp.experts.43.gate_proj.weight": "model-00075-of-00101.safetensors", + "model.layers.69.mlp.experts.43.up_proj.weight": "model-00075-of-00101.safetensors", + "model.layers.69.mlp.experts.43.down_proj.weight": "model-00075-of-00101.safetensors", + "model.layers.69.mlp.experts.44.gate_proj.weight": "model-00075-of-00101.safetensors", + "model.layers.69.mlp.experts.44.up_proj.weight": "model-00075-of-00101.safetensors", + "model.layers.69.mlp.experts.44.down_proj.weight": "model-00075-of-00101.safetensors", + "model.layers.69.mlp.experts.45.gate_proj.weight": "model-00075-of-00101.safetensors", + "model.layers.69.mlp.experts.45.up_proj.weight": "model-00075-of-00101.safetensors", + "model.layers.69.mlp.experts.45.down_proj.weight": "model-00075-of-00101.safetensors", + "model.layers.69.mlp.experts.46.gate_proj.weight": "model-00075-of-00101.safetensors", + "model.layers.69.mlp.experts.46.up_proj.weight": "model-00075-of-00101.safetensors", + "model.layers.69.mlp.experts.46.down_proj.weight": "model-00075-of-00101.safetensors", + "model.layers.69.mlp.experts.47.gate_proj.weight": "model-00075-of-00101.safetensors", + "model.layers.69.mlp.experts.47.up_proj.weight": "model-00075-of-00101.safetensors", + "model.layers.69.mlp.experts.47.down_proj.weight": "model-00075-of-00101.safetensors", + "model.layers.69.mlp.experts.48.gate_proj.weight": "model-00075-of-00101.safetensors", + "model.layers.69.mlp.experts.48.up_proj.weight": "model-00075-of-00101.safetensors", + "model.layers.69.mlp.experts.48.down_proj.weight": "model-00075-of-00101.safetensors", + "model.layers.69.mlp.experts.49.gate_proj.weight": "model-00075-of-00101.safetensors", + "model.layers.69.mlp.experts.49.up_proj.weight": "model-00075-of-00101.safetensors", + "model.layers.69.mlp.experts.49.down_proj.weight": "model-00075-of-00101.safetensors", + "model.layers.69.mlp.experts.50.gate_proj.weight": "model-00075-of-00101.safetensors", + "model.layers.69.mlp.experts.50.up_proj.weight": "model-00075-of-00101.safetensors", + "model.layers.69.mlp.experts.50.down_proj.weight": "model-00075-of-00101.safetensors", + "model.layers.69.mlp.experts.51.gate_proj.weight": "model-00075-of-00101.safetensors", + "model.layers.69.mlp.experts.51.up_proj.weight": "model-00075-of-00101.safetensors", + "model.layers.69.mlp.experts.51.down_proj.weight": "model-00075-of-00101.safetensors", + "model.layers.69.mlp.experts.52.gate_proj.weight": "model-00075-of-00101.safetensors", + "model.layers.69.mlp.experts.52.up_proj.weight": "model-00075-of-00101.safetensors", + "model.layers.69.mlp.experts.52.down_proj.weight": "model-00075-of-00101.safetensors", + "model.layers.69.mlp.experts.53.gate_proj.weight": "model-00075-of-00101.safetensors", + "model.layers.69.mlp.experts.53.up_proj.weight": "model-00075-of-00101.safetensors", + "model.layers.69.mlp.experts.53.down_proj.weight": "model-00075-of-00101.safetensors", + "model.layers.69.mlp.experts.54.gate_proj.weight": "model-00075-of-00101.safetensors", + "model.layers.69.mlp.experts.54.up_proj.weight": "model-00075-of-00101.safetensors", + "model.layers.69.mlp.experts.54.down_proj.weight": "model-00075-of-00101.safetensors", + "model.layers.69.mlp.experts.55.gate_proj.weight": "model-00075-of-00101.safetensors", + "model.layers.69.mlp.experts.55.up_proj.weight": "model-00075-of-00101.safetensors", + "model.layers.69.mlp.experts.55.down_proj.weight": "model-00075-of-00101.safetensors", + "model.layers.69.mlp.experts.56.gate_proj.weight": "model-00075-of-00101.safetensors", + "model.layers.69.mlp.experts.56.up_proj.weight": "model-00075-of-00101.safetensors", + "model.layers.69.mlp.experts.56.down_proj.weight": "model-00075-of-00101.safetensors", + "model.layers.69.mlp.experts.57.gate_proj.weight": "model-00075-of-00101.safetensors", + "model.layers.69.mlp.experts.57.up_proj.weight": "model-00075-of-00101.safetensors", + "model.layers.69.mlp.experts.57.down_proj.weight": "model-00075-of-00101.safetensors", + "model.layers.69.mlp.experts.58.gate_proj.weight": "model-00075-of-00101.safetensors", + "model.layers.69.mlp.experts.58.up_proj.weight": "model-00075-of-00101.safetensors", + "model.layers.69.mlp.experts.58.down_proj.weight": "model-00075-of-00101.safetensors", + "model.layers.69.mlp.experts.59.gate_proj.weight": "model-00075-of-00101.safetensors", + "model.layers.69.mlp.experts.59.up_proj.weight": "model-00075-of-00101.safetensors", + "model.layers.69.mlp.experts.59.down_proj.weight": "model-00075-of-00101.safetensors", + "model.layers.69.mlp.experts.60.gate_proj.weight": "model-00075-of-00101.safetensors", + "model.layers.69.mlp.experts.60.up_proj.weight": "model-00075-of-00101.safetensors", + "model.layers.69.mlp.experts.60.down_proj.weight": "model-00075-of-00101.safetensors", + "model.layers.69.mlp.experts.61.gate_proj.weight": "model-00075-of-00101.safetensors", + "model.layers.69.mlp.experts.61.up_proj.weight": "model-00076-of-00101.safetensors", + "model.layers.69.mlp.experts.61.down_proj.weight": "model-00076-of-00101.safetensors", + "model.layers.69.mlp.experts.62.gate_proj.weight": "model-00076-of-00101.safetensors", + "model.layers.69.mlp.experts.62.up_proj.weight": "model-00076-of-00101.safetensors", + "model.layers.69.mlp.experts.62.down_proj.weight": "model-00076-of-00101.safetensors", + "model.layers.69.mlp.experts.63.gate_proj.weight": "model-00076-of-00101.safetensors", + "model.layers.69.mlp.experts.63.up_proj.weight": "model-00076-of-00101.safetensors", + "model.layers.69.mlp.experts.63.down_proj.weight": "model-00076-of-00101.safetensors", + "model.layers.69.mlp.experts.64.gate_proj.weight": "model-00076-of-00101.safetensors", + "model.layers.69.mlp.experts.64.up_proj.weight": "model-00076-of-00101.safetensors", + "model.layers.69.mlp.experts.64.down_proj.weight": "model-00076-of-00101.safetensors", + "model.layers.69.mlp.experts.65.gate_proj.weight": "model-00076-of-00101.safetensors", + "model.layers.69.mlp.experts.65.up_proj.weight": "model-00076-of-00101.safetensors", + "model.layers.69.mlp.experts.65.down_proj.weight": "model-00076-of-00101.safetensors", + "model.layers.69.mlp.experts.66.gate_proj.weight": "model-00076-of-00101.safetensors", + "model.layers.69.mlp.experts.66.up_proj.weight": "model-00076-of-00101.safetensors", + "model.layers.69.mlp.experts.66.down_proj.weight": "model-00076-of-00101.safetensors", + "model.layers.69.mlp.experts.67.gate_proj.weight": "model-00076-of-00101.safetensors", + "model.layers.69.mlp.experts.67.up_proj.weight": "model-00076-of-00101.safetensors", + "model.layers.69.mlp.experts.67.down_proj.weight": "model-00076-of-00101.safetensors", + "model.layers.69.mlp.experts.68.gate_proj.weight": "model-00076-of-00101.safetensors", + "model.layers.69.mlp.experts.68.up_proj.weight": "model-00076-of-00101.safetensors", + "model.layers.69.mlp.experts.68.down_proj.weight": "model-00076-of-00101.safetensors", + "model.layers.69.mlp.experts.69.gate_proj.weight": "model-00076-of-00101.safetensors", + "model.layers.69.mlp.experts.69.up_proj.weight": "model-00076-of-00101.safetensors", + "model.layers.69.mlp.experts.69.down_proj.weight": "model-00076-of-00101.safetensors", + "model.layers.69.mlp.experts.70.gate_proj.weight": "model-00076-of-00101.safetensors", + "model.layers.69.mlp.experts.70.up_proj.weight": "model-00076-of-00101.safetensors", + "model.layers.69.mlp.experts.70.down_proj.weight": "model-00076-of-00101.safetensors", + "model.layers.69.mlp.experts.71.gate_proj.weight": "model-00076-of-00101.safetensors", + "model.layers.69.mlp.experts.71.up_proj.weight": "model-00076-of-00101.safetensors", + "model.layers.69.mlp.experts.71.down_proj.weight": "model-00076-of-00101.safetensors", + "model.layers.69.mlp.experts.72.gate_proj.weight": "model-00076-of-00101.safetensors", + "model.layers.69.mlp.experts.72.up_proj.weight": "model-00076-of-00101.safetensors", + "model.layers.69.mlp.experts.72.down_proj.weight": "model-00076-of-00101.safetensors", + "model.layers.69.mlp.experts.73.gate_proj.weight": "model-00076-of-00101.safetensors", + "model.layers.69.mlp.experts.73.up_proj.weight": "model-00076-of-00101.safetensors", + "model.layers.69.mlp.experts.73.down_proj.weight": "model-00076-of-00101.safetensors", + "model.layers.69.mlp.experts.74.gate_proj.weight": "model-00076-of-00101.safetensors", + "model.layers.69.mlp.experts.74.up_proj.weight": "model-00076-of-00101.safetensors", + "model.layers.69.mlp.experts.74.down_proj.weight": "model-00076-of-00101.safetensors", + "model.layers.69.mlp.experts.75.gate_proj.weight": "model-00076-of-00101.safetensors", + "model.layers.69.mlp.experts.75.up_proj.weight": "model-00076-of-00101.safetensors", + "model.layers.69.mlp.experts.75.down_proj.weight": "model-00076-of-00101.safetensors", + "model.layers.69.mlp.experts.76.gate_proj.weight": "model-00076-of-00101.safetensors", + "model.layers.69.mlp.experts.76.up_proj.weight": "model-00076-of-00101.safetensors", + "model.layers.69.mlp.experts.76.down_proj.weight": "model-00076-of-00101.safetensors", + "model.layers.69.mlp.experts.77.gate_proj.weight": "model-00076-of-00101.safetensors", + "model.layers.69.mlp.experts.77.up_proj.weight": "model-00076-of-00101.safetensors", + "model.layers.69.mlp.experts.77.down_proj.weight": "model-00076-of-00101.safetensors", + "model.layers.69.mlp.experts.78.gate_proj.weight": "model-00076-of-00101.safetensors", + "model.layers.69.mlp.experts.78.up_proj.weight": "model-00076-of-00101.safetensors", + "model.layers.69.mlp.experts.78.down_proj.weight": "model-00076-of-00101.safetensors", + "model.layers.69.mlp.experts.79.gate_proj.weight": "model-00076-of-00101.safetensors", + "model.layers.69.mlp.experts.79.up_proj.weight": "model-00076-of-00101.safetensors", + "model.layers.69.mlp.experts.79.down_proj.weight": "model-00076-of-00101.safetensors", + "model.layers.69.mlp.experts.80.gate_proj.weight": "model-00076-of-00101.safetensors", + "model.layers.69.mlp.experts.80.up_proj.weight": "model-00076-of-00101.safetensors", + "model.layers.69.mlp.experts.80.down_proj.weight": "model-00076-of-00101.safetensors", + "model.layers.69.mlp.experts.81.gate_proj.weight": "model-00076-of-00101.safetensors", + "model.layers.69.mlp.experts.81.up_proj.weight": "model-00076-of-00101.safetensors", + "model.layers.69.mlp.experts.81.down_proj.weight": "model-00076-of-00101.safetensors", + "model.layers.69.mlp.experts.82.gate_proj.weight": "model-00076-of-00101.safetensors", + "model.layers.69.mlp.experts.82.up_proj.weight": "model-00076-of-00101.safetensors", + "model.layers.69.mlp.experts.82.down_proj.weight": "model-00076-of-00101.safetensors", + "model.layers.69.mlp.experts.83.gate_proj.weight": "model-00076-of-00101.safetensors", + "model.layers.69.mlp.experts.83.up_proj.weight": "model-00076-of-00101.safetensors", + "model.layers.69.mlp.experts.83.down_proj.weight": "model-00076-of-00101.safetensors", + "model.layers.69.mlp.experts.84.gate_proj.weight": "model-00076-of-00101.safetensors", + "model.layers.69.mlp.experts.84.up_proj.weight": "model-00076-of-00101.safetensors", + "model.layers.69.mlp.experts.84.down_proj.weight": "model-00076-of-00101.safetensors", + "model.layers.69.mlp.experts.85.gate_proj.weight": "model-00076-of-00101.safetensors", + "model.layers.69.mlp.experts.85.up_proj.weight": "model-00076-of-00101.safetensors", + "model.layers.69.mlp.experts.85.down_proj.weight": "model-00076-of-00101.safetensors", + "model.layers.69.mlp.experts.86.gate_proj.weight": "model-00076-of-00101.safetensors", + "model.layers.69.mlp.experts.86.up_proj.weight": "model-00076-of-00101.safetensors", + "model.layers.69.mlp.experts.86.down_proj.weight": "model-00076-of-00101.safetensors", + "model.layers.69.mlp.experts.87.gate_proj.weight": "model-00076-of-00101.safetensors", + "model.layers.69.mlp.experts.87.up_proj.weight": "model-00076-of-00101.safetensors", + "model.layers.69.mlp.experts.87.down_proj.weight": "model-00076-of-00101.safetensors", + "model.layers.69.mlp.experts.88.gate_proj.weight": "model-00076-of-00101.safetensors", + "model.layers.69.mlp.experts.88.up_proj.weight": "model-00076-of-00101.safetensors", + "model.layers.69.mlp.experts.88.down_proj.weight": "model-00076-of-00101.safetensors", + "model.layers.69.mlp.experts.89.gate_proj.weight": "model-00076-of-00101.safetensors", + "model.layers.69.mlp.experts.89.up_proj.weight": "model-00076-of-00101.safetensors", + "model.layers.69.mlp.experts.89.down_proj.weight": "model-00076-of-00101.safetensors", + "model.layers.69.mlp.experts.90.gate_proj.weight": "model-00076-of-00101.safetensors", + "model.layers.69.mlp.experts.90.up_proj.weight": "model-00076-of-00101.safetensors", + "model.layers.69.mlp.experts.90.down_proj.weight": "model-00076-of-00101.safetensors", + "model.layers.69.mlp.experts.91.gate_proj.weight": "model-00076-of-00101.safetensors", + "model.layers.69.mlp.experts.91.up_proj.weight": "model-00076-of-00101.safetensors", + "model.layers.69.mlp.experts.91.down_proj.weight": "model-00076-of-00101.safetensors", + "model.layers.69.mlp.experts.92.gate_proj.weight": "model-00076-of-00101.safetensors", + "model.layers.69.mlp.experts.92.up_proj.weight": "model-00076-of-00101.safetensors", + "model.layers.69.mlp.experts.92.down_proj.weight": "model-00076-of-00101.safetensors", + "model.layers.69.mlp.experts.93.gate_proj.weight": "model-00076-of-00101.safetensors", + "model.layers.69.mlp.experts.93.up_proj.weight": "model-00076-of-00101.safetensors", + "model.layers.69.mlp.experts.93.down_proj.weight": "model-00076-of-00101.safetensors", + "model.layers.69.mlp.experts.94.gate_proj.weight": "model-00076-of-00101.safetensors", + "model.layers.69.mlp.experts.94.up_proj.weight": "model-00076-of-00101.safetensors", + "model.layers.69.mlp.experts.94.down_proj.weight": "model-00076-of-00101.safetensors", + "model.layers.69.mlp.experts.95.gate_proj.weight": "model-00076-of-00101.safetensors", + "model.layers.69.mlp.experts.95.up_proj.weight": "model-00076-of-00101.safetensors", + "model.layers.69.mlp.experts.95.down_proj.weight": "model-00076-of-00101.safetensors", + "model.layers.69.mlp.experts.96.gate_proj.weight": "model-00076-of-00101.safetensors", + "model.layers.69.mlp.experts.96.up_proj.weight": "model-00076-of-00101.safetensors", + "model.layers.69.mlp.experts.96.down_proj.weight": "model-00076-of-00101.safetensors", + "model.layers.69.mlp.experts.97.gate_proj.weight": "model-00076-of-00101.safetensors", + "model.layers.69.mlp.experts.97.up_proj.weight": "model-00076-of-00101.safetensors", + "model.layers.69.mlp.experts.97.down_proj.weight": "model-00076-of-00101.safetensors", + "model.layers.69.mlp.experts.98.gate_proj.weight": "model-00076-of-00101.safetensors", + "model.layers.69.mlp.experts.98.up_proj.weight": "model-00076-of-00101.safetensors", + "model.layers.69.mlp.experts.98.down_proj.weight": "model-00076-of-00101.safetensors", + "model.layers.69.mlp.experts.99.gate_proj.weight": "model-00076-of-00101.safetensors", + "model.layers.69.mlp.experts.99.up_proj.weight": "model-00076-of-00101.safetensors", + "model.layers.69.mlp.experts.99.down_proj.weight": "model-00076-of-00101.safetensors", + "model.layers.69.mlp.experts.100.gate_proj.weight": "model-00076-of-00101.safetensors", + "model.layers.69.mlp.experts.100.up_proj.weight": "model-00076-of-00101.safetensors", + "model.layers.69.mlp.experts.100.down_proj.weight": "model-00076-of-00101.safetensors", + "model.layers.69.mlp.experts.101.gate_proj.weight": "model-00076-of-00101.safetensors", + "model.layers.69.mlp.experts.101.up_proj.weight": "model-00076-of-00101.safetensors", + "model.layers.69.mlp.experts.101.down_proj.weight": "model-00076-of-00101.safetensors", + "model.layers.69.mlp.experts.102.gate_proj.weight": "model-00076-of-00101.safetensors", + "model.layers.69.mlp.experts.102.up_proj.weight": "model-00076-of-00101.safetensors", + "model.layers.69.mlp.experts.102.down_proj.weight": "model-00076-of-00101.safetensors", + "model.layers.69.mlp.experts.103.gate_proj.weight": "model-00076-of-00101.safetensors", + "model.layers.69.mlp.experts.103.up_proj.weight": "model-00076-of-00101.safetensors", + "model.layers.69.mlp.experts.103.down_proj.weight": "model-00076-of-00101.safetensors", + "model.layers.69.mlp.experts.104.gate_proj.weight": "model-00076-of-00101.safetensors", + "model.layers.69.mlp.experts.104.up_proj.weight": "model-00076-of-00101.safetensors", + "model.layers.69.mlp.experts.104.down_proj.weight": "model-00076-of-00101.safetensors", + "model.layers.69.mlp.experts.105.gate_proj.weight": "model-00076-of-00101.safetensors", + "model.layers.69.mlp.experts.105.up_proj.weight": "model-00076-of-00101.safetensors", + "model.layers.69.mlp.experts.105.down_proj.weight": "model-00076-of-00101.safetensors", + "model.layers.69.mlp.experts.106.gate_proj.weight": "model-00076-of-00101.safetensors", + "model.layers.69.mlp.experts.106.up_proj.weight": "model-00076-of-00101.safetensors", + "model.layers.69.mlp.experts.106.down_proj.weight": "model-00076-of-00101.safetensors", + "model.layers.69.mlp.experts.107.gate_proj.weight": "model-00076-of-00101.safetensors", + "model.layers.69.mlp.experts.107.up_proj.weight": "model-00076-of-00101.safetensors", + "model.layers.69.mlp.experts.107.down_proj.weight": "model-00076-of-00101.safetensors", + "model.layers.69.mlp.experts.108.gate_proj.weight": "model-00076-of-00101.safetensors", + "model.layers.69.mlp.experts.108.up_proj.weight": "model-00076-of-00101.safetensors", + "model.layers.69.mlp.experts.108.down_proj.weight": "model-00076-of-00101.safetensors", + "model.layers.69.mlp.experts.109.gate_proj.weight": "model-00076-of-00101.safetensors", + "model.layers.69.mlp.experts.109.up_proj.weight": "model-00076-of-00101.safetensors", + "model.layers.69.mlp.experts.109.down_proj.weight": "model-00076-of-00101.safetensors", + "model.layers.69.mlp.experts.110.gate_proj.weight": "model-00076-of-00101.safetensors", + "model.layers.69.mlp.experts.110.up_proj.weight": "model-00076-of-00101.safetensors", + "model.layers.69.mlp.experts.110.down_proj.weight": "model-00076-of-00101.safetensors", + "model.layers.69.mlp.experts.111.gate_proj.weight": "model-00076-of-00101.safetensors", + "model.layers.69.mlp.experts.111.up_proj.weight": "model-00076-of-00101.safetensors", + "model.layers.69.mlp.experts.111.down_proj.weight": "model-00076-of-00101.safetensors", + "model.layers.69.mlp.experts.112.gate_proj.weight": "model-00076-of-00101.safetensors", + "model.layers.69.mlp.experts.112.up_proj.weight": "model-00076-of-00101.safetensors", + "model.layers.69.mlp.experts.112.down_proj.weight": "model-00076-of-00101.safetensors", + "model.layers.69.mlp.experts.113.gate_proj.weight": "model-00076-of-00101.safetensors", + "model.layers.69.mlp.experts.113.up_proj.weight": "model-00076-of-00101.safetensors", + "model.layers.69.mlp.experts.113.down_proj.weight": "model-00076-of-00101.safetensors", + "model.layers.69.mlp.experts.114.gate_proj.weight": "model-00076-of-00101.safetensors", + "model.layers.69.mlp.experts.114.up_proj.weight": "model-00076-of-00101.safetensors", + "model.layers.69.mlp.experts.114.down_proj.weight": "model-00076-of-00101.safetensors", + "model.layers.69.mlp.experts.115.gate_proj.weight": "model-00076-of-00101.safetensors", + "model.layers.69.mlp.experts.115.up_proj.weight": "model-00076-of-00101.safetensors", + "model.layers.69.mlp.experts.115.down_proj.weight": "model-00076-of-00101.safetensors", + "model.layers.69.mlp.experts.116.gate_proj.weight": "model-00076-of-00101.safetensors", + "model.layers.69.mlp.experts.116.up_proj.weight": "model-00076-of-00101.safetensors", + "model.layers.69.mlp.experts.116.down_proj.weight": "model-00076-of-00101.safetensors", + "model.layers.69.mlp.experts.117.gate_proj.weight": "model-00076-of-00101.safetensors", + "model.layers.69.mlp.experts.117.up_proj.weight": "model-00076-of-00101.safetensors", + "model.layers.69.mlp.experts.117.down_proj.weight": "model-00076-of-00101.safetensors", + "model.layers.69.mlp.experts.118.gate_proj.weight": "model-00076-of-00101.safetensors", + "model.layers.69.mlp.experts.118.up_proj.weight": "model-00076-of-00101.safetensors", + "model.layers.69.mlp.experts.118.down_proj.weight": "model-00076-of-00101.safetensors", + "model.layers.69.mlp.experts.119.gate_proj.weight": "model-00076-of-00101.safetensors", + "model.layers.69.mlp.experts.119.up_proj.weight": "model-00076-of-00101.safetensors", + "model.layers.69.mlp.experts.119.down_proj.weight": "model-00076-of-00101.safetensors", + "model.layers.69.mlp.gate.weight": "model-00076-of-00101.safetensors", + "model.layers.69.mlp.gate.e_score_correction_bias": "model-00076-of-00101.safetensors", + "model.layers.69.mlp.shared_experts.gate_proj.weight": "model-00076-of-00101.safetensors", + "model.layers.69.mlp.shared_experts.up_proj.weight": "model-00076-of-00101.safetensors", + "model.layers.69.mlp.shared_experts.down_proj.weight": "model-00076-of-00101.safetensors", + "model.layers.69.input_layernorm.weight": "model-00076-of-00101.safetensors", + "model.layers.69.post_attention_layernorm.weight": "model-00076-of-00101.safetensors", + "model.layers.70.self_attn.q_proj.weight": "model-00076-of-00101.safetensors", + "model.layers.70.self_attn.q_proj.bias": "model-00076-of-00101.safetensors", + "model.layers.70.self_attn.k_proj.weight": "model-00076-of-00101.safetensors", + "model.layers.70.self_attn.k_proj.bias": "model-00076-of-00101.safetensors", + "model.layers.70.self_attn.v_proj.weight": "model-00076-of-00101.safetensors", + "model.layers.70.self_attn.v_proj.bias": "model-00076-of-00101.safetensors", + "model.layers.70.self_attn.o_proj.weight": "model-00076-of-00101.safetensors", + "model.layers.70.self_attn.q_norm.weight": "model-00076-of-00101.safetensors", + "model.layers.70.self_attn.k_norm.weight": "model-00076-of-00101.safetensors", + "model.layers.70.mlp.experts.0.gate_proj.weight": "model-00076-of-00101.safetensors", + "model.layers.70.mlp.experts.0.up_proj.weight": "model-00076-of-00101.safetensors", + "model.layers.70.mlp.experts.0.down_proj.weight": "model-00076-of-00101.safetensors", + "model.layers.70.mlp.experts.1.gate_proj.weight": "model-00076-of-00101.safetensors", + "model.layers.70.mlp.experts.1.up_proj.weight": "model-00076-of-00101.safetensors", + "model.layers.70.mlp.experts.1.down_proj.weight": "model-00076-of-00101.safetensors", + "model.layers.70.mlp.experts.2.gate_proj.weight": "model-00076-of-00101.safetensors", + "model.layers.70.mlp.experts.2.up_proj.weight": "model-00076-of-00101.safetensors", + "model.layers.70.mlp.experts.2.down_proj.weight": "model-00076-of-00101.safetensors", + "model.layers.70.mlp.experts.3.gate_proj.weight": "model-00076-of-00101.safetensors", + "model.layers.70.mlp.experts.3.up_proj.weight": "model-00076-of-00101.safetensors", + "model.layers.70.mlp.experts.3.down_proj.weight": "model-00076-of-00101.safetensors", + "model.layers.70.mlp.experts.4.gate_proj.weight": "model-00076-of-00101.safetensors", + "model.layers.70.mlp.experts.4.up_proj.weight": "model-00076-of-00101.safetensors", + "model.layers.70.mlp.experts.4.down_proj.weight": "model-00076-of-00101.safetensors", + "model.layers.70.mlp.experts.5.gate_proj.weight": "model-00076-of-00101.safetensors", + "model.layers.70.mlp.experts.5.up_proj.weight": "model-00076-of-00101.safetensors", + "model.layers.70.mlp.experts.5.down_proj.weight": "model-00076-of-00101.safetensors", + "model.layers.70.mlp.experts.6.gate_proj.weight": "model-00076-of-00101.safetensors", + "model.layers.70.mlp.experts.6.up_proj.weight": "model-00076-of-00101.safetensors", + "model.layers.70.mlp.experts.6.down_proj.weight": "model-00076-of-00101.safetensors", + "model.layers.70.mlp.experts.7.gate_proj.weight": "model-00076-of-00101.safetensors", + "model.layers.70.mlp.experts.7.up_proj.weight": "model-00076-of-00101.safetensors", + "model.layers.70.mlp.experts.7.down_proj.weight": "model-00076-of-00101.safetensors", + "model.layers.70.mlp.experts.8.gate_proj.weight": "model-00076-of-00101.safetensors", + "model.layers.70.mlp.experts.8.up_proj.weight": "model-00076-of-00101.safetensors", + "model.layers.70.mlp.experts.8.down_proj.weight": "model-00076-of-00101.safetensors", + "model.layers.70.mlp.experts.9.gate_proj.weight": "model-00076-of-00101.safetensors", + "model.layers.70.mlp.experts.9.up_proj.weight": "model-00076-of-00101.safetensors", + "model.layers.70.mlp.experts.9.down_proj.weight": "model-00076-of-00101.safetensors", + "model.layers.70.mlp.experts.10.gate_proj.weight": "model-00076-of-00101.safetensors", + "model.layers.70.mlp.experts.10.up_proj.weight": "model-00076-of-00101.safetensors", + "model.layers.70.mlp.experts.10.down_proj.weight": "model-00076-of-00101.safetensors", + "model.layers.70.mlp.experts.11.gate_proj.weight": "model-00076-of-00101.safetensors", + "model.layers.70.mlp.experts.11.up_proj.weight": "model-00076-of-00101.safetensors", + "model.layers.70.mlp.experts.11.down_proj.weight": "model-00076-of-00101.safetensors", + "model.layers.70.mlp.experts.12.gate_proj.weight": "model-00076-of-00101.safetensors", + "model.layers.70.mlp.experts.12.up_proj.weight": "model-00076-of-00101.safetensors", + "model.layers.70.mlp.experts.12.down_proj.weight": "model-00076-of-00101.safetensors", + "model.layers.70.mlp.experts.13.gate_proj.weight": "model-00076-of-00101.safetensors", + "model.layers.70.mlp.experts.13.up_proj.weight": "model-00076-of-00101.safetensors", + "model.layers.70.mlp.experts.13.down_proj.weight": "model-00076-of-00101.safetensors", + "model.layers.70.mlp.experts.14.gate_proj.weight": "model-00076-of-00101.safetensors", + "model.layers.70.mlp.experts.14.up_proj.weight": "model-00076-of-00101.safetensors", + "model.layers.70.mlp.experts.14.down_proj.weight": "model-00076-of-00101.safetensors", + "model.layers.70.mlp.experts.15.gate_proj.weight": "model-00076-of-00101.safetensors", + "model.layers.70.mlp.experts.15.up_proj.weight": "model-00076-of-00101.safetensors", + "model.layers.70.mlp.experts.15.down_proj.weight": "model-00076-of-00101.safetensors", + "model.layers.70.mlp.experts.16.gate_proj.weight": "model-00076-of-00101.safetensors", + "model.layers.70.mlp.experts.16.up_proj.weight": "model-00076-of-00101.safetensors", + "model.layers.70.mlp.experts.16.down_proj.weight": "model-00076-of-00101.safetensors", + "model.layers.70.mlp.experts.17.gate_proj.weight": "model-00076-of-00101.safetensors", + "model.layers.70.mlp.experts.17.up_proj.weight": "model-00076-of-00101.safetensors", + "model.layers.70.mlp.experts.17.down_proj.weight": "model-00076-of-00101.safetensors", + "model.layers.70.mlp.experts.18.gate_proj.weight": "model-00076-of-00101.safetensors", + "model.layers.70.mlp.experts.18.up_proj.weight": "model-00076-of-00101.safetensors", + "model.layers.70.mlp.experts.18.down_proj.weight": "model-00076-of-00101.safetensors", + "model.layers.70.mlp.experts.19.gate_proj.weight": "model-00076-of-00101.safetensors", + "model.layers.70.mlp.experts.19.up_proj.weight": "model-00076-of-00101.safetensors", + "model.layers.70.mlp.experts.19.down_proj.weight": "model-00076-of-00101.safetensors", + "model.layers.70.mlp.experts.20.gate_proj.weight": "model-00076-of-00101.safetensors", + "model.layers.70.mlp.experts.20.up_proj.weight": "model-00076-of-00101.safetensors", + "model.layers.70.mlp.experts.20.down_proj.weight": "model-00076-of-00101.safetensors", + "model.layers.70.mlp.experts.21.gate_proj.weight": "model-00076-of-00101.safetensors", + "model.layers.70.mlp.experts.21.up_proj.weight": "model-00076-of-00101.safetensors", + "model.layers.70.mlp.experts.21.down_proj.weight": "model-00076-of-00101.safetensors", + "model.layers.70.mlp.experts.22.gate_proj.weight": "model-00076-of-00101.safetensors", + "model.layers.70.mlp.experts.22.up_proj.weight": "model-00076-of-00101.safetensors", + "model.layers.70.mlp.experts.22.down_proj.weight": "model-00076-of-00101.safetensors", + "model.layers.70.mlp.experts.23.gate_proj.weight": "model-00076-of-00101.safetensors", + "model.layers.70.mlp.experts.23.up_proj.weight": "model-00076-of-00101.safetensors", + "model.layers.70.mlp.experts.23.down_proj.weight": "model-00076-of-00101.safetensors", + "model.layers.70.mlp.experts.24.gate_proj.weight": "model-00076-of-00101.safetensors", + "model.layers.70.mlp.experts.24.up_proj.weight": "model-00076-of-00101.safetensors", + "model.layers.70.mlp.experts.24.down_proj.weight": "model-00076-of-00101.safetensors", + "model.layers.70.mlp.experts.25.gate_proj.weight": "model-00076-of-00101.safetensors", + "model.layers.70.mlp.experts.25.up_proj.weight": "model-00076-of-00101.safetensors", + "model.layers.70.mlp.experts.25.down_proj.weight": "model-00076-of-00101.safetensors", + "model.layers.70.mlp.experts.26.gate_proj.weight": "model-00076-of-00101.safetensors", + "model.layers.70.mlp.experts.26.up_proj.weight": "model-00076-of-00101.safetensors", + "model.layers.70.mlp.experts.26.down_proj.weight": "model-00076-of-00101.safetensors", + "model.layers.70.mlp.experts.27.gate_proj.weight": "model-00076-of-00101.safetensors", + "model.layers.70.mlp.experts.27.up_proj.weight": "model-00076-of-00101.safetensors", + "model.layers.70.mlp.experts.27.down_proj.weight": "model-00076-of-00101.safetensors", + "model.layers.70.mlp.experts.28.gate_proj.weight": "model-00076-of-00101.safetensors", + "model.layers.70.mlp.experts.28.up_proj.weight": "model-00076-of-00101.safetensors", + "model.layers.70.mlp.experts.28.down_proj.weight": "model-00076-of-00101.safetensors", + "model.layers.70.mlp.experts.29.gate_proj.weight": "model-00076-of-00101.safetensors", + "model.layers.70.mlp.experts.29.up_proj.weight": "model-00076-of-00101.safetensors", + "model.layers.70.mlp.experts.29.down_proj.weight": "model-00076-of-00101.safetensors", + "model.layers.70.mlp.experts.30.gate_proj.weight": "model-00076-of-00101.safetensors", + "model.layers.70.mlp.experts.30.up_proj.weight": "model-00076-of-00101.safetensors", + "model.layers.70.mlp.experts.30.down_proj.weight": "model-00076-of-00101.safetensors", + "model.layers.70.mlp.experts.31.gate_proj.weight": "model-00076-of-00101.safetensors", + "model.layers.70.mlp.experts.31.up_proj.weight": "model-00076-of-00101.safetensors", + "model.layers.70.mlp.experts.31.down_proj.weight": "model-00076-of-00101.safetensors", + "model.layers.70.mlp.experts.32.gate_proj.weight": "model-00076-of-00101.safetensors", + "model.layers.70.mlp.experts.32.up_proj.weight": "model-00076-of-00101.safetensors", + "model.layers.70.mlp.experts.32.down_proj.weight": "model-00076-of-00101.safetensors", + "model.layers.70.mlp.experts.33.gate_proj.weight": "model-00076-of-00101.safetensors", + "model.layers.70.mlp.experts.33.up_proj.weight": "model-00076-of-00101.safetensors", + "model.layers.70.mlp.experts.33.down_proj.weight": "model-00076-of-00101.safetensors", + "model.layers.70.mlp.experts.34.gate_proj.weight": "model-00076-of-00101.safetensors", + "model.layers.70.mlp.experts.34.up_proj.weight": "model-00076-of-00101.safetensors", + "model.layers.70.mlp.experts.34.down_proj.weight": "model-00076-of-00101.safetensors", + "model.layers.70.mlp.experts.35.gate_proj.weight": "model-00076-of-00101.safetensors", + "model.layers.70.mlp.experts.35.up_proj.weight": "model-00076-of-00101.safetensors", + "model.layers.70.mlp.experts.35.down_proj.weight": "model-00076-of-00101.safetensors", + "model.layers.70.mlp.experts.36.gate_proj.weight": "model-00076-of-00101.safetensors", + "model.layers.70.mlp.experts.36.up_proj.weight": "model-00076-of-00101.safetensors", + "model.layers.70.mlp.experts.36.down_proj.weight": "model-00076-of-00101.safetensors", + "model.layers.70.mlp.experts.37.gate_proj.weight": "model-00076-of-00101.safetensors", + "model.layers.70.mlp.experts.37.up_proj.weight": "model-00076-of-00101.safetensors", + "model.layers.70.mlp.experts.37.down_proj.weight": "model-00076-of-00101.safetensors", + "model.layers.70.mlp.experts.38.gate_proj.weight": "model-00076-of-00101.safetensors", + "model.layers.70.mlp.experts.38.up_proj.weight": "model-00076-of-00101.safetensors", + "model.layers.70.mlp.experts.38.down_proj.weight": "model-00076-of-00101.safetensors", + "model.layers.70.mlp.experts.39.gate_proj.weight": "model-00076-of-00101.safetensors", + "model.layers.70.mlp.experts.39.up_proj.weight": "model-00076-of-00101.safetensors", + "model.layers.70.mlp.experts.39.down_proj.weight": "model-00076-of-00101.safetensors", + "model.layers.70.mlp.experts.40.gate_proj.weight": "model-00076-of-00101.safetensors", + "model.layers.70.mlp.experts.40.up_proj.weight": "model-00076-of-00101.safetensors", + "model.layers.70.mlp.experts.40.down_proj.weight": "model-00076-of-00101.safetensors", + "model.layers.70.mlp.experts.41.gate_proj.weight": "model-00076-of-00101.safetensors", + "model.layers.70.mlp.experts.41.up_proj.weight": "model-00076-of-00101.safetensors", + "model.layers.70.mlp.experts.41.down_proj.weight": "model-00076-of-00101.safetensors", + "model.layers.70.mlp.experts.42.gate_proj.weight": "model-00076-of-00101.safetensors", + "model.layers.70.mlp.experts.42.up_proj.weight": "model-00076-of-00101.safetensors", + "model.layers.70.mlp.experts.42.down_proj.weight": "model-00076-of-00101.safetensors", + "model.layers.70.mlp.experts.43.gate_proj.weight": "model-00076-of-00101.safetensors", + "model.layers.70.mlp.experts.43.up_proj.weight": "model-00076-of-00101.safetensors", + "model.layers.70.mlp.experts.43.down_proj.weight": "model-00076-of-00101.safetensors", + "model.layers.70.mlp.experts.44.gate_proj.weight": "model-00076-of-00101.safetensors", + "model.layers.70.mlp.experts.44.up_proj.weight": "model-00076-of-00101.safetensors", + "model.layers.70.mlp.experts.44.down_proj.weight": "model-00076-of-00101.safetensors", + "model.layers.70.mlp.experts.45.gate_proj.weight": "model-00076-of-00101.safetensors", + "model.layers.70.mlp.experts.45.up_proj.weight": "model-00076-of-00101.safetensors", + "model.layers.70.mlp.experts.45.down_proj.weight": "model-00076-of-00101.safetensors", + "model.layers.70.mlp.experts.46.gate_proj.weight": "model-00076-of-00101.safetensors", + "model.layers.70.mlp.experts.46.up_proj.weight": "model-00076-of-00101.safetensors", + "model.layers.70.mlp.experts.46.down_proj.weight": "model-00076-of-00101.safetensors", + "model.layers.70.mlp.experts.47.gate_proj.weight": "model-00076-of-00101.safetensors", + "model.layers.70.mlp.experts.47.up_proj.weight": "model-00076-of-00101.safetensors", + "model.layers.70.mlp.experts.47.down_proj.weight": "model-00076-of-00101.safetensors", + "model.layers.70.mlp.experts.48.gate_proj.weight": "model-00077-of-00101.safetensors", + "model.layers.70.mlp.experts.48.up_proj.weight": "model-00077-of-00101.safetensors", + "model.layers.70.mlp.experts.48.down_proj.weight": "model-00077-of-00101.safetensors", + "model.layers.70.mlp.experts.49.gate_proj.weight": "model-00077-of-00101.safetensors", + "model.layers.70.mlp.experts.49.up_proj.weight": "model-00077-of-00101.safetensors", + "model.layers.70.mlp.experts.49.down_proj.weight": "model-00077-of-00101.safetensors", + "model.layers.70.mlp.experts.50.gate_proj.weight": "model-00077-of-00101.safetensors", + "model.layers.70.mlp.experts.50.up_proj.weight": "model-00077-of-00101.safetensors", + "model.layers.70.mlp.experts.50.down_proj.weight": "model-00077-of-00101.safetensors", + "model.layers.70.mlp.experts.51.gate_proj.weight": "model-00077-of-00101.safetensors", + "model.layers.70.mlp.experts.51.up_proj.weight": "model-00077-of-00101.safetensors", + "model.layers.70.mlp.experts.51.down_proj.weight": "model-00077-of-00101.safetensors", + "model.layers.70.mlp.experts.52.gate_proj.weight": "model-00077-of-00101.safetensors", + "model.layers.70.mlp.experts.52.up_proj.weight": "model-00077-of-00101.safetensors", + "model.layers.70.mlp.experts.52.down_proj.weight": "model-00077-of-00101.safetensors", + "model.layers.70.mlp.experts.53.gate_proj.weight": "model-00077-of-00101.safetensors", + "model.layers.70.mlp.experts.53.up_proj.weight": "model-00077-of-00101.safetensors", + "model.layers.70.mlp.experts.53.down_proj.weight": "model-00077-of-00101.safetensors", + "model.layers.70.mlp.experts.54.gate_proj.weight": "model-00077-of-00101.safetensors", + "model.layers.70.mlp.experts.54.up_proj.weight": "model-00077-of-00101.safetensors", + "model.layers.70.mlp.experts.54.down_proj.weight": "model-00077-of-00101.safetensors", + "model.layers.70.mlp.experts.55.gate_proj.weight": "model-00077-of-00101.safetensors", + "model.layers.70.mlp.experts.55.up_proj.weight": "model-00077-of-00101.safetensors", + "model.layers.70.mlp.experts.55.down_proj.weight": "model-00077-of-00101.safetensors", + "model.layers.70.mlp.experts.56.gate_proj.weight": "model-00077-of-00101.safetensors", + "model.layers.70.mlp.experts.56.up_proj.weight": "model-00077-of-00101.safetensors", + "model.layers.70.mlp.experts.56.down_proj.weight": "model-00077-of-00101.safetensors", + "model.layers.70.mlp.experts.57.gate_proj.weight": "model-00077-of-00101.safetensors", + "model.layers.70.mlp.experts.57.up_proj.weight": "model-00077-of-00101.safetensors", + "model.layers.70.mlp.experts.57.down_proj.weight": "model-00077-of-00101.safetensors", + "model.layers.70.mlp.experts.58.gate_proj.weight": "model-00077-of-00101.safetensors", + "model.layers.70.mlp.experts.58.up_proj.weight": "model-00077-of-00101.safetensors", + "model.layers.70.mlp.experts.58.down_proj.weight": "model-00077-of-00101.safetensors", + "model.layers.70.mlp.experts.59.gate_proj.weight": "model-00077-of-00101.safetensors", + "model.layers.70.mlp.experts.59.up_proj.weight": "model-00077-of-00101.safetensors", + "model.layers.70.mlp.experts.59.down_proj.weight": "model-00077-of-00101.safetensors", + "model.layers.70.mlp.experts.60.gate_proj.weight": "model-00077-of-00101.safetensors", + "model.layers.70.mlp.experts.60.up_proj.weight": "model-00077-of-00101.safetensors", + "model.layers.70.mlp.experts.60.down_proj.weight": "model-00077-of-00101.safetensors", + "model.layers.70.mlp.experts.61.gate_proj.weight": "model-00077-of-00101.safetensors", + "model.layers.70.mlp.experts.61.up_proj.weight": "model-00077-of-00101.safetensors", + "model.layers.70.mlp.experts.61.down_proj.weight": "model-00077-of-00101.safetensors", + "model.layers.70.mlp.experts.62.gate_proj.weight": "model-00077-of-00101.safetensors", + "model.layers.70.mlp.experts.62.up_proj.weight": "model-00077-of-00101.safetensors", + "model.layers.70.mlp.experts.62.down_proj.weight": "model-00077-of-00101.safetensors", + "model.layers.70.mlp.experts.63.gate_proj.weight": "model-00077-of-00101.safetensors", + "model.layers.70.mlp.experts.63.up_proj.weight": "model-00077-of-00101.safetensors", + "model.layers.70.mlp.experts.63.down_proj.weight": "model-00077-of-00101.safetensors", + "model.layers.70.mlp.experts.64.gate_proj.weight": "model-00077-of-00101.safetensors", + "model.layers.70.mlp.experts.64.up_proj.weight": "model-00077-of-00101.safetensors", + "model.layers.70.mlp.experts.64.down_proj.weight": "model-00077-of-00101.safetensors", + "model.layers.70.mlp.experts.65.gate_proj.weight": "model-00077-of-00101.safetensors", + "model.layers.70.mlp.experts.65.up_proj.weight": "model-00077-of-00101.safetensors", + "model.layers.70.mlp.experts.65.down_proj.weight": "model-00077-of-00101.safetensors", + "model.layers.70.mlp.experts.66.gate_proj.weight": "model-00077-of-00101.safetensors", + "model.layers.70.mlp.experts.66.up_proj.weight": "model-00077-of-00101.safetensors", + "model.layers.70.mlp.experts.66.down_proj.weight": "model-00077-of-00101.safetensors", + "model.layers.70.mlp.experts.67.gate_proj.weight": "model-00077-of-00101.safetensors", + "model.layers.70.mlp.experts.67.up_proj.weight": "model-00077-of-00101.safetensors", + "model.layers.70.mlp.experts.67.down_proj.weight": "model-00077-of-00101.safetensors", + "model.layers.70.mlp.experts.68.gate_proj.weight": "model-00077-of-00101.safetensors", + "model.layers.70.mlp.experts.68.up_proj.weight": "model-00077-of-00101.safetensors", + "model.layers.70.mlp.experts.68.down_proj.weight": "model-00077-of-00101.safetensors", + "model.layers.70.mlp.experts.69.gate_proj.weight": "model-00077-of-00101.safetensors", + "model.layers.70.mlp.experts.69.up_proj.weight": "model-00077-of-00101.safetensors", + "model.layers.70.mlp.experts.69.down_proj.weight": "model-00077-of-00101.safetensors", + "model.layers.70.mlp.experts.70.gate_proj.weight": "model-00077-of-00101.safetensors", + "model.layers.70.mlp.experts.70.up_proj.weight": "model-00077-of-00101.safetensors", + "model.layers.70.mlp.experts.70.down_proj.weight": "model-00077-of-00101.safetensors", + "model.layers.70.mlp.experts.71.gate_proj.weight": "model-00077-of-00101.safetensors", + "model.layers.70.mlp.experts.71.up_proj.weight": "model-00077-of-00101.safetensors", + "model.layers.70.mlp.experts.71.down_proj.weight": "model-00077-of-00101.safetensors", + "model.layers.70.mlp.experts.72.gate_proj.weight": "model-00077-of-00101.safetensors", + "model.layers.70.mlp.experts.72.up_proj.weight": "model-00077-of-00101.safetensors", + "model.layers.70.mlp.experts.72.down_proj.weight": "model-00077-of-00101.safetensors", + "model.layers.70.mlp.experts.73.gate_proj.weight": "model-00077-of-00101.safetensors", + "model.layers.70.mlp.experts.73.up_proj.weight": "model-00077-of-00101.safetensors", + "model.layers.70.mlp.experts.73.down_proj.weight": "model-00077-of-00101.safetensors", + "model.layers.70.mlp.experts.74.gate_proj.weight": "model-00077-of-00101.safetensors", + "model.layers.70.mlp.experts.74.up_proj.weight": "model-00077-of-00101.safetensors", + "model.layers.70.mlp.experts.74.down_proj.weight": "model-00077-of-00101.safetensors", + "model.layers.70.mlp.experts.75.gate_proj.weight": "model-00077-of-00101.safetensors", + "model.layers.70.mlp.experts.75.up_proj.weight": "model-00077-of-00101.safetensors", + "model.layers.70.mlp.experts.75.down_proj.weight": "model-00077-of-00101.safetensors", + "model.layers.70.mlp.experts.76.gate_proj.weight": "model-00077-of-00101.safetensors", + "model.layers.70.mlp.experts.76.up_proj.weight": "model-00077-of-00101.safetensors", + "model.layers.70.mlp.experts.76.down_proj.weight": "model-00077-of-00101.safetensors", + "model.layers.70.mlp.experts.77.gate_proj.weight": "model-00077-of-00101.safetensors", + "model.layers.70.mlp.experts.77.up_proj.weight": "model-00077-of-00101.safetensors", + "model.layers.70.mlp.experts.77.down_proj.weight": "model-00077-of-00101.safetensors", + "model.layers.70.mlp.experts.78.gate_proj.weight": "model-00077-of-00101.safetensors", + "model.layers.70.mlp.experts.78.up_proj.weight": "model-00077-of-00101.safetensors", + "model.layers.70.mlp.experts.78.down_proj.weight": "model-00077-of-00101.safetensors", + "model.layers.70.mlp.experts.79.gate_proj.weight": "model-00077-of-00101.safetensors", + "model.layers.70.mlp.experts.79.up_proj.weight": "model-00077-of-00101.safetensors", + "model.layers.70.mlp.experts.79.down_proj.weight": "model-00077-of-00101.safetensors", + "model.layers.70.mlp.experts.80.gate_proj.weight": "model-00077-of-00101.safetensors", + "model.layers.70.mlp.experts.80.up_proj.weight": "model-00077-of-00101.safetensors", + "model.layers.70.mlp.experts.80.down_proj.weight": "model-00077-of-00101.safetensors", + "model.layers.70.mlp.experts.81.gate_proj.weight": "model-00077-of-00101.safetensors", + "model.layers.70.mlp.experts.81.up_proj.weight": "model-00077-of-00101.safetensors", + "model.layers.70.mlp.experts.81.down_proj.weight": "model-00077-of-00101.safetensors", + "model.layers.70.mlp.experts.82.gate_proj.weight": "model-00077-of-00101.safetensors", + "model.layers.70.mlp.experts.82.up_proj.weight": "model-00077-of-00101.safetensors", + "model.layers.70.mlp.experts.82.down_proj.weight": "model-00077-of-00101.safetensors", + "model.layers.70.mlp.experts.83.gate_proj.weight": "model-00077-of-00101.safetensors", + "model.layers.70.mlp.experts.83.up_proj.weight": "model-00077-of-00101.safetensors", + "model.layers.70.mlp.experts.83.down_proj.weight": "model-00077-of-00101.safetensors", + "model.layers.70.mlp.experts.84.gate_proj.weight": "model-00077-of-00101.safetensors", + "model.layers.70.mlp.experts.84.up_proj.weight": "model-00077-of-00101.safetensors", + "model.layers.70.mlp.experts.84.down_proj.weight": "model-00077-of-00101.safetensors", + "model.layers.70.mlp.experts.85.gate_proj.weight": "model-00077-of-00101.safetensors", + "model.layers.70.mlp.experts.85.up_proj.weight": "model-00077-of-00101.safetensors", + "model.layers.70.mlp.experts.85.down_proj.weight": "model-00077-of-00101.safetensors", + "model.layers.70.mlp.experts.86.gate_proj.weight": "model-00077-of-00101.safetensors", + "model.layers.70.mlp.experts.86.up_proj.weight": "model-00077-of-00101.safetensors", + "model.layers.70.mlp.experts.86.down_proj.weight": "model-00077-of-00101.safetensors", + "model.layers.70.mlp.experts.87.gate_proj.weight": "model-00077-of-00101.safetensors", + "model.layers.70.mlp.experts.87.up_proj.weight": "model-00077-of-00101.safetensors", + "model.layers.70.mlp.experts.87.down_proj.weight": "model-00077-of-00101.safetensors", + "model.layers.70.mlp.experts.88.gate_proj.weight": "model-00077-of-00101.safetensors", + "model.layers.70.mlp.experts.88.up_proj.weight": "model-00077-of-00101.safetensors", + "model.layers.70.mlp.experts.88.down_proj.weight": "model-00077-of-00101.safetensors", + "model.layers.70.mlp.experts.89.gate_proj.weight": "model-00077-of-00101.safetensors", + "model.layers.70.mlp.experts.89.up_proj.weight": "model-00077-of-00101.safetensors", + "model.layers.70.mlp.experts.89.down_proj.weight": "model-00077-of-00101.safetensors", + "model.layers.70.mlp.experts.90.gate_proj.weight": "model-00077-of-00101.safetensors", + "model.layers.70.mlp.experts.90.up_proj.weight": "model-00077-of-00101.safetensors", + "model.layers.70.mlp.experts.90.down_proj.weight": "model-00077-of-00101.safetensors", + "model.layers.70.mlp.experts.91.gate_proj.weight": "model-00077-of-00101.safetensors", + "model.layers.70.mlp.experts.91.up_proj.weight": "model-00077-of-00101.safetensors", + "model.layers.70.mlp.experts.91.down_proj.weight": "model-00077-of-00101.safetensors", + "model.layers.70.mlp.experts.92.gate_proj.weight": "model-00077-of-00101.safetensors", + "model.layers.70.mlp.experts.92.up_proj.weight": "model-00077-of-00101.safetensors", + "model.layers.70.mlp.experts.92.down_proj.weight": "model-00077-of-00101.safetensors", + "model.layers.70.mlp.experts.93.gate_proj.weight": "model-00077-of-00101.safetensors", + "model.layers.70.mlp.experts.93.up_proj.weight": "model-00077-of-00101.safetensors", + "model.layers.70.mlp.experts.93.down_proj.weight": "model-00077-of-00101.safetensors", + "model.layers.70.mlp.experts.94.gate_proj.weight": "model-00077-of-00101.safetensors", + "model.layers.70.mlp.experts.94.up_proj.weight": "model-00077-of-00101.safetensors", + "model.layers.70.mlp.experts.94.down_proj.weight": "model-00077-of-00101.safetensors", + "model.layers.70.mlp.experts.95.gate_proj.weight": "model-00077-of-00101.safetensors", + "model.layers.70.mlp.experts.95.up_proj.weight": "model-00077-of-00101.safetensors", + "model.layers.70.mlp.experts.95.down_proj.weight": "model-00077-of-00101.safetensors", + "model.layers.70.mlp.experts.96.gate_proj.weight": "model-00077-of-00101.safetensors", + "model.layers.70.mlp.experts.96.up_proj.weight": "model-00077-of-00101.safetensors", + "model.layers.70.mlp.experts.96.down_proj.weight": "model-00077-of-00101.safetensors", + "model.layers.70.mlp.experts.97.gate_proj.weight": "model-00077-of-00101.safetensors", + "model.layers.70.mlp.experts.97.up_proj.weight": "model-00077-of-00101.safetensors", + "model.layers.70.mlp.experts.97.down_proj.weight": "model-00077-of-00101.safetensors", + "model.layers.70.mlp.experts.98.gate_proj.weight": "model-00077-of-00101.safetensors", + "model.layers.70.mlp.experts.98.up_proj.weight": "model-00077-of-00101.safetensors", + "model.layers.70.mlp.experts.98.down_proj.weight": "model-00077-of-00101.safetensors", + "model.layers.70.mlp.experts.99.gate_proj.weight": "model-00077-of-00101.safetensors", + "model.layers.70.mlp.experts.99.up_proj.weight": "model-00077-of-00101.safetensors", + "model.layers.70.mlp.experts.99.down_proj.weight": "model-00077-of-00101.safetensors", + "model.layers.70.mlp.experts.100.gate_proj.weight": "model-00077-of-00101.safetensors", + "model.layers.70.mlp.experts.100.up_proj.weight": "model-00077-of-00101.safetensors", + "model.layers.70.mlp.experts.100.down_proj.weight": "model-00077-of-00101.safetensors", + "model.layers.70.mlp.experts.101.gate_proj.weight": "model-00077-of-00101.safetensors", + "model.layers.70.mlp.experts.101.up_proj.weight": "model-00077-of-00101.safetensors", + "model.layers.70.mlp.experts.101.down_proj.weight": "model-00077-of-00101.safetensors", + "model.layers.70.mlp.experts.102.gate_proj.weight": "model-00077-of-00101.safetensors", + "model.layers.70.mlp.experts.102.up_proj.weight": "model-00077-of-00101.safetensors", + "model.layers.70.mlp.experts.102.down_proj.weight": "model-00077-of-00101.safetensors", + "model.layers.70.mlp.experts.103.gate_proj.weight": "model-00077-of-00101.safetensors", + "model.layers.70.mlp.experts.103.up_proj.weight": "model-00077-of-00101.safetensors", + "model.layers.70.mlp.experts.103.down_proj.weight": "model-00077-of-00101.safetensors", + "model.layers.70.mlp.experts.104.gate_proj.weight": "model-00077-of-00101.safetensors", + "model.layers.70.mlp.experts.104.up_proj.weight": "model-00077-of-00101.safetensors", + "model.layers.70.mlp.experts.104.down_proj.weight": "model-00077-of-00101.safetensors", + "model.layers.70.mlp.experts.105.gate_proj.weight": "model-00077-of-00101.safetensors", + "model.layers.70.mlp.experts.105.up_proj.weight": "model-00077-of-00101.safetensors", + "model.layers.70.mlp.experts.105.down_proj.weight": "model-00077-of-00101.safetensors", + "model.layers.70.mlp.experts.106.gate_proj.weight": "model-00077-of-00101.safetensors", + "model.layers.70.mlp.experts.106.up_proj.weight": "model-00077-of-00101.safetensors", + "model.layers.70.mlp.experts.106.down_proj.weight": "model-00077-of-00101.safetensors", + "model.layers.70.mlp.experts.107.gate_proj.weight": "model-00077-of-00101.safetensors", + "model.layers.70.mlp.experts.107.up_proj.weight": "model-00077-of-00101.safetensors", + "model.layers.70.mlp.experts.107.down_proj.weight": "model-00077-of-00101.safetensors", + "model.layers.70.mlp.experts.108.gate_proj.weight": "model-00077-of-00101.safetensors", + "model.layers.70.mlp.experts.108.up_proj.weight": "model-00077-of-00101.safetensors", + "model.layers.70.mlp.experts.108.down_proj.weight": "model-00077-of-00101.safetensors", + "model.layers.70.mlp.experts.109.gate_proj.weight": "model-00077-of-00101.safetensors", + "model.layers.70.mlp.experts.109.up_proj.weight": "model-00077-of-00101.safetensors", + "model.layers.70.mlp.experts.109.down_proj.weight": "model-00077-of-00101.safetensors", + "model.layers.70.mlp.experts.110.gate_proj.weight": "model-00077-of-00101.safetensors", + "model.layers.70.mlp.experts.110.up_proj.weight": "model-00077-of-00101.safetensors", + "model.layers.70.mlp.experts.110.down_proj.weight": "model-00077-of-00101.safetensors", + "model.layers.70.mlp.experts.111.gate_proj.weight": "model-00077-of-00101.safetensors", + "model.layers.70.mlp.experts.111.up_proj.weight": "model-00077-of-00101.safetensors", + "model.layers.70.mlp.experts.111.down_proj.weight": "model-00077-of-00101.safetensors", + "model.layers.70.mlp.experts.112.gate_proj.weight": "model-00077-of-00101.safetensors", + "model.layers.70.mlp.experts.112.up_proj.weight": "model-00077-of-00101.safetensors", + "model.layers.70.mlp.experts.112.down_proj.weight": "model-00077-of-00101.safetensors", + "model.layers.70.mlp.experts.113.gate_proj.weight": "model-00077-of-00101.safetensors", + "model.layers.70.mlp.experts.113.up_proj.weight": "model-00077-of-00101.safetensors", + "model.layers.70.mlp.experts.113.down_proj.weight": "model-00077-of-00101.safetensors", + "model.layers.70.mlp.experts.114.gate_proj.weight": "model-00077-of-00101.safetensors", + "model.layers.70.mlp.experts.114.up_proj.weight": "model-00077-of-00101.safetensors", + "model.layers.70.mlp.experts.114.down_proj.weight": "model-00077-of-00101.safetensors", + "model.layers.70.mlp.experts.115.gate_proj.weight": "model-00077-of-00101.safetensors", + "model.layers.70.mlp.experts.115.up_proj.weight": "model-00077-of-00101.safetensors", + "model.layers.70.mlp.experts.115.down_proj.weight": "model-00077-of-00101.safetensors", + "model.layers.70.mlp.experts.116.gate_proj.weight": "model-00077-of-00101.safetensors", + "model.layers.70.mlp.experts.116.up_proj.weight": "model-00077-of-00101.safetensors", + "model.layers.70.mlp.experts.116.down_proj.weight": "model-00077-of-00101.safetensors", + "model.layers.70.mlp.experts.117.gate_proj.weight": "model-00077-of-00101.safetensors", + "model.layers.70.mlp.experts.117.up_proj.weight": "model-00077-of-00101.safetensors", + "model.layers.70.mlp.experts.117.down_proj.weight": "model-00077-of-00101.safetensors", + "model.layers.70.mlp.experts.118.gate_proj.weight": "model-00077-of-00101.safetensors", + "model.layers.70.mlp.experts.118.up_proj.weight": "model-00077-of-00101.safetensors", + "model.layers.70.mlp.experts.118.down_proj.weight": "model-00077-of-00101.safetensors", + "model.layers.70.mlp.experts.119.gate_proj.weight": "model-00077-of-00101.safetensors", + "model.layers.70.mlp.experts.119.up_proj.weight": "model-00077-of-00101.safetensors", + "model.layers.70.mlp.experts.119.down_proj.weight": "model-00077-of-00101.safetensors", + "model.layers.70.mlp.gate.weight": "model-00077-of-00101.safetensors", + "model.layers.70.mlp.gate.e_score_correction_bias": "model-00077-of-00101.safetensors", + "model.layers.70.mlp.shared_experts.gate_proj.weight": "model-00077-of-00101.safetensors", + "model.layers.70.mlp.shared_experts.up_proj.weight": "model-00077-of-00101.safetensors", + "model.layers.70.mlp.shared_experts.down_proj.weight": "model-00077-of-00101.safetensors", + "model.layers.70.input_layernorm.weight": "model-00077-of-00101.safetensors", + "model.layers.70.post_attention_layernorm.weight": "model-00077-of-00101.safetensors", + "model.layers.71.self_attn.q_proj.weight": "model-00077-of-00101.safetensors", + "model.layers.71.self_attn.q_proj.bias": "model-00077-of-00101.safetensors", + "model.layers.71.self_attn.k_proj.weight": "model-00077-of-00101.safetensors", + "model.layers.71.self_attn.k_proj.bias": "model-00077-of-00101.safetensors", + "model.layers.71.self_attn.v_proj.weight": "model-00077-of-00101.safetensors", + "model.layers.71.self_attn.v_proj.bias": "model-00077-of-00101.safetensors", + "model.layers.71.self_attn.o_proj.weight": "model-00077-of-00101.safetensors", + "model.layers.71.self_attn.q_norm.weight": "model-00077-of-00101.safetensors", + "model.layers.71.self_attn.k_norm.weight": "model-00077-of-00101.safetensors", + "model.layers.71.mlp.experts.0.gate_proj.weight": "model-00077-of-00101.safetensors", + "model.layers.71.mlp.experts.0.up_proj.weight": "model-00077-of-00101.safetensors", + "model.layers.71.mlp.experts.0.down_proj.weight": "model-00077-of-00101.safetensors", + "model.layers.71.mlp.experts.1.gate_proj.weight": "model-00077-of-00101.safetensors", + "model.layers.71.mlp.experts.1.up_proj.weight": "model-00077-of-00101.safetensors", + "model.layers.71.mlp.experts.1.down_proj.weight": "model-00077-of-00101.safetensors", + "model.layers.71.mlp.experts.2.gate_proj.weight": "model-00077-of-00101.safetensors", + "model.layers.71.mlp.experts.2.up_proj.weight": "model-00077-of-00101.safetensors", + "model.layers.71.mlp.experts.2.down_proj.weight": "model-00077-of-00101.safetensors", + "model.layers.71.mlp.experts.3.gate_proj.weight": "model-00077-of-00101.safetensors", + "model.layers.71.mlp.experts.3.up_proj.weight": "model-00077-of-00101.safetensors", + "model.layers.71.mlp.experts.3.down_proj.weight": "model-00077-of-00101.safetensors", + "model.layers.71.mlp.experts.4.gate_proj.weight": "model-00077-of-00101.safetensors", + "model.layers.71.mlp.experts.4.up_proj.weight": "model-00077-of-00101.safetensors", + "model.layers.71.mlp.experts.4.down_proj.weight": "model-00077-of-00101.safetensors", + "model.layers.71.mlp.experts.5.gate_proj.weight": "model-00077-of-00101.safetensors", + "model.layers.71.mlp.experts.5.up_proj.weight": "model-00077-of-00101.safetensors", + "model.layers.71.mlp.experts.5.down_proj.weight": "model-00077-of-00101.safetensors", + "model.layers.71.mlp.experts.6.gate_proj.weight": "model-00077-of-00101.safetensors", + "model.layers.71.mlp.experts.6.up_proj.weight": "model-00077-of-00101.safetensors", + "model.layers.71.mlp.experts.6.down_proj.weight": "model-00077-of-00101.safetensors", + "model.layers.71.mlp.experts.7.gate_proj.weight": "model-00077-of-00101.safetensors", + "model.layers.71.mlp.experts.7.up_proj.weight": "model-00077-of-00101.safetensors", + "model.layers.71.mlp.experts.7.down_proj.weight": "model-00077-of-00101.safetensors", + "model.layers.71.mlp.experts.8.gate_proj.weight": "model-00077-of-00101.safetensors", + "model.layers.71.mlp.experts.8.up_proj.weight": "model-00077-of-00101.safetensors", + "model.layers.71.mlp.experts.8.down_proj.weight": "model-00077-of-00101.safetensors", + "model.layers.71.mlp.experts.9.gate_proj.weight": "model-00077-of-00101.safetensors", + "model.layers.71.mlp.experts.9.up_proj.weight": "model-00077-of-00101.safetensors", + "model.layers.71.mlp.experts.9.down_proj.weight": "model-00077-of-00101.safetensors", + "model.layers.71.mlp.experts.10.gate_proj.weight": "model-00077-of-00101.safetensors", + "model.layers.71.mlp.experts.10.up_proj.weight": "model-00077-of-00101.safetensors", + "model.layers.71.mlp.experts.10.down_proj.weight": "model-00077-of-00101.safetensors", + "model.layers.71.mlp.experts.11.gate_proj.weight": "model-00077-of-00101.safetensors", + "model.layers.71.mlp.experts.11.up_proj.weight": "model-00077-of-00101.safetensors", + "model.layers.71.mlp.experts.11.down_proj.weight": "model-00077-of-00101.safetensors", + "model.layers.71.mlp.experts.12.gate_proj.weight": "model-00077-of-00101.safetensors", + "model.layers.71.mlp.experts.12.up_proj.weight": "model-00077-of-00101.safetensors", + "model.layers.71.mlp.experts.12.down_proj.weight": "model-00077-of-00101.safetensors", + "model.layers.71.mlp.experts.13.gate_proj.weight": "model-00077-of-00101.safetensors", + "model.layers.71.mlp.experts.13.up_proj.weight": "model-00077-of-00101.safetensors", + "model.layers.71.mlp.experts.13.down_proj.weight": "model-00077-of-00101.safetensors", + "model.layers.71.mlp.experts.14.gate_proj.weight": "model-00077-of-00101.safetensors", + "model.layers.71.mlp.experts.14.up_proj.weight": "model-00077-of-00101.safetensors", + "model.layers.71.mlp.experts.14.down_proj.weight": "model-00077-of-00101.safetensors", + "model.layers.71.mlp.experts.15.gate_proj.weight": "model-00077-of-00101.safetensors", + "model.layers.71.mlp.experts.15.up_proj.weight": "model-00077-of-00101.safetensors", + "model.layers.71.mlp.experts.15.down_proj.weight": "model-00077-of-00101.safetensors", + "model.layers.71.mlp.experts.16.gate_proj.weight": "model-00077-of-00101.safetensors", + "model.layers.71.mlp.experts.16.up_proj.weight": "model-00077-of-00101.safetensors", + "model.layers.71.mlp.experts.16.down_proj.weight": "model-00077-of-00101.safetensors", + "model.layers.71.mlp.experts.17.gate_proj.weight": "model-00077-of-00101.safetensors", + "model.layers.71.mlp.experts.17.up_proj.weight": "model-00077-of-00101.safetensors", + "model.layers.71.mlp.experts.17.down_proj.weight": "model-00077-of-00101.safetensors", + "model.layers.71.mlp.experts.18.gate_proj.weight": "model-00077-of-00101.safetensors", + "model.layers.71.mlp.experts.18.up_proj.weight": "model-00077-of-00101.safetensors", + "model.layers.71.mlp.experts.18.down_proj.weight": "model-00077-of-00101.safetensors", + "model.layers.71.mlp.experts.19.gate_proj.weight": "model-00077-of-00101.safetensors", + "model.layers.71.mlp.experts.19.up_proj.weight": "model-00077-of-00101.safetensors", + "model.layers.71.mlp.experts.19.down_proj.weight": "model-00077-of-00101.safetensors", + "model.layers.71.mlp.experts.20.gate_proj.weight": "model-00077-of-00101.safetensors", + "model.layers.71.mlp.experts.20.up_proj.weight": "model-00077-of-00101.safetensors", + "model.layers.71.mlp.experts.20.down_proj.weight": "model-00077-of-00101.safetensors", + "model.layers.71.mlp.experts.21.gate_proj.weight": "model-00077-of-00101.safetensors", + "model.layers.71.mlp.experts.21.up_proj.weight": "model-00077-of-00101.safetensors", + "model.layers.71.mlp.experts.21.down_proj.weight": "model-00077-of-00101.safetensors", + "model.layers.71.mlp.experts.22.gate_proj.weight": "model-00077-of-00101.safetensors", + "model.layers.71.mlp.experts.22.up_proj.weight": "model-00077-of-00101.safetensors", + "model.layers.71.mlp.experts.22.down_proj.weight": "model-00077-of-00101.safetensors", + "model.layers.71.mlp.experts.23.gate_proj.weight": "model-00077-of-00101.safetensors", + "model.layers.71.mlp.experts.23.up_proj.weight": "model-00077-of-00101.safetensors", + "model.layers.71.mlp.experts.23.down_proj.weight": "model-00077-of-00101.safetensors", + "model.layers.71.mlp.experts.24.gate_proj.weight": "model-00077-of-00101.safetensors", + "model.layers.71.mlp.experts.24.up_proj.weight": "model-00077-of-00101.safetensors", + "model.layers.71.mlp.experts.24.down_proj.weight": "model-00077-of-00101.safetensors", + "model.layers.71.mlp.experts.25.gate_proj.weight": "model-00077-of-00101.safetensors", + "model.layers.71.mlp.experts.25.up_proj.weight": "model-00077-of-00101.safetensors", + "model.layers.71.mlp.experts.25.down_proj.weight": "model-00077-of-00101.safetensors", + "model.layers.71.mlp.experts.26.gate_proj.weight": "model-00077-of-00101.safetensors", + "model.layers.71.mlp.experts.26.up_proj.weight": "model-00077-of-00101.safetensors", + "model.layers.71.mlp.experts.26.down_proj.weight": "model-00077-of-00101.safetensors", + "model.layers.71.mlp.experts.27.gate_proj.weight": "model-00077-of-00101.safetensors", + "model.layers.71.mlp.experts.27.up_proj.weight": "model-00077-of-00101.safetensors", + "model.layers.71.mlp.experts.27.down_proj.weight": "model-00077-of-00101.safetensors", + "model.layers.71.mlp.experts.28.gate_proj.weight": "model-00077-of-00101.safetensors", + "model.layers.71.mlp.experts.28.up_proj.weight": "model-00077-of-00101.safetensors", + "model.layers.71.mlp.experts.28.down_proj.weight": "model-00077-of-00101.safetensors", + "model.layers.71.mlp.experts.29.gate_proj.weight": "model-00077-of-00101.safetensors", + "model.layers.71.mlp.experts.29.up_proj.weight": "model-00077-of-00101.safetensors", + "model.layers.71.mlp.experts.29.down_proj.weight": "model-00077-of-00101.safetensors", + "model.layers.71.mlp.experts.30.gate_proj.weight": "model-00077-of-00101.safetensors", + "model.layers.71.mlp.experts.30.up_proj.weight": "model-00077-of-00101.safetensors", + "model.layers.71.mlp.experts.30.down_proj.weight": "model-00077-of-00101.safetensors", + "model.layers.71.mlp.experts.31.gate_proj.weight": "model-00077-of-00101.safetensors", + "model.layers.71.mlp.experts.31.up_proj.weight": "model-00077-of-00101.safetensors", + "model.layers.71.mlp.experts.31.down_proj.weight": "model-00077-of-00101.safetensors", + "model.layers.71.mlp.experts.32.gate_proj.weight": "model-00077-of-00101.safetensors", + "model.layers.71.mlp.experts.32.up_proj.weight": "model-00077-of-00101.safetensors", + "model.layers.71.mlp.experts.32.down_proj.weight": "model-00077-of-00101.safetensors", + "model.layers.71.mlp.experts.33.gate_proj.weight": "model-00077-of-00101.safetensors", + "model.layers.71.mlp.experts.33.up_proj.weight": "model-00077-of-00101.safetensors", + "model.layers.71.mlp.experts.33.down_proj.weight": "model-00077-of-00101.safetensors", + "model.layers.71.mlp.experts.34.gate_proj.weight": "model-00077-of-00101.safetensors", + "model.layers.71.mlp.experts.34.up_proj.weight": "model-00077-of-00101.safetensors", + "model.layers.71.mlp.experts.34.down_proj.weight": "model-00078-of-00101.safetensors", + "model.layers.71.mlp.experts.35.gate_proj.weight": "model-00078-of-00101.safetensors", + "model.layers.71.mlp.experts.35.up_proj.weight": "model-00078-of-00101.safetensors", + "model.layers.71.mlp.experts.35.down_proj.weight": "model-00078-of-00101.safetensors", + "model.layers.71.mlp.experts.36.gate_proj.weight": "model-00078-of-00101.safetensors", + "model.layers.71.mlp.experts.36.up_proj.weight": "model-00078-of-00101.safetensors", + "model.layers.71.mlp.experts.36.down_proj.weight": "model-00078-of-00101.safetensors", + "model.layers.71.mlp.experts.37.gate_proj.weight": "model-00078-of-00101.safetensors", + "model.layers.71.mlp.experts.37.up_proj.weight": "model-00078-of-00101.safetensors", + "model.layers.71.mlp.experts.37.down_proj.weight": "model-00078-of-00101.safetensors", + "model.layers.71.mlp.experts.38.gate_proj.weight": "model-00078-of-00101.safetensors", + "model.layers.71.mlp.experts.38.up_proj.weight": "model-00078-of-00101.safetensors", + "model.layers.71.mlp.experts.38.down_proj.weight": "model-00078-of-00101.safetensors", + "model.layers.71.mlp.experts.39.gate_proj.weight": "model-00078-of-00101.safetensors", + "model.layers.71.mlp.experts.39.up_proj.weight": "model-00078-of-00101.safetensors", + "model.layers.71.mlp.experts.39.down_proj.weight": "model-00078-of-00101.safetensors", + "model.layers.71.mlp.experts.40.gate_proj.weight": "model-00078-of-00101.safetensors", + "model.layers.71.mlp.experts.40.up_proj.weight": "model-00078-of-00101.safetensors", + "model.layers.71.mlp.experts.40.down_proj.weight": "model-00078-of-00101.safetensors", + "model.layers.71.mlp.experts.41.gate_proj.weight": "model-00078-of-00101.safetensors", + "model.layers.71.mlp.experts.41.up_proj.weight": "model-00078-of-00101.safetensors", + "model.layers.71.mlp.experts.41.down_proj.weight": "model-00078-of-00101.safetensors", + "model.layers.71.mlp.experts.42.gate_proj.weight": "model-00078-of-00101.safetensors", + "model.layers.71.mlp.experts.42.up_proj.weight": "model-00078-of-00101.safetensors", + "model.layers.71.mlp.experts.42.down_proj.weight": "model-00078-of-00101.safetensors", + "model.layers.71.mlp.experts.43.gate_proj.weight": "model-00078-of-00101.safetensors", + "model.layers.71.mlp.experts.43.up_proj.weight": "model-00078-of-00101.safetensors", + "model.layers.71.mlp.experts.43.down_proj.weight": "model-00078-of-00101.safetensors", + "model.layers.71.mlp.experts.44.gate_proj.weight": "model-00078-of-00101.safetensors", + "model.layers.71.mlp.experts.44.up_proj.weight": "model-00078-of-00101.safetensors", + "model.layers.71.mlp.experts.44.down_proj.weight": "model-00078-of-00101.safetensors", + "model.layers.71.mlp.experts.45.gate_proj.weight": "model-00078-of-00101.safetensors", + "model.layers.71.mlp.experts.45.up_proj.weight": "model-00078-of-00101.safetensors", + "model.layers.71.mlp.experts.45.down_proj.weight": "model-00078-of-00101.safetensors", + "model.layers.71.mlp.experts.46.gate_proj.weight": "model-00078-of-00101.safetensors", + "model.layers.71.mlp.experts.46.up_proj.weight": "model-00078-of-00101.safetensors", + "model.layers.71.mlp.experts.46.down_proj.weight": "model-00078-of-00101.safetensors", + "model.layers.71.mlp.experts.47.gate_proj.weight": "model-00078-of-00101.safetensors", + "model.layers.71.mlp.experts.47.up_proj.weight": "model-00078-of-00101.safetensors", + "model.layers.71.mlp.experts.47.down_proj.weight": "model-00078-of-00101.safetensors", + "model.layers.71.mlp.experts.48.gate_proj.weight": "model-00078-of-00101.safetensors", + "model.layers.71.mlp.experts.48.up_proj.weight": "model-00078-of-00101.safetensors", + "model.layers.71.mlp.experts.48.down_proj.weight": "model-00078-of-00101.safetensors", + "model.layers.71.mlp.experts.49.gate_proj.weight": "model-00078-of-00101.safetensors", + "model.layers.71.mlp.experts.49.up_proj.weight": "model-00078-of-00101.safetensors", + "model.layers.71.mlp.experts.49.down_proj.weight": "model-00078-of-00101.safetensors", + "model.layers.71.mlp.experts.50.gate_proj.weight": "model-00078-of-00101.safetensors", + "model.layers.71.mlp.experts.50.up_proj.weight": "model-00078-of-00101.safetensors", + "model.layers.71.mlp.experts.50.down_proj.weight": "model-00078-of-00101.safetensors", + "model.layers.71.mlp.experts.51.gate_proj.weight": "model-00078-of-00101.safetensors", + "model.layers.71.mlp.experts.51.up_proj.weight": "model-00078-of-00101.safetensors", + "model.layers.71.mlp.experts.51.down_proj.weight": "model-00078-of-00101.safetensors", + "model.layers.71.mlp.experts.52.gate_proj.weight": "model-00078-of-00101.safetensors", + "model.layers.71.mlp.experts.52.up_proj.weight": "model-00078-of-00101.safetensors", + "model.layers.71.mlp.experts.52.down_proj.weight": "model-00078-of-00101.safetensors", + "model.layers.71.mlp.experts.53.gate_proj.weight": "model-00078-of-00101.safetensors", + "model.layers.71.mlp.experts.53.up_proj.weight": "model-00078-of-00101.safetensors", + "model.layers.71.mlp.experts.53.down_proj.weight": "model-00078-of-00101.safetensors", + "model.layers.71.mlp.experts.54.gate_proj.weight": "model-00078-of-00101.safetensors", + "model.layers.71.mlp.experts.54.up_proj.weight": "model-00078-of-00101.safetensors", + "model.layers.71.mlp.experts.54.down_proj.weight": "model-00078-of-00101.safetensors", + "model.layers.71.mlp.experts.55.gate_proj.weight": "model-00078-of-00101.safetensors", + "model.layers.71.mlp.experts.55.up_proj.weight": "model-00078-of-00101.safetensors", + "model.layers.71.mlp.experts.55.down_proj.weight": "model-00078-of-00101.safetensors", + "model.layers.71.mlp.experts.56.gate_proj.weight": "model-00078-of-00101.safetensors", + "model.layers.71.mlp.experts.56.up_proj.weight": "model-00078-of-00101.safetensors", + "model.layers.71.mlp.experts.56.down_proj.weight": "model-00078-of-00101.safetensors", + "model.layers.71.mlp.experts.57.gate_proj.weight": "model-00078-of-00101.safetensors", + "model.layers.71.mlp.experts.57.up_proj.weight": "model-00078-of-00101.safetensors", + "model.layers.71.mlp.experts.57.down_proj.weight": "model-00078-of-00101.safetensors", + "model.layers.71.mlp.experts.58.gate_proj.weight": "model-00078-of-00101.safetensors", + "model.layers.71.mlp.experts.58.up_proj.weight": "model-00078-of-00101.safetensors", + "model.layers.71.mlp.experts.58.down_proj.weight": "model-00078-of-00101.safetensors", + "model.layers.71.mlp.experts.59.gate_proj.weight": "model-00078-of-00101.safetensors", + "model.layers.71.mlp.experts.59.up_proj.weight": "model-00078-of-00101.safetensors", + "model.layers.71.mlp.experts.59.down_proj.weight": "model-00078-of-00101.safetensors", + "model.layers.71.mlp.experts.60.gate_proj.weight": "model-00078-of-00101.safetensors", + "model.layers.71.mlp.experts.60.up_proj.weight": "model-00078-of-00101.safetensors", + "model.layers.71.mlp.experts.60.down_proj.weight": "model-00078-of-00101.safetensors", + "model.layers.71.mlp.experts.61.gate_proj.weight": "model-00078-of-00101.safetensors", + "model.layers.71.mlp.experts.61.up_proj.weight": "model-00078-of-00101.safetensors", + "model.layers.71.mlp.experts.61.down_proj.weight": "model-00078-of-00101.safetensors", + "model.layers.71.mlp.experts.62.gate_proj.weight": "model-00078-of-00101.safetensors", + "model.layers.71.mlp.experts.62.up_proj.weight": "model-00078-of-00101.safetensors", + "model.layers.71.mlp.experts.62.down_proj.weight": "model-00078-of-00101.safetensors", + "model.layers.71.mlp.experts.63.gate_proj.weight": "model-00078-of-00101.safetensors", + "model.layers.71.mlp.experts.63.up_proj.weight": "model-00078-of-00101.safetensors", + "model.layers.71.mlp.experts.63.down_proj.weight": "model-00078-of-00101.safetensors", + "model.layers.71.mlp.experts.64.gate_proj.weight": "model-00078-of-00101.safetensors", + "model.layers.71.mlp.experts.64.up_proj.weight": "model-00078-of-00101.safetensors", + "model.layers.71.mlp.experts.64.down_proj.weight": "model-00078-of-00101.safetensors", + "model.layers.71.mlp.experts.65.gate_proj.weight": "model-00078-of-00101.safetensors", + "model.layers.71.mlp.experts.65.up_proj.weight": "model-00078-of-00101.safetensors", + "model.layers.71.mlp.experts.65.down_proj.weight": "model-00078-of-00101.safetensors", + "model.layers.71.mlp.experts.66.gate_proj.weight": "model-00078-of-00101.safetensors", + "model.layers.71.mlp.experts.66.up_proj.weight": "model-00078-of-00101.safetensors", + "model.layers.71.mlp.experts.66.down_proj.weight": "model-00078-of-00101.safetensors", + "model.layers.71.mlp.experts.67.gate_proj.weight": "model-00078-of-00101.safetensors", + "model.layers.71.mlp.experts.67.up_proj.weight": "model-00078-of-00101.safetensors", + "model.layers.71.mlp.experts.67.down_proj.weight": "model-00078-of-00101.safetensors", + "model.layers.71.mlp.experts.68.gate_proj.weight": "model-00078-of-00101.safetensors", + "model.layers.71.mlp.experts.68.up_proj.weight": "model-00078-of-00101.safetensors", + "model.layers.71.mlp.experts.68.down_proj.weight": "model-00078-of-00101.safetensors", + "model.layers.71.mlp.experts.69.gate_proj.weight": "model-00078-of-00101.safetensors", + "model.layers.71.mlp.experts.69.up_proj.weight": "model-00078-of-00101.safetensors", + "model.layers.71.mlp.experts.69.down_proj.weight": "model-00078-of-00101.safetensors", + "model.layers.71.mlp.experts.70.gate_proj.weight": "model-00078-of-00101.safetensors", + "model.layers.71.mlp.experts.70.up_proj.weight": "model-00078-of-00101.safetensors", + "model.layers.71.mlp.experts.70.down_proj.weight": "model-00078-of-00101.safetensors", + "model.layers.71.mlp.experts.71.gate_proj.weight": "model-00078-of-00101.safetensors", + "model.layers.71.mlp.experts.71.up_proj.weight": "model-00078-of-00101.safetensors", + "model.layers.71.mlp.experts.71.down_proj.weight": "model-00078-of-00101.safetensors", + "model.layers.71.mlp.experts.72.gate_proj.weight": "model-00078-of-00101.safetensors", + "model.layers.71.mlp.experts.72.up_proj.weight": "model-00078-of-00101.safetensors", + "model.layers.71.mlp.experts.72.down_proj.weight": "model-00078-of-00101.safetensors", + "model.layers.71.mlp.experts.73.gate_proj.weight": "model-00078-of-00101.safetensors", + "model.layers.71.mlp.experts.73.up_proj.weight": "model-00078-of-00101.safetensors", + "model.layers.71.mlp.experts.73.down_proj.weight": "model-00078-of-00101.safetensors", + "model.layers.71.mlp.experts.74.gate_proj.weight": "model-00078-of-00101.safetensors", + "model.layers.71.mlp.experts.74.up_proj.weight": "model-00078-of-00101.safetensors", + "model.layers.71.mlp.experts.74.down_proj.weight": "model-00078-of-00101.safetensors", + "model.layers.71.mlp.experts.75.gate_proj.weight": "model-00078-of-00101.safetensors", + "model.layers.71.mlp.experts.75.up_proj.weight": "model-00078-of-00101.safetensors", + "model.layers.71.mlp.experts.75.down_proj.weight": "model-00078-of-00101.safetensors", + "model.layers.71.mlp.experts.76.gate_proj.weight": "model-00078-of-00101.safetensors", + "model.layers.71.mlp.experts.76.up_proj.weight": "model-00078-of-00101.safetensors", + "model.layers.71.mlp.experts.76.down_proj.weight": "model-00078-of-00101.safetensors", + "model.layers.71.mlp.experts.77.gate_proj.weight": "model-00078-of-00101.safetensors", + "model.layers.71.mlp.experts.77.up_proj.weight": "model-00078-of-00101.safetensors", + "model.layers.71.mlp.experts.77.down_proj.weight": "model-00078-of-00101.safetensors", + "model.layers.71.mlp.experts.78.gate_proj.weight": "model-00078-of-00101.safetensors", + "model.layers.71.mlp.experts.78.up_proj.weight": "model-00078-of-00101.safetensors", + "model.layers.71.mlp.experts.78.down_proj.weight": "model-00078-of-00101.safetensors", + "model.layers.71.mlp.experts.79.gate_proj.weight": "model-00078-of-00101.safetensors", + "model.layers.71.mlp.experts.79.up_proj.weight": "model-00078-of-00101.safetensors", + "model.layers.71.mlp.experts.79.down_proj.weight": "model-00078-of-00101.safetensors", + "model.layers.71.mlp.experts.80.gate_proj.weight": "model-00078-of-00101.safetensors", + "model.layers.71.mlp.experts.80.up_proj.weight": "model-00078-of-00101.safetensors", + "model.layers.71.mlp.experts.80.down_proj.weight": "model-00078-of-00101.safetensors", + "model.layers.71.mlp.experts.81.gate_proj.weight": "model-00078-of-00101.safetensors", + "model.layers.71.mlp.experts.81.up_proj.weight": "model-00078-of-00101.safetensors", + "model.layers.71.mlp.experts.81.down_proj.weight": "model-00078-of-00101.safetensors", + "model.layers.71.mlp.experts.82.gate_proj.weight": "model-00078-of-00101.safetensors", + "model.layers.71.mlp.experts.82.up_proj.weight": "model-00078-of-00101.safetensors", + "model.layers.71.mlp.experts.82.down_proj.weight": "model-00078-of-00101.safetensors", + "model.layers.71.mlp.experts.83.gate_proj.weight": "model-00078-of-00101.safetensors", + "model.layers.71.mlp.experts.83.up_proj.weight": "model-00078-of-00101.safetensors", + "model.layers.71.mlp.experts.83.down_proj.weight": "model-00078-of-00101.safetensors", + "model.layers.71.mlp.experts.84.gate_proj.weight": "model-00078-of-00101.safetensors", + "model.layers.71.mlp.experts.84.up_proj.weight": "model-00078-of-00101.safetensors", + "model.layers.71.mlp.experts.84.down_proj.weight": "model-00078-of-00101.safetensors", + "model.layers.71.mlp.experts.85.gate_proj.weight": "model-00078-of-00101.safetensors", + "model.layers.71.mlp.experts.85.up_proj.weight": "model-00078-of-00101.safetensors", + "model.layers.71.mlp.experts.85.down_proj.weight": "model-00078-of-00101.safetensors", + "model.layers.71.mlp.experts.86.gate_proj.weight": "model-00078-of-00101.safetensors", + "model.layers.71.mlp.experts.86.up_proj.weight": "model-00078-of-00101.safetensors", + "model.layers.71.mlp.experts.86.down_proj.weight": "model-00078-of-00101.safetensors", + "model.layers.71.mlp.experts.87.gate_proj.weight": "model-00078-of-00101.safetensors", + "model.layers.71.mlp.experts.87.up_proj.weight": "model-00078-of-00101.safetensors", + "model.layers.71.mlp.experts.87.down_proj.weight": "model-00078-of-00101.safetensors", + "model.layers.71.mlp.experts.88.gate_proj.weight": "model-00078-of-00101.safetensors", + "model.layers.71.mlp.experts.88.up_proj.weight": "model-00078-of-00101.safetensors", + "model.layers.71.mlp.experts.88.down_proj.weight": "model-00078-of-00101.safetensors", + "model.layers.71.mlp.experts.89.gate_proj.weight": "model-00078-of-00101.safetensors", + "model.layers.71.mlp.experts.89.up_proj.weight": "model-00078-of-00101.safetensors", + "model.layers.71.mlp.experts.89.down_proj.weight": "model-00078-of-00101.safetensors", + "model.layers.71.mlp.experts.90.gate_proj.weight": "model-00078-of-00101.safetensors", + "model.layers.71.mlp.experts.90.up_proj.weight": "model-00078-of-00101.safetensors", + "model.layers.71.mlp.experts.90.down_proj.weight": "model-00078-of-00101.safetensors", + "model.layers.71.mlp.experts.91.gate_proj.weight": "model-00078-of-00101.safetensors", + "model.layers.71.mlp.experts.91.up_proj.weight": "model-00078-of-00101.safetensors", + "model.layers.71.mlp.experts.91.down_proj.weight": "model-00078-of-00101.safetensors", + "model.layers.71.mlp.experts.92.gate_proj.weight": "model-00078-of-00101.safetensors", + "model.layers.71.mlp.experts.92.up_proj.weight": "model-00078-of-00101.safetensors", + "model.layers.71.mlp.experts.92.down_proj.weight": "model-00078-of-00101.safetensors", + "model.layers.71.mlp.experts.93.gate_proj.weight": "model-00078-of-00101.safetensors", + "model.layers.71.mlp.experts.93.up_proj.weight": "model-00078-of-00101.safetensors", + "model.layers.71.mlp.experts.93.down_proj.weight": "model-00078-of-00101.safetensors", + "model.layers.71.mlp.experts.94.gate_proj.weight": "model-00078-of-00101.safetensors", + "model.layers.71.mlp.experts.94.up_proj.weight": "model-00078-of-00101.safetensors", + "model.layers.71.mlp.experts.94.down_proj.weight": "model-00078-of-00101.safetensors", + "model.layers.71.mlp.experts.95.gate_proj.weight": "model-00078-of-00101.safetensors", + "model.layers.71.mlp.experts.95.up_proj.weight": "model-00078-of-00101.safetensors", + "model.layers.71.mlp.experts.95.down_proj.weight": "model-00078-of-00101.safetensors", + "model.layers.71.mlp.experts.96.gate_proj.weight": "model-00078-of-00101.safetensors", + "model.layers.71.mlp.experts.96.up_proj.weight": "model-00078-of-00101.safetensors", + "model.layers.71.mlp.experts.96.down_proj.weight": "model-00078-of-00101.safetensors", + "model.layers.71.mlp.experts.97.gate_proj.weight": "model-00078-of-00101.safetensors", + "model.layers.71.mlp.experts.97.up_proj.weight": "model-00078-of-00101.safetensors", + "model.layers.71.mlp.experts.97.down_proj.weight": "model-00078-of-00101.safetensors", + "model.layers.71.mlp.experts.98.gate_proj.weight": "model-00078-of-00101.safetensors", + "model.layers.71.mlp.experts.98.up_proj.weight": "model-00078-of-00101.safetensors", + "model.layers.71.mlp.experts.98.down_proj.weight": "model-00078-of-00101.safetensors", + "model.layers.71.mlp.experts.99.gate_proj.weight": "model-00078-of-00101.safetensors", + "model.layers.71.mlp.experts.99.up_proj.weight": "model-00078-of-00101.safetensors", + "model.layers.71.mlp.experts.99.down_proj.weight": "model-00078-of-00101.safetensors", + "model.layers.71.mlp.experts.100.gate_proj.weight": "model-00078-of-00101.safetensors", + "model.layers.71.mlp.experts.100.up_proj.weight": "model-00078-of-00101.safetensors", + "model.layers.71.mlp.experts.100.down_proj.weight": "model-00078-of-00101.safetensors", + "model.layers.71.mlp.experts.101.gate_proj.weight": "model-00078-of-00101.safetensors", + "model.layers.71.mlp.experts.101.up_proj.weight": "model-00078-of-00101.safetensors", + "model.layers.71.mlp.experts.101.down_proj.weight": "model-00078-of-00101.safetensors", + "model.layers.71.mlp.experts.102.gate_proj.weight": "model-00078-of-00101.safetensors", + "model.layers.71.mlp.experts.102.up_proj.weight": "model-00078-of-00101.safetensors", + "model.layers.71.mlp.experts.102.down_proj.weight": "model-00078-of-00101.safetensors", + "model.layers.71.mlp.experts.103.gate_proj.weight": "model-00078-of-00101.safetensors", + "model.layers.71.mlp.experts.103.up_proj.weight": "model-00078-of-00101.safetensors", + "model.layers.71.mlp.experts.103.down_proj.weight": "model-00078-of-00101.safetensors", + "model.layers.71.mlp.experts.104.gate_proj.weight": "model-00078-of-00101.safetensors", + "model.layers.71.mlp.experts.104.up_proj.weight": "model-00078-of-00101.safetensors", + "model.layers.71.mlp.experts.104.down_proj.weight": "model-00078-of-00101.safetensors", + "model.layers.71.mlp.experts.105.gate_proj.weight": "model-00078-of-00101.safetensors", + "model.layers.71.mlp.experts.105.up_proj.weight": "model-00078-of-00101.safetensors", + "model.layers.71.mlp.experts.105.down_proj.weight": "model-00078-of-00101.safetensors", + "model.layers.71.mlp.experts.106.gate_proj.weight": "model-00078-of-00101.safetensors", + "model.layers.71.mlp.experts.106.up_proj.weight": "model-00078-of-00101.safetensors", + "model.layers.71.mlp.experts.106.down_proj.weight": "model-00078-of-00101.safetensors", + "model.layers.71.mlp.experts.107.gate_proj.weight": "model-00078-of-00101.safetensors", + "model.layers.71.mlp.experts.107.up_proj.weight": "model-00078-of-00101.safetensors", + "model.layers.71.mlp.experts.107.down_proj.weight": "model-00078-of-00101.safetensors", + "model.layers.71.mlp.experts.108.gate_proj.weight": "model-00078-of-00101.safetensors", + "model.layers.71.mlp.experts.108.up_proj.weight": "model-00078-of-00101.safetensors", + "model.layers.71.mlp.experts.108.down_proj.weight": "model-00078-of-00101.safetensors", + "model.layers.71.mlp.experts.109.gate_proj.weight": "model-00078-of-00101.safetensors", + "model.layers.71.mlp.experts.109.up_proj.weight": "model-00078-of-00101.safetensors", + "model.layers.71.mlp.experts.109.down_proj.weight": "model-00078-of-00101.safetensors", + "model.layers.71.mlp.experts.110.gate_proj.weight": "model-00078-of-00101.safetensors", + "model.layers.71.mlp.experts.110.up_proj.weight": "model-00078-of-00101.safetensors", + "model.layers.71.mlp.experts.110.down_proj.weight": "model-00078-of-00101.safetensors", + "model.layers.71.mlp.experts.111.gate_proj.weight": "model-00078-of-00101.safetensors", + "model.layers.71.mlp.experts.111.up_proj.weight": "model-00078-of-00101.safetensors", + "model.layers.71.mlp.experts.111.down_proj.weight": "model-00078-of-00101.safetensors", + "model.layers.71.mlp.experts.112.gate_proj.weight": "model-00078-of-00101.safetensors", + "model.layers.71.mlp.experts.112.up_proj.weight": "model-00078-of-00101.safetensors", + "model.layers.71.mlp.experts.112.down_proj.weight": "model-00078-of-00101.safetensors", + "model.layers.71.mlp.experts.113.gate_proj.weight": "model-00078-of-00101.safetensors", + "model.layers.71.mlp.experts.113.up_proj.weight": "model-00078-of-00101.safetensors", + "model.layers.71.mlp.experts.113.down_proj.weight": "model-00078-of-00101.safetensors", + "model.layers.71.mlp.experts.114.gate_proj.weight": "model-00078-of-00101.safetensors", + "model.layers.71.mlp.experts.114.up_proj.weight": "model-00078-of-00101.safetensors", + "model.layers.71.mlp.experts.114.down_proj.weight": "model-00078-of-00101.safetensors", + "model.layers.71.mlp.experts.115.gate_proj.weight": "model-00078-of-00101.safetensors", + "model.layers.71.mlp.experts.115.up_proj.weight": "model-00078-of-00101.safetensors", + "model.layers.71.mlp.experts.115.down_proj.weight": "model-00078-of-00101.safetensors", + "model.layers.71.mlp.experts.116.gate_proj.weight": "model-00078-of-00101.safetensors", + "model.layers.71.mlp.experts.116.up_proj.weight": "model-00078-of-00101.safetensors", + "model.layers.71.mlp.experts.116.down_proj.weight": "model-00078-of-00101.safetensors", + "model.layers.71.mlp.experts.117.gate_proj.weight": "model-00078-of-00101.safetensors", + "model.layers.71.mlp.experts.117.up_proj.weight": "model-00078-of-00101.safetensors", + "model.layers.71.mlp.experts.117.down_proj.weight": "model-00078-of-00101.safetensors", + "model.layers.71.mlp.experts.118.gate_proj.weight": "model-00078-of-00101.safetensors", + "model.layers.71.mlp.experts.118.up_proj.weight": "model-00078-of-00101.safetensors", + "model.layers.71.mlp.experts.118.down_proj.weight": "model-00078-of-00101.safetensors", + "model.layers.71.mlp.experts.119.gate_proj.weight": "model-00078-of-00101.safetensors", + "model.layers.71.mlp.experts.119.up_proj.weight": "model-00078-of-00101.safetensors", + "model.layers.71.mlp.experts.119.down_proj.weight": "model-00078-of-00101.safetensors", + "model.layers.71.mlp.gate.weight": "model-00078-of-00101.safetensors", + "model.layers.71.mlp.gate.e_score_correction_bias": "model-00078-of-00101.safetensors", + "model.layers.71.mlp.shared_experts.gate_proj.weight": "model-00078-of-00101.safetensors", + "model.layers.71.mlp.shared_experts.up_proj.weight": "model-00078-of-00101.safetensors", + "model.layers.71.mlp.shared_experts.down_proj.weight": "model-00078-of-00101.safetensors", + "model.layers.71.input_layernorm.weight": "model-00078-of-00101.safetensors", + "model.layers.71.post_attention_layernorm.weight": "model-00078-of-00101.safetensors", + "model.layers.72.self_attn.q_proj.weight": "model-00078-of-00101.safetensors", + "model.layers.72.self_attn.q_proj.bias": "model-00078-of-00101.safetensors", + "model.layers.72.self_attn.k_proj.weight": "model-00078-of-00101.safetensors", + "model.layers.72.self_attn.k_proj.bias": "model-00078-of-00101.safetensors", + "model.layers.72.self_attn.v_proj.weight": "model-00078-of-00101.safetensors", + "model.layers.72.self_attn.v_proj.bias": "model-00078-of-00101.safetensors", + "model.layers.72.self_attn.o_proj.weight": "model-00078-of-00101.safetensors", + "model.layers.72.self_attn.q_norm.weight": "model-00078-of-00101.safetensors", + "model.layers.72.self_attn.k_norm.weight": "model-00078-of-00101.safetensors", + "model.layers.72.mlp.experts.0.gate_proj.weight": "model-00078-of-00101.safetensors", + "model.layers.72.mlp.experts.0.up_proj.weight": "model-00078-of-00101.safetensors", + "model.layers.72.mlp.experts.0.down_proj.weight": "model-00078-of-00101.safetensors", + "model.layers.72.mlp.experts.1.gate_proj.weight": "model-00078-of-00101.safetensors", + "model.layers.72.mlp.experts.1.up_proj.weight": "model-00078-of-00101.safetensors", + "model.layers.72.mlp.experts.1.down_proj.weight": "model-00078-of-00101.safetensors", + "model.layers.72.mlp.experts.2.gate_proj.weight": "model-00078-of-00101.safetensors", + "model.layers.72.mlp.experts.2.up_proj.weight": "model-00078-of-00101.safetensors", + "model.layers.72.mlp.experts.2.down_proj.weight": "model-00078-of-00101.safetensors", + "model.layers.72.mlp.experts.3.gate_proj.weight": "model-00078-of-00101.safetensors", + "model.layers.72.mlp.experts.3.up_proj.weight": "model-00078-of-00101.safetensors", + "model.layers.72.mlp.experts.3.down_proj.weight": "model-00078-of-00101.safetensors", + "model.layers.72.mlp.experts.4.gate_proj.weight": "model-00078-of-00101.safetensors", + "model.layers.72.mlp.experts.4.up_proj.weight": "model-00078-of-00101.safetensors", + "model.layers.72.mlp.experts.4.down_proj.weight": "model-00078-of-00101.safetensors", + "model.layers.72.mlp.experts.5.gate_proj.weight": "model-00078-of-00101.safetensors", + "model.layers.72.mlp.experts.5.up_proj.weight": "model-00078-of-00101.safetensors", + "model.layers.72.mlp.experts.5.down_proj.weight": "model-00078-of-00101.safetensors", + "model.layers.72.mlp.experts.6.gate_proj.weight": "model-00078-of-00101.safetensors", + "model.layers.72.mlp.experts.6.up_proj.weight": "model-00078-of-00101.safetensors", + "model.layers.72.mlp.experts.6.down_proj.weight": "model-00078-of-00101.safetensors", + "model.layers.72.mlp.experts.7.gate_proj.weight": "model-00078-of-00101.safetensors", + "model.layers.72.mlp.experts.7.up_proj.weight": "model-00078-of-00101.safetensors", + "model.layers.72.mlp.experts.7.down_proj.weight": "model-00078-of-00101.safetensors", + "model.layers.72.mlp.experts.8.gate_proj.weight": "model-00078-of-00101.safetensors", + "model.layers.72.mlp.experts.8.up_proj.weight": "model-00078-of-00101.safetensors", + "model.layers.72.mlp.experts.8.down_proj.weight": "model-00078-of-00101.safetensors", + "model.layers.72.mlp.experts.9.gate_proj.weight": "model-00078-of-00101.safetensors", + "model.layers.72.mlp.experts.9.up_proj.weight": "model-00078-of-00101.safetensors", + "model.layers.72.mlp.experts.9.down_proj.weight": "model-00078-of-00101.safetensors", + "model.layers.72.mlp.experts.10.gate_proj.weight": "model-00078-of-00101.safetensors", + "model.layers.72.mlp.experts.10.up_proj.weight": "model-00078-of-00101.safetensors", + "model.layers.72.mlp.experts.10.down_proj.weight": "model-00078-of-00101.safetensors", + "model.layers.72.mlp.experts.11.gate_proj.weight": "model-00078-of-00101.safetensors", + "model.layers.72.mlp.experts.11.up_proj.weight": "model-00078-of-00101.safetensors", + "model.layers.72.mlp.experts.11.down_proj.weight": "model-00078-of-00101.safetensors", + "model.layers.72.mlp.experts.12.gate_proj.weight": "model-00078-of-00101.safetensors", + "model.layers.72.mlp.experts.12.up_proj.weight": "model-00078-of-00101.safetensors", + "model.layers.72.mlp.experts.12.down_proj.weight": "model-00078-of-00101.safetensors", + "model.layers.72.mlp.experts.13.gate_proj.weight": "model-00078-of-00101.safetensors", + "model.layers.72.mlp.experts.13.up_proj.weight": "model-00078-of-00101.safetensors", + "model.layers.72.mlp.experts.13.down_proj.weight": "model-00078-of-00101.safetensors", + "model.layers.72.mlp.experts.14.gate_proj.weight": "model-00078-of-00101.safetensors", + "model.layers.72.mlp.experts.14.up_proj.weight": "model-00078-of-00101.safetensors", + "model.layers.72.mlp.experts.14.down_proj.weight": "model-00078-of-00101.safetensors", + "model.layers.72.mlp.experts.15.gate_proj.weight": "model-00078-of-00101.safetensors", + "model.layers.72.mlp.experts.15.up_proj.weight": "model-00078-of-00101.safetensors", + "model.layers.72.mlp.experts.15.down_proj.weight": "model-00078-of-00101.safetensors", + "model.layers.72.mlp.experts.16.gate_proj.weight": "model-00078-of-00101.safetensors", + "model.layers.72.mlp.experts.16.up_proj.weight": "model-00078-of-00101.safetensors", + "model.layers.72.mlp.experts.16.down_proj.weight": "model-00078-of-00101.safetensors", + "model.layers.72.mlp.experts.17.gate_proj.weight": "model-00078-of-00101.safetensors", + "model.layers.72.mlp.experts.17.up_proj.weight": "model-00078-of-00101.safetensors", + "model.layers.72.mlp.experts.17.down_proj.weight": "model-00078-of-00101.safetensors", + "model.layers.72.mlp.experts.18.gate_proj.weight": "model-00078-of-00101.safetensors", + "model.layers.72.mlp.experts.18.up_proj.weight": "model-00078-of-00101.safetensors", + "model.layers.72.mlp.experts.18.down_proj.weight": "model-00078-of-00101.safetensors", + "model.layers.72.mlp.experts.19.gate_proj.weight": "model-00078-of-00101.safetensors", + "model.layers.72.mlp.experts.19.up_proj.weight": "model-00078-of-00101.safetensors", + "model.layers.72.mlp.experts.19.down_proj.weight": "model-00078-of-00101.safetensors", + "model.layers.72.mlp.experts.20.gate_proj.weight": "model-00078-of-00101.safetensors", + "model.layers.72.mlp.experts.20.up_proj.weight": "model-00078-of-00101.safetensors", + "model.layers.72.mlp.experts.20.down_proj.weight": "model-00078-of-00101.safetensors", + "model.layers.72.mlp.experts.21.gate_proj.weight": "model-00078-of-00101.safetensors", + "model.layers.72.mlp.experts.21.up_proj.weight": "model-00079-of-00101.safetensors", + "model.layers.72.mlp.experts.21.down_proj.weight": "model-00079-of-00101.safetensors", + "model.layers.72.mlp.experts.22.gate_proj.weight": "model-00079-of-00101.safetensors", + "model.layers.72.mlp.experts.22.up_proj.weight": "model-00079-of-00101.safetensors", + "model.layers.72.mlp.experts.22.down_proj.weight": "model-00079-of-00101.safetensors", + "model.layers.72.mlp.experts.23.gate_proj.weight": "model-00079-of-00101.safetensors", + "model.layers.72.mlp.experts.23.up_proj.weight": "model-00079-of-00101.safetensors", + "model.layers.72.mlp.experts.23.down_proj.weight": "model-00079-of-00101.safetensors", + "model.layers.72.mlp.experts.24.gate_proj.weight": "model-00079-of-00101.safetensors", + "model.layers.72.mlp.experts.24.up_proj.weight": "model-00079-of-00101.safetensors", + "model.layers.72.mlp.experts.24.down_proj.weight": "model-00079-of-00101.safetensors", + "model.layers.72.mlp.experts.25.gate_proj.weight": "model-00079-of-00101.safetensors", + "model.layers.72.mlp.experts.25.up_proj.weight": "model-00079-of-00101.safetensors", + "model.layers.72.mlp.experts.25.down_proj.weight": "model-00079-of-00101.safetensors", + "model.layers.72.mlp.experts.26.gate_proj.weight": "model-00079-of-00101.safetensors", + "model.layers.72.mlp.experts.26.up_proj.weight": "model-00079-of-00101.safetensors", + "model.layers.72.mlp.experts.26.down_proj.weight": "model-00079-of-00101.safetensors", + "model.layers.72.mlp.experts.27.gate_proj.weight": "model-00079-of-00101.safetensors", + "model.layers.72.mlp.experts.27.up_proj.weight": "model-00079-of-00101.safetensors", + "model.layers.72.mlp.experts.27.down_proj.weight": "model-00079-of-00101.safetensors", + "model.layers.72.mlp.experts.28.gate_proj.weight": "model-00079-of-00101.safetensors", + "model.layers.72.mlp.experts.28.up_proj.weight": "model-00079-of-00101.safetensors", + "model.layers.72.mlp.experts.28.down_proj.weight": "model-00079-of-00101.safetensors", + "model.layers.72.mlp.experts.29.gate_proj.weight": "model-00079-of-00101.safetensors", + "model.layers.72.mlp.experts.29.up_proj.weight": "model-00079-of-00101.safetensors", + "model.layers.72.mlp.experts.29.down_proj.weight": "model-00079-of-00101.safetensors", + "model.layers.72.mlp.experts.30.gate_proj.weight": "model-00079-of-00101.safetensors", + "model.layers.72.mlp.experts.30.up_proj.weight": "model-00079-of-00101.safetensors", + "model.layers.72.mlp.experts.30.down_proj.weight": "model-00079-of-00101.safetensors", + "model.layers.72.mlp.experts.31.gate_proj.weight": "model-00079-of-00101.safetensors", + "model.layers.72.mlp.experts.31.up_proj.weight": "model-00079-of-00101.safetensors", + "model.layers.72.mlp.experts.31.down_proj.weight": "model-00079-of-00101.safetensors", + "model.layers.72.mlp.experts.32.gate_proj.weight": "model-00079-of-00101.safetensors", + "model.layers.72.mlp.experts.32.up_proj.weight": "model-00079-of-00101.safetensors", + "model.layers.72.mlp.experts.32.down_proj.weight": "model-00079-of-00101.safetensors", + "model.layers.72.mlp.experts.33.gate_proj.weight": "model-00079-of-00101.safetensors", + "model.layers.72.mlp.experts.33.up_proj.weight": "model-00079-of-00101.safetensors", + "model.layers.72.mlp.experts.33.down_proj.weight": "model-00079-of-00101.safetensors", + "model.layers.72.mlp.experts.34.gate_proj.weight": "model-00079-of-00101.safetensors", + "model.layers.72.mlp.experts.34.up_proj.weight": "model-00079-of-00101.safetensors", + "model.layers.72.mlp.experts.34.down_proj.weight": "model-00079-of-00101.safetensors", + "model.layers.72.mlp.experts.35.gate_proj.weight": "model-00079-of-00101.safetensors", + "model.layers.72.mlp.experts.35.up_proj.weight": "model-00079-of-00101.safetensors", + "model.layers.72.mlp.experts.35.down_proj.weight": "model-00079-of-00101.safetensors", + "model.layers.72.mlp.experts.36.gate_proj.weight": "model-00079-of-00101.safetensors", + "model.layers.72.mlp.experts.36.up_proj.weight": "model-00079-of-00101.safetensors", + "model.layers.72.mlp.experts.36.down_proj.weight": "model-00079-of-00101.safetensors", + "model.layers.72.mlp.experts.37.gate_proj.weight": "model-00079-of-00101.safetensors", + "model.layers.72.mlp.experts.37.up_proj.weight": "model-00079-of-00101.safetensors", + "model.layers.72.mlp.experts.37.down_proj.weight": "model-00079-of-00101.safetensors", + "model.layers.72.mlp.experts.38.gate_proj.weight": "model-00079-of-00101.safetensors", + "model.layers.72.mlp.experts.38.up_proj.weight": "model-00079-of-00101.safetensors", + "model.layers.72.mlp.experts.38.down_proj.weight": "model-00079-of-00101.safetensors", + "model.layers.72.mlp.experts.39.gate_proj.weight": "model-00079-of-00101.safetensors", + "model.layers.72.mlp.experts.39.up_proj.weight": "model-00079-of-00101.safetensors", + "model.layers.72.mlp.experts.39.down_proj.weight": "model-00079-of-00101.safetensors", + "model.layers.72.mlp.experts.40.gate_proj.weight": "model-00079-of-00101.safetensors", + "model.layers.72.mlp.experts.40.up_proj.weight": "model-00079-of-00101.safetensors", + "model.layers.72.mlp.experts.40.down_proj.weight": "model-00079-of-00101.safetensors", + "model.layers.72.mlp.experts.41.gate_proj.weight": "model-00079-of-00101.safetensors", + "model.layers.72.mlp.experts.41.up_proj.weight": "model-00079-of-00101.safetensors", + "model.layers.72.mlp.experts.41.down_proj.weight": "model-00079-of-00101.safetensors", + "model.layers.72.mlp.experts.42.gate_proj.weight": "model-00079-of-00101.safetensors", + "model.layers.72.mlp.experts.42.up_proj.weight": "model-00079-of-00101.safetensors", + "model.layers.72.mlp.experts.42.down_proj.weight": "model-00079-of-00101.safetensors", + "model.layers.72.mlp.experts.43.gate_proj.weight": "model-00079-of-00101.safetensors", + "model.layers.72.mlp.experts.43.up_proj.weight": "model-00079-of-00101.safetensors", + "model.layers.72.mlp.experts.43.down_proj.weight": "model-00079-of-00101.safetensors", + "model.layers.72.mlp.experts.44.gate_proj.weight": "model-00079-of-00101.safetensors", + "model.layers.72.mlp.experts.44.up_proj.weight": "model-00079-of-00101.safetensors", + "model.layers.72.mlp.experts.44.down_proj.weight": "model-00079-of-00101.safetensors", + "model.layers.72.mlp.experts.45.gate_proj.weight": "model-00079-of-00101.safetensors", + "model.layers.72.mlp.experts.45.up_proj.weight": "model-00079-of-00101.safetensors", + "model.layers.72.mlp.experts.45.down_proj.weight": "model-00079-of-00101.safetensors", + "model.layers.72.mlp.experts.46.gate_proj.weight": "model-00079-of-00101.safetensors", + "model.layers.72.mlp.experts.46.up_proj.weight": "model-00079-of-00101.safetensors", + "model.layers.72.mlp.experts.46.down_proj.weight": "model-00079-of-00101.safetensors", + "model.layers.72.mlp.experts.47.gate_proj.weight": "model-00079-of-00101.safetensors", + "model.layers.72.mlp.experts.47.up_proj.weight": "model-00079-of-00101.safetensors", + "model.layers.72.mlp.experts.47.down_proj.weight": "model-00079-of-00101.safetensors", + "model.layers.72.mlp.experts.48.gate_proj.weight": "model-00079-of-00101.safetensors", + "model.layers.72.mlp.experts.48.up_proj.weight": "model-00079-of-00101.safetensors", + "model.layers.72.mlp.experts.48.down_proj.weight": "model-00079-of-00101.safetensors", + "model.layers.72.mlp.experts.49.gate_proj.weight": "model-00079-of-00101.safetensors", + "model.layers.72.mlp.experts.49.up_proj.weight": "model-00079-of-00101.safetensors", + "model.layers.72.mlp.experts.49.down_proj.weight": "model-00079-of-00101.safetensors", + "model.layers.72.mlp.experts.50.gate_proj.weight": "model-00079-of-00101.safetensors", + "model.layers.72.mlp.experts.50.up_proj.weight": "model-00079-of-00101.safetensors", + "model.layers.72.mlp.experts.50.down_proj.weight": "model-00079-of-00101.safetensors", + "model.layers.72.mlp.experts.51.gate_proj.weight": "model-00079-of-00101.safetensors", + "model.layers.72.mlp.experts.51.up_proj.weight": "model-00079-of-00101.safetensors", + "model.layers.72.mlp.experts.51.down_proj.weight": "model-00079-of-00101.safetensors", + "model.layers.72.mlp.experts.52.gate_proj.weight": "model-00079-of-00101.safetensors", + "model.layers.72.mlp.experts.52.up_proj.weight": "model-00079-of-00101.safetensors", + "model.layers.72.mlp.experts.52.down_proj.weight": "model-00079-of-00101.safetensors", + "model.layers.72.mlp.experts.53.gate_proj.weight": "model-00079-of-00101.safetensors", + "model.layers.72.mlp.experts.53.up_proj.weight": "model-00079-of-00101.safetensors", + "model.layers.72.mlp.experts.53.down_proj.weight": "model-00079-of-00101.safetensors", + "model.layers.72.mlp.experts.54.gate_proj.weight": "model-00079-of-00101.safetensors", + "model.layers.72.mlp.experts.54.up_proj.weight": "model-00079-of-00101.safetensors", + "model.layers.72.mlp.experts.54.down_proj.weight": "model-00079-of-00101.safetensors", + "model.layers.72.mlp.experts.55.gate_proj.weight": "model-00079-of-00101.safetensors", + "model.layers.72.mlp.experts.55.up_proj.weight": "model-00079-of-00101.safetensors", + "model.layers.72.mlp.experts.55.down_proj.weight": "model-00079-of-00101.safetensors", + "model.layers.72.mlp.experts.56.gate_proj.weight": "model-00079-of-00101.safetensors", + "model.layers.72.mlp.experts.56.up_proj.weight": "model-00079-of-00101.safetensors", + "model.layers.72.mlp.experts.56.down_proj.weight": "model-00079-of-00101.safetensors", + "model.layers.72.mlp.experts.57.gate_proj.weight": "model-00079-of-00101.safetensors", + "model.layers.72.mlp.experts.57.up_proj.weight": "model-00079-of-00101.safetensors", + "model.layers.72.mlp.experts.57.down_proj.weight": "model-00079-of-00101.safetensors", + "model.layers.72.mlp.experts.58.gate_proj.weight": "model-00079-of-00101.safetensors", + "model.layers.72.mlp.experts.58.up_proj.weight": "model-00079-of-00101.safetensors", + "model.layers.72.mlp.experts.58.down_proj.weight": "model-00079-of-00101.safetensors", + "model.layers.72.mlp.experts.59.gate_proj.weight": "model-00079-of-00101.safetensors", + "model.layers.72.mlp.experts.59.up_proj.weight": "model-00079-of-00101.safetensors", + "model.layers.72.mlp.experts.59.down_proj.weight": "model-00079-of-00101.safetensors", + "model.layers.72.mlp.experts.60.gate_proj.weight": "model-00079-of-00101.safetensors", + "model.layers.72.mlp.experts.60.up_proj.weight": "model-00079-of-00101.safetensors", + "model.layers.72.mlp.experts.60.down_proj.weight": "model-00079-of-00101.safetensors", + "model.layers.72.mlp.experts.61.gate_proj.weight": "model-00079-of-00101.safetensors", + "model.layers.72.mlp.experts.61.up_proj.weight": "model-00079-of-00101.safetensors", + "model.layers.72.mlp.experts.61.down_proj.weight": "model-00079-of-00101.safetensors", + "model.layers.72.mlp.experts.62.gate_proj.weight": "model-00079-of-00101.safetensors", + "model.layers.72.mlp.experts.62.up_proj.weight": "model-00079-of-00101.safetensors", + "model.layers.72.mlp.experts.62.down_proj.weight": "model-00079-of-00101.safetensors", + "model.layers.72.mlp.experts.63.gate_proj.weight": "model-00079-of-00101.safetensors", + "model.layers.72.mlp.experts.63.up_proj.weight": "model-00079-of-00101.safetensors", + "model.layers.72.mlp.experts.63.down_proj.weight": "model-00079-of-00101.safetensors", + "model.layers.72.mlp.experts.64.gate_proj.weight": "model-00079-of-00101.safetensors", + "model.layers.72.mlp.experts.64.up_proj.weight": "model-00079-of-00101.safetensors", + "model.layers.72.mlp.experts.64.down_proj.weight": "model-00079-of-00101.safetensors", + "model.layers.72.mlp.experts.65.gate_proj.weight": "model-00079-of-00101.safetensors", + "model.layers.72.mlp.experts.65.up_proj.weight": "model-00079-of-00101.safetensors", + "model.layers.72.mlp.experts.65.down_proj.weight": "model-00079-of-00101.safetensors", + "model.layers.72.mlp.experts.66.gate_proj.weight": "model-00079-of-00101.safetensors", + "model.layers.72.mlp.experts.66.up_proj.weight": "model-00079-of-00101.safetensors", + "model.layers.72.mlp.experts.66.down_proj.weight": "model-00079-of-00101.safetensors", + "model.layers.72.mlp.experts.67.gate_proj.weight": "model-00079-of-00101.safetensors", + "model.layers.72.mlp.experts.67.up_proj.weight": "model-00079-of-00101.safetensors", + "model.layers.72.mlp.experts.67.down_proj.weight": "model-00079-of-00101.safetensors", + "model.layers.72.mlp.experts.68.gate_proj.weight": "model-00079-of-00101.safetensors", + "model.layers.72.mlp.experts.68.up_proj.weight": "model-00079-of-00101.safetensors", + "model.layers.72.mlp.experts.68.down_proj.weight": "model-00079-of-00101.safetensors", + "model.layers.72.mlp.experts.69.gate_proj.weight": "model-00079-of-00101.safetensors", + "model.layers.72.mlp.experts.69.up_proj.weight": "model-00079-of-00101.safetensors", + "model.layers.72.mlp.experts.69.down_proj.weight": "model-00079-of-00101.safetensors", + "model.layers.72.mlp.experts.70.gate_proj.weight": "model-00079-of-00101.safetensors", + "model.layers.72.mlp.experts.70.up_proj.weight": "model-00079-of-00101.safetensors", + "model.layers.72.mlp.experts.70.down_proj.weight": "model-00079-of-00101.safetensors", + "model.layers.72.mlp.experts.71.gate_proj.weight": "model-00079-of-00101.safetensors", + "model.layers.72.mlp.experts.71.up_proj.weight": "model-00079-of-00101.safetensors", + "model.layers.72.mlp.experts.71.down_proj.weight": "model-00079-of-00101.safetensors", + "model.layers.72.mlp.experts.72.gate_proj.weight": "model-00079-of-00101.safetensors", + "model.layers.72.mlp.experts.72.up_proj.weight": "model-00079-of-00101.safetensors", + "model.layers.72.mlp.experts.72.down_proj.weight": "model-00079-of-00101.safetensors", + "model.layers.72.mlp.experts.73.gate_proj.weight": "model-00079-of-00101.safetensors", + "model.layers.72.mlp.experts.73.up_proj.weight": "model-00079-of-00101.safetensors", + "model.layers.72.mlp.experts.73.down_proj.weight": "model-00079-of-00101.safetensors", + "model.layers.72.mlp.experts.74.gate_proj.weight": "model-00079-of-00101.safetensors", + "model.layers.72.mlp.experts.74.up_proj.weight": "model-00079-of-00101.safetensors", + "model.layers.72.mlp.experts.74.down_proj.weight": "model-00079-of-00101.safetensors", + "model.layers.72.mlp.experts.75.gate_proj.weight": "model-00079-of-00101.safetensors", + "model.layers.72.mlp.experts.75.up_proj.weight": "model-00079-of-00101.safetensors", + "model.layers.72.mlp.experts.75.down_proj.weight": "model-00079-of-00101.safetensors", + "model.layers.72.mlp.experts.76.gate_proj.weight": "model-00079-of-00101.safetensors", + "model.layers.72.mlp.experts.76.up_proj.weight": "model-00079-of-00101.safetensors", + "model.layers.72.mlp.experts.76.down_proj.weight": "model-00079-of-00101.safetensors", + "model.layers.72.mlp.experts.77.gate_proj.weight": "model-00079-of-00101.safetensors", + "model.layers.72.mlp.experts.77.up_proj.weight": "model-00079-of-00101.safetensors", + "model.layers.72.mlp.experts.77.down_proj.weight": "model-00079-of-00101.safetensors", + "model.layers.72.mlp.experts.78.gate_proj.weight": "model-00079-of-00101.safetensors", + "model.layers.72.mlp.experts.78.up_proj.weight": "model-00079-of-00101.safetensors", + "model.layers.72.mlp.experts.78.down_proj.weight": "model-00079-of-00101.safetensors", + "model.layers.72.mlp.experts.79.gate_proj.weight": "model-00079-of-00101.safetensors", + "model.layers.72.mlp.experts.79.up_proj.weight": "model-00079-of-00101.safetensors", + "model.layers.72.mlp.experts.79.down_proj.weight": "model-00079-of-00101.safetensors", + "model.layers.72.mlp.experts.80.gate_proj.weight": "model-00079-of-00101.safetensors", + "model.layers.72.mlp.experts.80.up_proj.weight": "model-00079-of-00101.safetensors", + "model.layers.72.mlp.experts.80.down_proj.weight": "model-00079-of-00101.safetensors", + "model.layers.72.mlp.experts.81.gate_proj.weight": "model-00079-of-00101.safetensors", + "model.layers.72.mlp.experts.81.up_proj.weight": "model-00079-of-00101.safetensors", + "model.layers.72.mlp.experts.81.down_proj.weight": "model-00079-of-00101.safetensors", + "model.layers.72.mlp.experts.82.gate_proj.weight": "model-00079-of-00101.safetensors", + "model.layers.72.mlp.experts.82.up_proj.weight": "model-00079-of-00101.safetensors", + "model.layers.72.mlp.experts.82.down_proj.weight": "model-00079-of-00101.safetensors", + "model.layers.72.mlp.experts.83.gate_proj.weight": "model-00079-of-00101.safetensors", + "model.layers.72.mlp.experts.83.up_proj.weight": "model-00079-of-00101.safetensors", + "model.layers.72.mlp.experts.83.down_proj.weight": "model-00079-of-00101.safetensors", + "model.layers.72.mlp.experts.84.gate_proj.weight": "model-00079-of-00101.safetensors", + "model.layers.72.mlp.experts.84.up_proj.weight": "model-00079-of-00101.safetensors", + "model.layers.72.mlp.experts.84.down_proj.weight": "model-00079-of-00101.safetensors", + "model.layers.72.mlp.experts.85.gate_proj.weight": "model-00079-of-00101.safetensors", + "model.layers.72.mlp.experts.85.up_proj.weight": "model-00079-of-00101.safetensors", + "model.layers.72.mlp.experts.85.down_proj.weight": "model-00079-of-00101.safetensors", + "model.layers.72.mlp.experts.86.gate_proj.weight": "model-00079-of-00101.safetensors", + "model.layers.72.mlp.experts.86.up_proj.weight": "model-00079-of-00101.safetensors", + "model.layers.72.mlp.experts.86.down_proj.weight": "model-00079-of-00101.safetensors", + "model.layers.72.mlp.experts.87.gate_proj.weight": "model-00079-of-00101.safetensors", + "model.layers.72.mlp.experts.87.up_proj.weight": "model-00079-of-00101.safetensors", + "model.layers.72.mlp.experts.87.down_proj.weight": "model-00079-of-00101.safetensors", + "model.layers.72.mlp.experts.88.gate_proj.weight": "model-00079-of-00101.safetensors", + "model.layers.72.mlp.experts.88.up_proj.weight": "model-00079-of-00101.safetensors", + "model.layers.72.mlp.experts.88.down_proj.weight": "model-00079-of-00101.safetensors", + "model.layers.72.mlp.experts.89.gate_proj.weight": "model-00079-of-00101.safetensors", + "model.layers.72.mlp.experts.89.up_proj.weight": "model-00079-of-00101.safetensors", + "model.layers.72.mlp.experts.89.down_proj.weight": "model-00079-of-00101.safetensors", + "model.layers.72.mlp.experts.90.gate_proj.weight": "model-00079-of-00101.safetensors", + "model.layers.72.mlp.experts.90.up_proj.weight": "model-00079-of-00101.safetensors", + "model.layers.72.mlp.experts.90.down_proj.weight": "model-00079-of-00101.safetensors", + "model.layers.72.mlp.experts.91.gate_proj.weight": "model-00079-of-00101.safetensors", + "model.layers.72.mlp.experts.91.up_proj.weight": "model-00079-of-00101.safetensors", + "model.layers.72.mlp.experts.91.down_proj.weight": "model-00079-of-00101.safetensors", + "model.layers.72.mlp.experts.92.gate_proj.weight": "model-00079-of-00101.safetensors", + "model.layers.72.mlp.experts.92.up_proj.weight": "model-00079-of-00101.safetensors", + "model.layers.72.mlp.experts.92.down_proj.weight": "model-00079-of-00101.safetensors", + "model.layers.72.mlp.experts.93.gate_proj.weight": "model-00079-of-00101.safetensors", + "model.layers.72.mlp.experts.93.up_proj.weight": "model-00079-of-00101.safetensors", + "model.layers.72.mlp.experts.93.down_proj.weight": "model-00079-of-00101.safetensors", + "model.layers.72.mlp.experts.94.gate_proj.weight": "model-00079-of-00101.safetensors", + "model.layers.72.mlp.experts.94.up_proj.weight": "model-00079-of-00101.safetensors", + "model.layers.72.mlp.experts.94.down_proj.weight": "model-00079-of-00101.safetensors", + "model.layers.72.mlp.experts.95.gate_proj.weight": "model-00079-of-00101.safetensors", + "model.layers.72.mlp.experts.95.up_proj.weight": "model-00079-of-00101.safetensors", + "model.layers.72.mlp.experts.95.down_proj.weight": "model-00079-of-00101.safetensors", + "model.layers.72.mlp.experts.96.gate_proj.weight": "model-00079-of-00101.safetensors", + "model.layers.72.mlp.experts.96.up_proj.weight": "model-00079-of-00101.safetensors", + "model.layers.72.mlp.experts.96.down_proj.weight": "model-00079-of-00101.safetensors", + "model.layers.72.mlp.experts.97.gate_proj.weight": "model-00079-of-00101.safetensors", + "model.layers.72.mlp.experts.97.up_proj.weight": "model-00079-of-00101.safetensors", + "model.layers.72.mlp.experts.97.down_proj.weight": "model-00079-of-00101.safetensors", + "model.layers.72.mlp.experts.98.gate_proj.weight": "model-00079-of-00101.safetensors", + "model.layers.72.mlp.experts.98.up_proj.weight": "model-00079-of-00101.safetensors", + "model.layers.72.mlp.experts.98.down_proj.weight": "model-00079-of-00101.safetensors", + "model.layers.72.mlp.experts.99.gate_proj.weight": "model-00079-of-00101.safetensors", + "model.layers.72.mlp.experts.99.up_proj.weight": "model-00079-of-00101.safetensors", + "model.layers.72.mlp.experts.99.down_proj.weight": "model-00079-of-00101.safetensors", + "model.layers.72.mlp.experts.100.gate_proj.weight": "model-00079-of-00101.safetensors", + "model.layers.72.mlp.experts.100.up_proj.weight": "model-00079-of-00101.safetensors", + "model.layers.72.mlp.experts.100.down_proj.weight": "model-00079-of-00101.safetensors", + "model.layers.72.mlp.experts.101.gate_proj.weight": "model-00079-of-00101.safetensors", + "model.layers.72.mlp.experts.101.up_proj.weight": "model-00079-of-00101.safetensors", + "model.layers.72.mlp.experts.101.down_proj.weight": "model-00079-of-00101.safetensors", + "model.layers.72.mlp.experts.102.gate_proj.weight": "model-00079-of-00101.safetensors", + "model.layers.72.mlp.experts.102.up_proj.weight": "model-00079-of-00101.safetensors", + "model.layers.72.mlp.experts.102.down_proj.weight": "model-00079-of-00101.safetensors", + "model.layers.72.mlp.experts.103.gate_proj.weight": "model-00079-of-00101.safetensors", + "model.layers.72.mlp.experts.103.up_proj.weight": "model-00079-of-00101.safetensors", + "model.layers.72.mlp.experts.103.down_proj.weight": "model-00079-of-00101.safetensors", + "model.layers.72.mlp.experts.104.gate_proj.weight": "model-00079-of-00101.safetensors", + "model.layers.72.mlp.experts.104.up_proj.weight": "model-00079-of-00101.safetensors", + "model.layers.72.mlp.experts.104.down_proj.weight": "model-00079-of-00101.safetensors", + "model.layers.72.mlp.experts.105.gate_proj.weight": "model-00079-of-00101.safetensors", + "model.layers.72.mlp.experts.105.up_proj.weight": "model-00079-of-00101.safetensors", + "model.layers.72.mlp.experts.105.down_proj.weight": "model-00079-of-00101.safetensors", + "model.layers.72.mlp.experts.106.gate_proj.weight": "model-00079-of-00101.safetensors", + "model.layers.72.mlp.experts.106.up_proj.weight": "model-00079-of-00101.safetensors", + "model.layers.72.mlp.experts.106.down_proj.weight": "model-00079-of-00101.safetensors", + "model.layers.72.mlp.experts.107.gate_proj.weight": "model-00079-of-00101.safetensors", + "model.layers.72.mlp.experts.107.up_proj.weight": "model-00079-of-00101.safetensors", + "model.layers.72.mlp.experts.107.down_proj.weight": "model-00079-of-00101.safetensors", + "model.layers.72.mlp.experts.108.gate_proj.weight": "model-00079-of-00101.safetensors", + "model.layers.72.mlp.experts.108.up_proj.weight": "model-00079-of-00101.safetensors", + "model.layers.72.mlp.experts.108.down_proj.weight": "model-00079-of-00101.safetensors", + "model.layers.72.mlp.experts.109.gate_proj.weight": "model-00079-of-00101.safetensors", + "model.layers.72.mlp.experts.109.up_proj.weight": "model-00079-of-00101.safetensors", + "model.layers.72.mlp.experts.109.down_proj.weight": "model-00079-of-00101.safetensors", + "model.layers.72.mlp.experts.110.gate_proj.weight": "model-00079-of-00101.safetensors", + "model.layers.72.mlp.experts.110.up_proj.weight": "model-00079-of-00101.safetensors", + "model.layers.72.mlp.experts.110.down_proj.weight": "model-00079-of-00101.safetensors", + "model.layers.72.mlp.experts.111.gate_proj.weight": "model-00079-of-00101.safetensors", + "model.layers.72.mlp.experts.111.up_proj.weight": "model-00079-of-00101.safetensors", + "model.layers.72.mlp.experts.111.down_proj.weight": "model-00079-of-00101.safetensors", + "model.layers.72.mlp.experts.112.gate_proj.weight": "model-00079-of-00101.safetensors", + "model.layers.72.mlp.experts.112.up_proj.weight": "model-00079-of-00101.safetensors", + "model.layers.72.mlp.experts.112.down_proj.weight": "model-00079-of-00101.safetensors", + "model.layers.72.mlp.experts.113.gate_proj.weight": "model-00079-of-00101.safetensors", + "model.layers.72.mlp.experts.113.up_proj.weight": "model-00079-of-00101.safetensors", + "model.layers.72.mlp.experts.113.down_proj.weight": "model-00079-of-00101.safetensors", + "model.layers.72.mlp.experts.114.gate_proj.weight": "model-00079-of-00101.safetensors", + "model.layers.72.mlp.experts.114.up_proj.weight": "model-00079-of-00101.safetensors", + "model.layers.72.mlp.experts.114.down_proj.weight": "model-00079-of-00101.safetensors", + "model.layers.72.mlp.experts.115.gate_proj.weight": "model-00079-of-00101.safetensors", + "model.layers.72.mlp.experts.115.up_proj.weight": "model-00079-of-00101.safetensors", + "model.layers.72.mlp.experts.115.down_proj.weight": "model-00079-of-00101.safetensors", + "model.layers.72.mlp.experts.116.gate_proj.weight": "model-00079-of-00101.safetensors", + "model.layers.72.mlp.experts.116.up_proj.weight": "model-00079-of-00101.safetensors", + "model.layers.72.mlp.experts.116.down_proj.weight": "model-00079-of-00101.safetensors", + "model.layers.72.mlp.experts.117.gate_proj.weight": "model-00079-of-00101.safetensors", + "model.layers.72.mlp.experts.117.up_proj.weight": "model-00079-of-00101.safetensors", + "model.layers.72.mlp.experts.117.down_proj.weight": "model-00079-of-00101.safetensors", + "model.layers.72.mlp.experts.118.gate_proj.weight": "model-00079-of-00101.safetensors", + "model.layers.72.mlp.experts.118.up_proj.weight": "model-00079-of-00101.safetensors", + "model.layers.72.mlp.experts.118.down_proj.weight": "model-00079-of-00101.safetensors", + "model.layers.72.mlp.experts.119.gate_proj.weight": "model-00079-of-00101.safetensors", + "model.layers.72.mlp.experts.119.up_proj.weight": "model-00079-of-00101.safetensors", + "model.layers.72.mlp.experts.119.down_proj.weight": "model-00079-of-00101.safetensors", + "model.layers.72.mlp.gate.weight": "model-00079-of-00101.safetensors", + "model.layers.72.mlp.gate.e_score_correction_bias": "model-00079-of-00101.safetensors", + "model.layers.72.mlp.shared_experts.gate_proj.weight": "model-00079-of-00101.safetensors", + "model.layers.72.mlp.shared_experts.up_proj.weight": "model-00079-of-00101.safetensors", + "model.layers.72.mlp.shared_experts.down_proj.weight": "model-00079-of-00101.safetensors", + "model.layers.72.input_layernorm.weight": "model-00079-of-00101.safetensors", + "model.layers.72.post_attention_layernorm.weight": "model-00079-of-00101.safetensors", + "model.layers.73.self_attn.q_proj.weight": "model-00079-of-00101.safetensors", + "model.layers.73.self_attn.q_proj.bias": "model-00079-of-00101.safetensors", + "model.layers.73.self_attn.k_proj.weight": "model-00079-of-00101.safetensors", + "model.layers.73.self_attn.k_proj.bias": "model-00079-of-00101.safetensors", + "model.layers.73.self_attn.v_proj.weight": "model-00079-of-00101.safetensors", + "model.layers.73.self_attn.v_proj.bias": "model-00079-of-00101.safetensors", + "model.layers.73.self_attn.o_proj.weight": "model-00079-of-00101.safetensors", + "model.layers.73.self_attn.q_norm.weight": "model-00079-of-00101.safetensors", + "model.layers.73.self_attn.k_norm.weight": "model-00079-of-00101.safetensors", + "model.layers.73.mlp.experts.0.gate_proj.weight": "model-00079-of-00101.safetensors", + "model.layers.73.mlp.experts.0.up_proj.weight": "model-00079-of-00101.safetensors", + "model.layers.73.mlp.experts.0.down_proj.weight": "model-00079-of-00101.safetensors", + "model.layers.73.mlp.experts.1.gate_proj.weight": "model-00079-of-00101.safetensors", + "model.layers.73.mlp.experts.1.up_proj.weight": "model-00079-of-00101.safetensors", + "model.layers.73.mlp.experts.1.down_proj.weight": "model-00079-of-00101.safetensors", + "model.layers.73.mlp.experts.2.gate_proj.weight": "model-00079-of-00101.safetensors", + "model.layers.73.mlp.experts.2.up_proj.weight": "model-00079-of-00101.safetensors", + "model.layers.73.mlp.experts.2.down_proj.weight": "model-00079-of-00101.safetensors", + "model.layers.73.mlp.experts.3.gate_proj.weight": "model-00079-of-00101.safetensors", + "model.layers.73.mlp.experts.3.up_proj.weight": "model-00079-of-00101.safetensors", + "model.layers.73.mlp.experts.3.down_proj.weight": "model-00079-of-00101.safetensors", + "model.layers.73.mlp.experts.4.gate_proj.weight": "model-00079-of-00101.safetensors", + "model.layers.73.mlp.experts.4.up_proj.weight": "model-00079-of-00101.safetensors", + "model.layers.73.mlp.experts.4.down_proj.weight": "model-00079-of-00101.safetensors", + "model.layers.73.mlp.experts.5.gate_proj.weight": "model-00079-of-00101.safetensors", + "model.layers.73.mlp.experts.5.up_proj.weight": "model-00079-of-00101.safetensors", + "model.layers.73.mlp.experts.5.down_proj.weight": "model-00079-of-00101.safetensors", + "model.layers.73.mlp.experts.6.gate_proj.weight": "model-00079-of-00101.safetensors", + "model.layers.73.mlp.experts.6.up_proj.weight": "model-00079-of-00101.safetensors", + "model.layers.73.mlp.experts.6.down_proj.weight": "model-00079-of-00101.safetensors", + "model.layers.73.mlp.experts.7.gate_proj.weight": "model-00079-of-00101.safetensors", + "model.layers.73.mlp.experts.7.up_proj.weight": "model-00079-of-00101.safetensors", + "model.layers.73.mlp.experts.7.down_proj.weight": "model-00079-of-00101.safetensors", + "model.layers.73.mlp.experts.8.gate_proj.weight": "model-00080-of-00101.safetensors", + "model.layers.73.mlp.experts.8.up_proj.weight": "model-00080-of-00101.safetensors", + "model.layers.73.mlp.experts.8.down_proj.weight": "model-00080-of-00101.safetensors", + "model.layers.73.mlp.experts.9.gate_proj.weight": "model-00080-of-00101.safetensors", + "model.layers.73.mlp.experts.9.up_proj.weight": "model-00080-of-00101.safetensors", + "model.layers.73.mlp.experts.9.down_proj.weight": "model-00080-of-00101.safetensors", + "model.layers.73.mlp.experts.10.gate_proj.weight": "model-00080-of-00101.safetensors", + "model.layers.73.mlp.experts.10.up_proj.weight": "model-00080-of-00101.safetensors", + "model.layers.73.mlp.experts.10.down_proj.weight": "model-00080-of-00101.safetensors", + "model.layers.73.mlp.experts.11.gate_proj.weight": "model-00080-of-00101.safetensors", + "model.layers.73.mlp.experts.11.up_proj.weight": "model-00080-of-00101.safetensors", + "model.layers.73.mlp.experts.11.down_proj.weight": "model-00080-of-00101.safetensors", + "model.layers.73.mlp.experts.12.gate_proj.weight": "model-00080-of-00101.safetensors", + "model.layers.73.mlp.experts.12.up_proj.weight": "model-00080-of-00101.safetensors", + "model.layers.73.mlp.experts.12.down_proj.weight": "model-00080-of-00101.safetensors", + "model.layers.73.mlp.experts.13.gate_proj.weight": "model-00080-of-00101.safetensors", + "model.layers.73.mlp.experts.13.up_proj.weight": "model-00080-of-00101.safetensors", + "model.layers.73.mlp.experts.13.down_proj.weight": "model-00080-of-00101.safetensors", + "model.layers.73.mlp.experts.14.gate_proj.weight": "model-00080-of-00101.safetensors", + "model.layers.73.mlp.experts.14.up_proj.weight": "model-00080-of-00101.safetensors", + "model.layers.73.mlp.experts.14.down_proj.weight": "model-00080-of-00101.safetensors", + "model.layers.73.mlp.experts.15.gate_proj.weight": "model-00080-of-00101.safetensors", + "model.layers.73.mlp.experts.15.up_proj.weight": "model-00080-of-00101.safetensors", + "model.layers.73.mlp.experts.15.down_proj.weight": "model-00080-of-00101.safetensors", + "model.layers.73.mlp.experts.16.gate_proj.weight": "model-00080-of-00101.safetensors", + "model.layers.73.mlp.experts.16.up_proj.weight": "model-00080-of-00101.safetensors", + "model.layers.73.mlp.experts.16.down_proj.weight": "model-00080-of-00101.safetensors", + "model.layers.73.mlp.experts.17.gate_proj.weight": "model-00080-of-00101.safetensors", + "model.layers.73.mlp.experts.17.up_proj.weight": "model-00080-of-00101.safetensors", + "model.layers.73.mlp.experts.17.down_proj.weight": "model-00080-of-00101.safetensors", + "model.layers.73.mlp.experts.18.gate_proj.weight": "model-00080-of-00101.safetensors", + "model.layers.73.mlp.experts.18.up_proj.weight": "model-00080-of-00101.safetensors", + "model.layers.73.mlp.experts.18.down_proj.weight": "model-00080-of-00101.safetensors", + "model.layers.73.mlp.experts.19.gate_proj.weight": "model-00080-of-00101.safetensors", + "model.layers.73.mlp.experts.19.up_proj.weight": "model-00080-of-00101.safetensors", + "model.layers.73.mlp.experts.19.down_proj.weight": "model-00080-of-00101.safetensors", + "model.layers.73.mlp.experts.20.gate_proj.weight": "model-00080-of-00101.safetensors", + "model.layers.73.mlp.experts.20.up_proj.weight": "model-00080-of-00101.safetensors", + "model.layers.73.mlp.experts.20.down_proj.weight": "model-00080-of-00101.safetensors", + "model.layers.73.mlp.experts.21.gate_proj.weight": "model-00080-of-00101.safetensors", + "model.layers.73.mlp.experts.21.up_proj.weight": "model-00080-of-00101.safetensors", + "model.layers.73.mlp.experts.21.down_proj.weight": "model-00080-of-00101.safetensors", + "model.layers.73.mlp.experts.22.gate_proj.weight": "model-00080-of-00101.safetensors", + "model.layers.73.mlp.experts.22.up_proj.weight": "model-00080-of-00101.safetensors", + "model.layers.73.mlp.experts.22.down_proj.weight": "model-00080-of-00101.safetensors", + "model.layers.73.mlp.experts.23.gate_proj.weight": "model-00080-of-00101.safetensors", + "model.layers.73.mlp.experts.23.up_proj.weight": "model-00080-of-00101.safetensors", + "model.layers.73.mlp.experts.23.down_proj.weight": "model-00080-of-00101.safetensors", + "model.layers.73.mlp.experts.24.gate_proj.weight": "model-00080-of-00101.safetensors", + "model.layers.73.mlp.experts.24.up_proj.weight": "model-00080-of-00101.safetensors", + "model.layers.73.mlp.experts.24.down_proj.weight": "model-00080-of-00101.safetensors", + "model.layers.73.mlp.experts.25.gate_proj.weight": "model-00080-of-00101.safetensors", + "model.layers.73.mlp.experts.25.up_proj.weight": "model-00080-of-00101.safetensors", + "model.layers.73.mlp.experts.25.down_proj.weight": "model-00080-of-00101.safetensors", + "model.layers.73.mlp.experts.26.gate_proj.weight": "model-00080-of-00101.safetensors", + "model.layers.73.mlp.experts.26.up_proj.weight": "model-00080-of-00101.safetensors", + "model.layers.73.mlp.experts.26.down_proj.weight": "model-00080-of-00101.safetensors", + "model.layers.73.mlp.experts.27.gate_proj.weight": "model-00080-of-00101.safetensors", + "model.layers.73.mlp.experts.27.up_proj.weight": "model-00080-of-00101.safetensors", + "model.layers.73.mlp.experts.27.down_proj.weight": "model-00080-of-00101.safetensors", + "model.layers.73.mlp.experts.28.gate_proj.weight": "model-00080-of-00101.safetensors", + "model.layers.73.mlp.experts.28.up_proj.weight": "model-00080-of-00101.safetensors", + "model.layers.73.mlp.experts.28.down_proj.weight": "model-00080-of-00101.safetensors", + "model.layers.73.mlp.experts.29.gate_proj.weight": "model-00080-of-00101.safetensors", + "model.layers.73.mlp.experts.29.up_proj.weight": "model-00080-of-00101.safetensors", + "model.layers.73.mlp.experts.29.down_proj.weight": "model-00080-of-00101.safetensors", + "model.layers.73.mlp.experts.30.gate_proj.weight": "model-00080-of-00101.safetensors", + "model.layers.73.mlp.experts.30.up_proj.weight": "model-00080-of-00101.safetensors", + "model.layers.73.mlp.experts.30.down_proj.weight": "model-00080-of-00101.safetensors", + "model.layers.73.mlp.experts.31.gate_proj.weight": "model-00080-of-00101.safetensors", + "model.layers.73.mlp.experts.31.up_proj.weight": "model-00080-of-00101.safetensors", + "model.layers.73.mlp.experts.31.down_proj.weight": "model-00080-of-00101.safetensors", + "model.layers.73.mlp.experts.32.gate_proj.weight": "model-00080-of-00101.safetensors", + "model.layers.73.mlp.experts.32.up_proj.weight": "model-00080-of-00101.safetensors", + "model.layers.73.mlp.experts.32.down_proj.weight": "model-00080-of-00101.safetensors", + "model.layers.73.mlp.experts.33.gate_proj.weight": "model-00080-of-00101.safetensors", + "model.layers.73.mlp.experts.33.up_proj.weight": "model-00080-of-00101.safetensors", + "model.layers.73.mlp.experts.33.down_proj.weight": "model-00080-of-00101.safetensors", + "model.layers.73.mlp.experts.34.gate_proj.weight": "model-00080-of-00101.safetensors", + "model.layers.73.mlp.experts.34.up_proj.weight": "model-00080-of-00101.safetensors", + "model.layers.73.mlp.experts.34.down_proj.weight": "model-00080-of-00101.safetensors", + "model.layers.73.mlp.experts.35.gate_proj.weight": "model-00080-of-00101.safetensors", + "model.layers.73.mlp.experts.35.up_proj.weight": "model-00080-of-00101.safetensors", + "model.layers.73.mlp.experts.35.down_proj.weight": "model-00080-of-00101.safetensors", + "model.layers.73.mlp.experts.36.gate_proj.weight": "model-00080-of-00101.safetensors", + "model.layers.73.mlp.experts.36.up_proj.weight": "model-00080-of-00101.safetensors", + "model.layers.73.mlp.experts.36.down_proj.weight": "model-00080-of-00101.safetensors", + "model.layers.73.mlp.experts.37.gate_proj.weight": "model-00080-of-00101.safetensors", + "model.layers.73.mlp.experts.37.up_proj.weight": "model-00080-of-00101.safetensors", + "model.layers.73.mlp.experts.37.down_proj.weight": "model-00080-of-00101.safetensors", + "model.layers.73.mlp.experts.38.gate_proj.weight": "model-00080-of-00101.safetensors", + "model.layers.73.mlp.experts.38.up_proj.weight": "model-00080-of-00101.safetensors", + "model.layers.73.mlp.experts.38.down_proj.weight": "model-00080-of-00101.safetensors", + "model.layers.73.mlp.experts.39.gate_proj.weight": "model-00080-of-00101.safetensors", + "model.layers.73.mlp.experts.39.up_proj.weight": "model-00080-of-00101.safetensors", + "model.layers.73.mlp.experts.39.down_proj.weight": "model-00080-of-00101.safetensors", + "model.layers.73.mlp.experts.40.gate_proj.weight": "model-00080-of-00101.safetensors", + "model.layers.73.mlp.experts.40.up_proj.weight": "model-00080-of-00101.safetensors", + "model.layers.73.mlp.experts.40.down_proj.weight": "model-00080-of-00101.safetensors", + "model.layers.73.mlp.experts.41.gate_proj.weight": "model-00080-of-00101.safetensors", + "model.layers.73.mlp.experts.41.up_proj.weight": "model-00080-of-00101.safetensors", + "model.layers.73.mlp.experts.41.down_proj.weight": "model-00080-of-00101.safetensors", + "model.layers.73.mlp.experts.42.gate_proj.weight": "model-00080-of-00101.safetensors", + "model.layers.73.mlp.experts.42.up_proj.weight": "model-00080-of-00101.safetensors", + "model.layers.73.mlp.experts.42.down_proj.weight": "model-00080-of-00101.safetensors", + "model.layers.73.mlp.experts.43.gate_proj.weight": "model-00080-of-00101.safetensors", + "model.layers.73.mlp.experts.43.up_proj.weight": "model-00080-of-00101.safetensors", + "model.layers.73.mlp.experts.43.down_proj.weight": "model-00080-of-00101.safetensors", + "model.layers.73.mlp.experts.44.gate_proj.weight": "model-00080-of-00101.safetensors", + "model.layers.73.mlp.experts.44.up_proj.weight": "model-00080-of-00101.safetensors", + "model.layers.73.mlp.experts.44.down_proj.weight": "model-00080-of-00101.safetensors", + "model.layers.73.mlp.experts.45.gate_proj.weight": "model-00080-of-00101.safetensors", + "model.layers.73.mlp.experts.45.up_proj.weight": "model-00080-of-00101.safetensors", + "model.layers.73.mlp.experts.45.down_proj.weight": "model-00080-of-00101.safetensors", + "model.layers.73.mlp.experts.46.gate_proj.weight": "model-00080-of-00101.safetensors", + "model.layers.73.mlp.experts.46.up_proj.weight": "model-00080-of-00101.safetensors", + "model.layers.73.mlp.experts.46.down_proj.weight": "model-00080-of-00101.safetensors", + "model.layers.73.mlp.experts.47.gate_proj.weight": "model-00080-of-00101.safetensors", + "model.layers.73.mlp.experts.47.up_proj.weight": "model-00080-of-00101.safetensors", + "model.layers.73.mlp.experts.47.down_proj.weight": "model-00080-of-00101.safetensors", + "model.layers.73.mlp.experts.48.gate_proj.weight": "model-00080-of-00101.safetensors", + "model.layers.73.mlp.experts.48.up_proj.weight": "model-00080-of-00101.safetensors", + "model.layers.73.mlp.experts.48.down_proj.weight": "model-00080-of-00101.safetensors", + "model.layers.73.mlp.experts.49.gate_proj.weight": "model-00080-of-00101.safetensors", + "model.layers.73.mlp.experts.49.up_proj.weight": "model-00080-of-00101.safetensors", + "model.layers.73.mlp.experts.49.down_proj.weight": "model-00080-of-00101.safetensors", + "model.layers.73.mlp.experts.50.gate_proj.weight": "model-00080-of-00101.safetensors", + "model.layers.73.mlp.experts.50.up_proj.weight": "model-00080-of-00101.safetensors", + "model.layers.73.mlp.experts.50.down_proj.weight": "model-00080-of-00101.safetensors", + "model.layers.73.mlp.experts.51.gate_proj.weight": "model-00080-of-00101.safetensors", + "model.layers.73.mlp.experts.51.up_proj.weight": "model-00080-of-00101.safetensors", + "model.layers.73.mlp.experts.51.down_proj.weight": "model-00080-of-00101.safetensors", + "model.layers.73.mlp.experts.52.gate_proj.weight": "model-00080-of-00101.safetensors", + "model.layers.73.mlp.experts.52.up_proj.weight": "model-00080-of-00101.safetensors", + "model.layers.73.mlp.experts.52.down_proj.weight": "model-00080-of-00101.safetensors", + "model.layers.73.mlp.experts.53.gate_proj.weight": "model-00080-of-00101.safetensors", + "model.layers.73.mlp.experts.53.up_proj.weight": "model-00080-of-00101.safetensors", + "model.layers.73.mlp.experts.53.down_proj.weight": "model-00080-of-00101.safetensors", + "model.layers.73.mlp.experts.54.gate_proj.weight": "model-00080-of-00101.safetensors", + "model.layers.73.mlp.experts.54.up_proj.weight": "model-00080-of-00101.safetensors", + "model.layers.73.mlp.experts.54.down_proj.weight": "model-00080-of-00101.safetensors", + "model.layers.73.mlp.experts.55.gate_proj.weight": "model-00080-of-00101.safetensors", + "model.layers.73.mlp.experts.55.up_proj.weight": "model-00080-of-00101.safetensors", + "model.layers.73.mlp.experts.55.down_proj.weight": "model-00080-of-00101.safetensors", + "model.layers.73.mlp.experts.56.gate_proj.weight": "model-00080-of-00101.safetensors", + "model.layers.73.mlp.experts.56.up_proj.weight": "model-00080-of-00101.safetensors", + "model.layers.73.mlp.experts.56.down_proj.weight": "model-00080-of-00101.safetensors", + "model.layers.73.mlp.experts.57.gate_proj.weight": "model-00080-of-00101.safetensors", + "model.layers.73.mlp.experts.57.up_proj.weight": "model-00080-of-00101.safetensors", + "model.layers.73.mlp.experts.57.down_proj.weight": "model-00080-of-00101.safetensors", + "model.layers.73.mlp.experts.58.gate_proj.weight": "model-00080-of-00101.safetensors", + "model.layers.73.mlp.experts.58.up_proj.weight": "model-00080-of-00101.safetensors", + "model.layers.73.mlp.experts.58.down_proj.weight": "model-00080-of-00101.safetensors", + "model.layers.73.mlp.experts.59.gate_proj.weight": "model-00080-of-00101.safetensors", + "model.layers.73.mlp.experts.59.up_proj.weight": "model-00080-of-00101.safetensors", + "model.layers.73.mlp.experts.59.down_proj.weight": "model-00080-of-00101.safetensors", + "model.layers.73.mlp.experts.60.gate_proj.weight": "model-00080-of-00101.safetensors", + "model.layers.73.mlp.experts.60.up_proj.weight": "model-00080-of-00101.safetensors", + "model.layers.73.mlp.experts.60.down_proj.weight": "model-00080-of-00101.safetensors", + "model.layers.73.mlp.experts.61.gate_proj.weight": "model-00080-of-00101.safetensors", + "model.layers.73.mlp.experts.61.up_proj.weight": "model-00080-of-00101.safetensors", + "model.layers.73.mlp.experts.61.down_proj.weight": "model-00080-of-00101.safetensors", + "model.layers.73.mlp.experts.62.gate_proj.weight": "model-00080-of-00101.safetensors", + "model.layers.73.mlp.experts.62.up_proj.weight": "model-00080-of-00101.safetensors", + "model.layers.73.mlp.experts.62.down_proj.weight": "model-00080-of-00101.safetensors", + "model.layers.73.mlp.experts.63.gate_proj.weight": "model-00080-of-00101.safetensors", + "model.layers.73.mlp.experts.63.up_proj.weight": "model-00080-of-00101.safetensors", + "model.layers.73.mlp.experts.63.down_proj.weight": "model-00080-of-00101.safetensors", + "model.layers.73.mlp.experts.64.gate_proj.weight": "model-00080-of-00101.safetensors", + "model.layers.73.mlp.experts.64.up_proj.weight": "model-00080-of-00101.safetensors", + "model.layers.73.mlp.experts.64.down_proj.weight": "model-00080-of-00101.safetensors", + "model.layers.73.mlp.experts.65.gate_proj.weight": "model-00080-of-00101.safetensors", + "model.layers.73.mlp.experts.65.up_proj.weight": "model-00080-of-00101.safetensors", + "model.layers.73.mlp.experts.65.down_proj.weight": "model-00080-of-00101.safetensors", + "model.layers.73.mlp.experts.66.gate_proj.weight": "model-00080-of-00101.safetensors", + "model.layers.73.mlp.experts.66.up_proj.weight": "model-00080-of-00101.safetensors", + "model.layers.73.mlp.experts.66.down_proj.weight": "model-00080-of-00101.safetensors", + "model.layers.73.mlp.experts.67.gate_proj.weight": "model-00080-of-00101.safetensors", + "model.layers.73.mlp.experts.67.up_proj.weight": "model-00080-of-00101.safetensors", + "model.layers.73.mlp.experts.67.down_proj.weight": "model-00080-of-00101.safetensors", + "model.layers.73.mlp.experts.68.gate_proj.weight": "model-00080-of-00101.safetensors", + "model.layers.73.mlp.experts.68.up_proj.weight": "model-00080-of-00101.safetensors", + "model.layers.73.mlp.experts.68.down_proj.weight": "model-00080-of-00101.safetensors", + "model.layers.73.mlp.experts.69.gate_proj.weight": "model-00080-of-00101.safetensors", + "model.layers.73.mlp.experts.69.up_proj.weight": "model-00080-of-00101.safetensors", + "model.layers.73.mlp.experts.69.down_proj.weight": "model-00080-of-00101.safetensors", + "model.layers.73.mlp.experts.70.gate_proj.weight": "model-00080-of-00101.safetensors", + "model.layers.73.mlp.experts.70.up_proj.weight": "model-00080-of-00101.safetensors", + "model.layers.73.mlp.experts.70.down_proj.weight": "model-00080-of-00101.safetensors", + "model.layers.73.mlp.experts.71.gate_proj.weight": "model-00080-of-00101.safetensors", + "model.layers.73.mlp.experts.71.up_proj.weight": "model-00080-of-00101.safetensors", + "model.layers.73.mlp.experts.71.down_proj.weight": "model-00080-of-00101.safetensors", + "model.layers.73.mlp.experts.72.gate_proj.weight": "model-00080-of-00101.safetensors", + "model.layers.73.mlp.experts.72.up_proj.weight": "model-00080-of-00101.safetensors", + "model.layers.73.mlp.experts.72.down_proj.weight": "model-00080-of-00101.safetensors", + "model.layers.73.mlp.experts.73.gate_proj.weight": "model-00080-of-00101.safetensors", + "model.layers.73.mlp.experts.73.up_proj.weight": "model-00080-of-00101.safetensors", + "model.layers.73.mlp.experts.73.down_proj.weight": "model-00080-of-00101.safetensors", + "model.layers.73.mlp.experts.74.gate_proj.weight": "model-00080-of-00101.safetensors", + "model.layers.73.mlp.experts.74.up_proj.weight": "model-00080-of-00101.safetensors", + "model.layers.73.mlp.experts.74.down_proj.weight": "model-00080-of-00101.safetensors", + "model.layers.73.mlp.experts.75.gate_proj.weight": "model-00080-of-00101.safetensors", + "model.layers.73.mlp.experts.75.up_proj.weight": "model-00080-of-00101.safetensors", + "model.layers.73.mlp.experts.75.down_proj.weight": "model-00080-of-00101.safetensors", + "model.layers.73.mlp.experts.76.gate_proj.weight": "model-00080-of-00101.safetensors", + "model.layers.73.mlp.experts.76.up_proj.weight": "model-00080-of-00101.safetensors", + "model.layers.73.mlp.experts.76.down_proj.weight": "model-00080-of-00101.safetensors", + "model.layers.73.mlp.experts.77.gate_proj.weight": "model-00080-of-00101.safetensors", + "model.layers.73.mlp.experts.77.up_proj.weight": "model-00080-of-00101.safetensors", + "model.layers.73.mlp.experts.77.down_proj.weight": "model-00080-of-00101.safetensors", + "model.layers.73.mlp.experts.78.gate_proj.weight": "model-00080-of-00101.safetensors", + "model.layers.73.mlp.experts.78.up_proj.weight": "model-00080-of-00101.safetensors", + "model.layers.73.mlp.experts.78.down_proj.weight": "model-00080-of-00101.safetensors", + "model.layers.73.mlp.experts.79.gate_proj.weight": "model-00080-of-00101.safetensors", + "model.layers.73.mlp.experts.79.up_proj.weight": "model-00080-of-00101.safetensors", + "model.layers.73.mlp.experts.79.down_proj.weight": "model-00080-of-00101.safetensors", + "model.layers.73.mlp.experts.80.gate_proj.weight": "model-00080-of-00101.safetensors", + "model.layers.73.mlp.experts.80.up_proj.weight": "model-00080-of-00101.safetensors", + "model.layers.73.mlp.experts.80.down_proj.weight": "model-00080-of-00101.safetensors", + "model.layers.73.mlp.experts.81.gate_proj.weight": "model-00080-of-00101.safetensors", + "model.layers.73.mlp.experts.81.up_proj.weight": "model-00080-of-00101.safetensors", + "model.layers.73.mlp.experts.81.down_proj.weight": "model-00080-of-00101.safetensors", + "model.layers.73.mlp.experts.82.gate_proj.weight": "model-00080-of-00101.safetensors", + "model.layers.73.mlp.experts.82.up_proj.weight": "model-00080-of-00101.safetensors", + "model.layers.73.mlp.experts.82.down_proj.weight": "model-00080-of-00101.safetensors", + "model.layers.73.mlp.experts.83.gate_proj.weight": "model-00080-of-00101.safetensors", + "model.layers.73.mlp.experts.83.up_proj.weight": "model-00080-of-00101.safetensors", + "model.layers.73.mlp.experts.83.down_proj.weight": "model-00080-of-00101.safetensors", + "model.layers.73.mlp.experts.84.gate_proj.weight": "model-00080-of-00101.safetensors", + "model.layers.73.mlp.experts.84.up_proj.weight": "model-00080-of-00101.safetensors", + "model.layers.73.mlp.experts.84.down_proj.weight": "model-00080-of-00101.safetensors", + "model.layers.73.mlp.experts.85.gate_proj.weight": "model-00080-of-00101.safetensors", + "model.layers.73.mlp.experts.85.up_proj.weight": "model-00080-of-00101.safetensors", + "model.layers.73.mlp.experts.85.down_proj.weight": "model-00080-of-00101.safetensors", + "model.layers.73.mlp.experts.86.gate_proj.weight": "model-00080-of-00101.safetensors", + "model.layers.73.mlp.experts.86.up_proj.weight": "model-00080-of-00101.safetensors", + "model.layers.73.mlp.experts.86.down_proj.weight": "model-00080-of-00101.safetensors", + "model.layers.73.mlp.experts.87.gate_proj.weight": "model-00080-of-00101.safetensors", + "model.layers.73.mlp.experts.87.up_proj.weight": "model-00080-of-00101.safetensors", + "model.layers.73.mlp.experts.87.down_proj.weight": "model-00080-of-00101.safetensors", + "model.layers.73.mlp.experts.88.gate_proj.weight": "model-00080-of-00101.safetensors", + "model.layers.73.mlp.experts.88.up_proj.weight": "model-00080-of-00101.safetensors", + "model.layers.73.mlp.experts.88.down_proj.weight": "model-00080-of-00101.safetensors", + "model.layers.73.mlp.experts.89.gate_proj.weight": "model-00080-of-00101.safetensors", + "model.layers.73.mlp.experts.89.up_proj.weight": "model-00080-of-00101.safetensors", + "model.layers.73.mlp.experts.89.down_proj.weight": "model-00080-of-00101.safetensors", + "model.layers.73.mlp.experts.90.gate_proj.weight": "model-00080-of-00101.safetensors", + "model.layers.73.mlp.experts.90.up_proj.weight": "model-00080-of-00101.safetensors", + "model.layers.73.mlp.experts.90.down_proj.weight": "model-00080-of-00101.safetensors", + "model.layers.73.mlp.experts.91.gate_proj.weight": "model-00080-of-00101.safetensors", + "model.layers.73.mlp.experts.91.up_proj.weight": "model-00080-of-00101.safetensors", + "model.layers.73.mlp.experts.91.down_proj.weight": "model-00080-of-00101.safetensors", + "model.layers.73.mlp.experts.92.gate_proj.weight": "model-00080-of-00101.safetensors", + "model.layers.73.mlp.experts.92.up_proj.weight": "model-00080-of-00101.safetensors", + "model.layers.73.mlp.experts.92.down_proj.weight": "model-00080-of-00101.safetensors", + "model.layers.73.mlp.experts.93.gate_proj.weight": "model-00080-of-00101.safetensors", + "model.layers.73.mlp.experts.93.up_proj.weight": "model-00080-of-00101.safetensors", + "model.layers.73.mlp.experts.93.down_proj.weight": "model-00080-of-00101.safetensors", + "model.layers.73.mlp.experts.94.gate_proj.weight": "model-00080-of-00101.safetensors", + "model.layers.73.mlp.experts.94.up_proj.weight": "model-00080-of-00101.safetensors", + "model.layers.73.mlp.experts.94.down_proj.weight": "model-00080-of-00101.safetensors", + "model.layers.73.mlp.experts.95.gate_proj.weight": "model-00080-of-00101.safetensors", + "model.layers.73.mlp.experts.95.up_proj.weight": "model-00080-of-00101.safetensors", + "model.layers.73.mlp.experts.95.down_proj.weight": "model-00080-of-00101.safetensors", + "model.layers.73.mlp.experts.96.gate_proj.weight": "model-00080-of-00101.safetensors", + "model.layers.73.mlp.experts.96.up_proj.weight": "model-00080-of-00101.safetensors", + "model.layers.73.mlp.experts.96.down_proj.weight": "model-00080-of-00101.safetensors", + "model.layers.73.mlp.experts.97.gate_proj.weight": "model-00080-of-00101.safetensors", + "model.layers.73.mlp.experts.97.up_proj.weight": "model-00080-of-00101.safetensors", + "model.layers.73.mlp.experts.97.down_proj.weight": "model-00080-of-00101.safetensors", + "model.layers.73.mlp.experts.98.gate_proj.weight": "model-00080-of-00101.safetensors", + "model.layers.73.mlp.experts.98.up_proj.weight": "model-00080-of-00101.safetensors", + "model.layers.73.mlp.experts.98.down_proj.weight": "model-00080-of-00101.safetensors", + "model.layers.73.mlp.experts.99.gate_proj.weight": "model-00080-of-00101.safetensors", + "model.layers.73.mlp.experts.99.up_proj.weight": "model-00080-of-00101.safetensors", + "model.layers.73.mlp.experts.99.down_proj.weight": "model-00080-of-00101.safetensors", + "model.layers.73.mlp.experts.100.gate_proj.weight": "model-00080-of-00101.safetensors", + "model.layers.73.mlp.experts.100.up_proj.weight": "model-00080-of-00101.safetensors", + "model.layers.73.mlp.experts.100.down_proj.weight": "model-00080-of-00101.safetensors", + "model.layers.73.mlp.experts.101.gate_proj.weight": "model-00080-of-00101.safetensors", + "model.layers.73.mlp.experts.101.up_proj.weight": "model-00080-of-00101.safetensors", + "model.layers.73.mlp.experts.101.down_proj.weight": "model-00080-of-00101.safetensors", + "model.layers.73.mlp.experts.102.gate_proj.weight": "model-00080-of-00101.safetensors", + "model.layers.73.mlp.experts.102.up_proj.weight": "model-00080-of-00101.safetensors", + "model.layers.73.mlp.experts.102.down_proj.weight": "model-00080-of-00101.safetensors", + "model.layers.73.mlp.experts.103.gate_proj.weight": "model-00080-of-00101.safetensors", + "model.layers.73.mlp.experts.103.up_proj.weight": "model-00080-of-00101.safetensors", + "model.layers.73.mlp.experts.103.down_proj.weight": "model-00080-of-00101.safetensors", + "model.layers.73.mlp.experts.104.gate_proj.weight": "model-00080-of-00101.safetensors", + "model.layers.73.mlp.experts.104.up_proj.weight": "model-00080-of-00101.safetensors", + "model.layers.73.mlp.experts.104.down_proj.weight": "model-00080-of-00101.safetensors", + "model.layers.73.mlp.experts.105.gate_proj.weight": "model-00080-of-00101.safetensors", + "model.layers.73.mlp.experts.105.up_proj.weight": "model-00080-of-00101.safetensors", + "model.layers.73.mlp.experts.105.down_proj.weight": "model-00080-of-00101.safetensors", + "model.layers.73.mlp.experts.106.gate_proj.weight": "model-00080-of-00101.safetensors", + "model.layers.73.mlp.experts.106.up_proj.weight": "model-00080-of-00101.safetensors", + "model.layers.73.mlp.experts.106.down_proj.weight": "model-00080-of-00101.safetensors", + "model.layers.73.mlp.experts.107.gate_proj.weight": "model-00080-of-00101.safetensors", + "model.layers.73.mlp.experts.107.up_proj.weight": "model-00080-of-00101.safetensors", + "model.layers.73.mlp.experts.107.down_proj.weight": "model-00080-of-00101.safetensors", + "model.layers.73.mlp.experts.108.gate_proj.weight": "model-00080-of-00101.safetensors", + "model.layers.73.mlp.experts.108.up_proj.weight": "model-00080-of-00101.safetensors", + "model.layers.73.mlp.experts.108.down_proj.weight": "model-00080-of-00101.safetensors", + "model.layers.73.mlp.experts.109.gate_proj.weight": "model-00080-of-00101.safetensors", + "model.layers.73.mlp.experts.109.up_proj.weight": "model-00080-of-00101.safetensors", + "model.layers.73.mlp.experts.109.down_proj.weight": "model-00080-of-00101.safetensors", + "model.layers.73.mlp.experts.110.gate_proj.weight": "model-00080-of-00101.safetensors", + "model.layers.73.mlp.experts.110.up_proj.weight": "model-00080-of-00101.safetensors", + "model.layers.73.mlp.experts.110.down_proj.weight": "model-00080-of-00101.safetensors", + "model.layers.73.mlp.experts.111.gate_proj.weight": "model-00080-of-00101.safetensors", + "model.layers.73.mlp.experts.111.up_proj.weight": "model-00080-of-00101.safetensors", + "model.layers.73.mlp.experts.111.down_proj.weight": "model-00080-of-00101.safetensors", + "model.layers.73.mlp.experts.112.gate_proj.weight": "model-00080-of-00101.safetensors", + "model.layers.73.mlp.experts.112.up_proj.weight": "model-00080-of-00101.safetensors", + "model.layers.73.mlp.experts.112.down_proj.weight": "model-00080-of-00101.safetensors", + "model.layers.73.mlp.experts.113.gate_proj.weight": "model-00080-of-00101.safetensors", + "model.layers.73.mlp.experts.113.up_proj.weight": "model-00080-of-00101.safetensors", + "model.layers.73.mlp.experts.113.down_proj.weight": "model-00080-of-00101.safetensors", + "model.layers.73.mlp.experts.114.gate_proj.weight": "model-00080-of-00101.safetensors", + "model.layers.73.mlp.experts.114.up_proj.weight": "model-00080-of-00101.safetensors", + "model.layers.73.mlp.experts.114.down_proj.weight": "model-00080-of-00101.safetensors", + "model.layers.73.mlp.experts.115.gate_proj.weight": "model-00080-of-00101.safetensors", + "model.layers.73.mlp.experts.115.up_proj.weight": "model-00080-of-00101.safetensors", + "model.layers.73.mlp.experts.115.down_proj.weight": "model-00080-of-00101.safetensors", + "model.layers.73.mlp.experts.116.gate_proj.weight": "model-00080-of-00101.safetensors", + "model.layers.73.mlp.experts.116.up_proj.weight": "model-00080-of-00101.safetensors", + "model.layers.73.mlp.experts.116.down_proj.weight": "model-00080-of-00101.safetensors", + "model.layers.73.mlp.experts.117.gate_proj.weight": "model-00080-of-00101.safetensors", + "model.layers.73.mlp.experts.117.up_proj.weight": "model-00080-of-00101.safetensors", + "model.layers.73.mlp.experts.117.down_proj.weight": "model-00080-of-00101.safetensors", + "model.layers.73.mlp.experts.118.gate_proj.weight": "model-00080-of-00101.safetensors", + "model.layers.73.mlp.experts.118.up_proj.weight": "model-00080-of-00101.safetensors", + "model.layers.73.mlp.experts.118.down_proj.weight": "model-00080-of-00101.safetensors", + "model.layers.73.mlp.experts.119.gate_proj.weight": "model-00080-of-00101.safetensors", + "model.layers.73.mlp.experts.119.up_proj.weight": "model-00080-of-00101.safetensors", + "model.layers.73.mlp.experts.119.down_proj.weight": "model-00080-of-00101.safetensors", + "model.layers.73.mlp.gate.weight": "model-00080-of-00101.safetensors", + "model.layers.73.mlp.gate.e_score_correction_bias": "model-00080-of-00101.safetensors", + "model.layers.73.mlp.shared_experts.gate_proj.weight": "model-00080-of-00101.safetensors", + "model.layers.73.mlp.shared_experts.up_proj.weight": "model-00080-of-00101.safetensors", + "model.layers.73.mlp.shared_experts.down_proj.weight": "model-00080-of-00101.safetensors", + "model.layers.73.input_layernorm.weight": "model-00080-of-00101.safetensors", + "model.layers.73.post_attention_layernorm.weight": "model-00080-of-00101.safetensors", + "model.layers.74.self_attn.q_proj.weight": "model-00081-of-00101.safetensors", + "model.layers.74.self_attn.q_proj.bias": "model-00081-of-00101.safetensors", + "model.layers.74.self_attn.k_proj.weight": "model-00081-of-00101.safetensors", + "model.layers.74.self_attn.k_proj.bias": "model-00081-of-00101.safetensors", + "model.layers.74.self_attn.v_proj.weight": "model-00081-of-00101.safetensors", + "model.layers.74.self_attn.v_proj.bias": "model-00081-of-00101.safetensors", + "model.layers.74.self_attn.o_proj.weight": "model-00081-of-00101.safetensors", + "model.layers.74.self_attn.q_norm.weight": "model-00081-of-00101.safetensors", + "model.layers.74.self_attn.k_norm.weight": "model-00081-of-00101.safetensors", + "model.layers.74.mlp.experts.0.gate_proj.weight": "model-00081-of-00101.safetensors", + "model.layers.74.mlp.experts.0.up_proj.weight": "model-00081-of-00101.safetensors", + "model.layers.74.mlp.experts.0.down_proj.weight": "model-00081-of-00101.safetensors", + "model.layers.74.mlp.experts.1.gate_proj.weight": "model-00081-of-00101.safetensors", + "model.layers.74.mlp.experts.1.up_proj.weight": "model-00081-of-00101.safetensors", + "model.layers.74.mlp.experts.1.down_proj.weight": "model-00081-of-00101.safetensors", + "model.layers.74.mlp.experts.2.gate_proj.weight": "model-00081-of-00101.safetensors", + "model.layers.74.mlp.experts.2.up_proj.weight": "model-00081-of-00101.safetensors", + "model.layers.74.mlp.experts.2.down_proj.weight": "model-00081-of-00101.safetensors", + "model.layers.74.mlp.experts.3.gate_proj.weight": "model-00081-of-00101.safetensors", + "model.layers.74.mlp.experts.3.up_proj.weight": "model-00081-of-00101.safetensors", + "model.layers.74.mlp.experts.3.down_proj.weight": "model-00081-of-00101.safetensors", + "model.layers.74.mlp.experts.4.gate_proj.weight": "model-00081-of-00101.safetensors", + "model.layers.74.mlp.experts.4.up_proj.weight": "model-00081-of-00101.safetensors", + "model.layers.74.mlp.experts.4.down_proj.weight": "model-00081-of-00101.safetensors", + "model.layers.74.mlp.experts.5.gate_proj.weight": "model-00081-of-00101.safetensors", + "model.layers.74.mlp.experts.5.up_proj.weight": "model-00081-of-00101.safetensors", + "model.layers.74.mlp.experts.5.down_proj.weight": "model-00081-of-00101.safetensors", + "model.layers.74.mlp.experts.6.gate_proj.weight": "model-00081-of-00101.safetensors", + "model.layers.74.mlp.experts.6.up_proj.weight": "model-00081-of-00101.safetensors", + "model.layers.74.mlp.experts.6.down_proj.weight": "model-00081-of-00101.safetensors", + "model.layers.74.mlp.experts.7.gate_proj.weight": "model-00081-of-00101.safetensors", + "model.layers.74.mlp.experts.7.up_proj.weight": "model-00081-of-00101.safetensors", + "model.layers.74.mlp.experts.7.down_proj.weight": "model-00081-of-00101.safetensors", + "model.layers.74.mlp.experts.8.gate_proj.weight": "model-00081-of-00101.safetensors", + "model.layers.74.mlp.experts.8.up_proj.weight": "model-00081-of-00101.safetensors", + "model.layers.74.mlp.experts.8.down_proj.weight": "model-00081-of-00101.safetensors", + "model.layers.74.mlp.experts.9.gate_proj.weight": "model-00081-of-00101.safetensors", + "model.layers.74.mlp.experts.9.up_proj.weight": "model-00081-of-00101.safetensors", + "model.layers.74.mlp.experts.9.down_proj.weight": "model-00081-of-00101.safetensors", + "model.layers.74.mlp.experts.10.gate_proj.weight": "model-00081-of-00101.safetensors", + "model.layers.74.mlp.experts.10.up_proj.weight": "model-00081-of-00101.safetensors", + "model.layers.74.mlp.experts.10.down_proj.weight": "model-00081-of-00101.safetensors", + "model.layers.74.mlp.experts.11.gate_proj.weight": "model-00081-of-00101.safetensors", + "model.layers.74.mlp.experts.11.up_proj.weight": "model-00081-of-00101.safetensors", + "model.layers.74.mlp.experts.11.down_proj.weight": "model-00081-of-00101.safetensors", + "model.layers.74.mlp.experts.12.gate_proj.weight": "model-00081-of-00101.safetensors", + "model.layers.74.mlp.experts.12.up_proj.weight": "model-00081-of-00101.safetensors", + "model.layers.74.mlp.experts.12.down_proj.weight": "model-00081-of-00101.safetensors", + "model.layers.74.mlp.experts.13.gate_proj.weight": "model-00081-of-00101.safetensors", + "model.layers.74.mlp.experts.13.up_proj.weight": "model-00081-of-00101.safetensors", + "model.layers.74.mlp.experts.13.down_proj.weight": "model-00081-of-00101.safetensors", + "model.layers.74.mlp.experts.14.gate_proj.weight": "model-00081-of-00101.safetensors", + "model.layers.74.mlp.experts.14.up_proj.weight": "model-00081-of-00101.safetensors", + "model.layers.74.mlp.experts.14.down_proj.weight": "model-00081-of-00101.safetensors", + "model.layers.74.mlp.experts.15.gate_proj.weight": "model-00081-of-00101.safetensors", + "model.layers.74.mlp.experts.15.up_proj.weight": "model-00081-of-00101.safetensors", + "model.layers.74.mlp.experts.15.down_proj.weight": "model-00081-of-00101.safetensors", + "model.layers.74.mlp.experts.16.gate_proj.weight": "model-00081-of-00101.safetensors", + "model.layers.74.mlp.experts.16.up_proj.weight": "model-00081-of-00101.safetensors", + "model.layers.74.mlp.experts.16.down_proj.weight": "model-00081-of-00101.safetensors", + "model.layers.74.mlp.experts.17.gate_proj.weight": "model-00081-of-00101.safetensors", + "model.layers.74.mlp.experts.17.up_proj.weight": "model-00081-of-00101.safetensors", + "model.layers.74.mlp.experts.17.down_proj.weight": "model-00081-of-00101.safetensors", + "model.layers.74.mlp.experts.18.gate_proj.weight": "model-00081-of-00101.safetensors", + "model.layers.74.mlp.experts.18.up_proj.weight": "model-00081-of-00101.safetensors", + "model.layers.74.mlp.experts.18.down_proj.weight": "model-00081-of-00101.safetensors", + "model.layers.74.mlp.experts.19.gate_proj.weight": "model-00081-of-00101.safetensors", + "model.layers.74.mlp.experts.19.up_proj.weight": "model-00081-of-00101.safetensors", + "model.layers.74.mlp.experts.19.down_proj.weight": "model-00081-of-00101.safetensors", + "model.layers.74.mlp.experts.20.gate_proj.weight": "model-00081-of-00101.safetensors", + "model.layers.74.mlp.experts.20.up_proj.weight": "model-00081-of-00101.safetensors", + "model.layers.74.mlp.experts.20.down_proj.weight": "model-00081-of-00101.safetensors", + "model.layers.74.mlp.experts.21.gate_proj.weight": "model-00081-of-00101.safetensors", + "model.layers.74.mlp.experts.21.up_proj.weight": "model-00081-of-00101.safetensors", + "model.layers.74.mlp.experts.21.down_proj.weight": "model-00081-of-00101.safetensors", + "model.layers.74.mlp.experts.22.gate_proj.weight": "model-00081-of-00101.safetensors", + "model.layers.74.mlp.experts.22.up_proj.weight": "model-00081-of-00101.safetensors", + "model.layers.74.mlp.experts.22.down_proj.weight": "model-00081-of-00101.safetensors", + "model.layers.74.mlp.experts.23.gate_proj.weight": "model-00081-of-00101.safetensors", + "model.layers.74.mlp.experts.23.up_proj.weight": "model-00081-of-00101.safetensors", + "model.layers.74.mlp.experts.23.down_proj.weight": "model-00081-of-00101.safetensors", + "model.layers.74.mlp.experts.24.gate_proj.weight": "model-00081-of-00101.safetensors", + "model.layers.74.mlp.experts.24.up_proj.weight": "model-00081-of-00101.safetensors", + "model.layers.74.mlp.experts.24.down_proj.weight": "model-00081-of-00101.safetensors", + "model.layers.74.mlp.experts.25.gate_proj.weight": "model-00081-of-00101.safetensors", + "model.layers.74.mlp.experts.25.up_proj.weight": "model-00081-of-00101.safetensors", + "model.layers.74.mlp.experts.25.down_proj.weight": "model-00081-of-00101.safetensors", + "model.layers.74.mlp.experts.26.gate_proj.weight": "model-00081-of-00101.safetensors", + "model.layers.74.mlp.experts.26.up_proj.weight": "model-00081-of-00101.safetensors", + "model.layers.74.mlp.experts.26.down_proj.weight": "model-00081-of-00101.safetensors", + "model.layers.74.mlp.experts.27.gate_proj.weight": "model-00081-of-00101.safetensors", + "model.layers.74.mlp.experts.27.up_proj.weight": "model-00081-of-00101.safetensors", + "model.layers.74.mlp.experts.27.down_proj.weight": "model-00081-of-00101.safetensors", + "model.layers.74.mlp.experts.28.gate_proj.weight": "model-00081-of-00101.safetensors", + "model.layers.74.mlp.experts.28.up_proj.weight": "model-00081-of-00101.safetensors", + "model.layers.74.mlp.experts.28.down_proj.weight": "model-00081-of-00101.safetensors", + "model.layers.74.mlp.experts.29.gate_proj.weight": "model-00081-of-00101.safetensors", + "model.layers.74.mlp.experts.29.up_proj.weight": "model-00081-of-00101.safetensors", + "model.layers.74.mlp.experts.29.down_proj.weight": "model-00081-of-00101.safetensors", + "model.layers.74.mlp.experts.30.gate_proj.weight": "model-00081-of-00101.safetensors", + "model.layers.74.mlp.experts.30.up_proj.weight": "model-00081-of-00101.safetensors", + "model.layers.74.mlp.experts.30.down_proj.weight": "model-00081-of-00101.safetensors", + "model.layers.74.mlp.experts.31.gate_proj.weight": "model-00081-of-00101.safetensors", + "model.layers.74.mlp.experts.31.up_proj.weight": "model-00081-of-00101.safetensors", + "model.layers.74.mlp.experts.31.down_proj.weight": "model-00081-of-00101.safetensors", + "model.layers.74.mlp.experts.32.gate_proj.weight": "model-00081-of-00101.safetensors", + "model.layers.74.mlp.experts.32.up_proj.weight": "model-00081-of-00101.safetensors", + "model.layers.74.mlp.experts.32.down_proj.weight": "model-00081-of-00101.safetensors", + "model.layers.74.mlp.experts.33.gate_proj.weight": "model-00081-of-00101.safetensors", + "model.layers.74.mlp.experts.33.up_proj.weight": "model-00081-of-00101.safetensors", + "model.layers.74.mlp.experts.33.down_proj.weight": "model-00081-of-00101.safetensors", + "model.layers.74.mlp.experts.34.gate_proj.weight": "model-00081-of-00101.safetensors", + "model.layers.74.mlp.experts.34.up_proj.weight": "model-00081-of-00101.safetensors", + "model.layers.74.mlp.experts.34.down_proj.weight": "model-00081-of-00101.safetensors", + "model.layers.74.mlp.experts.35.gate_proj.weight": "model-00081-of-00101.safetensors", + "model.layers.74.mlp.experts.35.up_proj.weight": "model-00081-of-00101.safetensors", + "model.layers.74.mlp.experts.35.down_proj.weight": "model-00081-of-00101.safetensors", + "model.layers.74.mlp.experts.36.gate_proj.weight": "model-00081-of-00101.safetensors", + "model.layers.74.mlp.experts.36.up_proj.weight": "model-00081-of-00101.safetensors", + "model.layers.74.mlp.experts.36.down_proj.weight": "model-00081-of-00101.safetensors", + "model.layers.74.mlp.experts.37.gate_proj.weight": "model-00081-of-00101.safetensors", + "model.layers.74.mlp.experts.37.up_proj.weight": "model-00081-of-00101.safetensors", + "model.layers.74.mlp.experts.37.down_proj.weight": "model-00081-of-00101.safetensors", + "model.layers.74.mlp.experts.38.gate_proj.weight": "model-00081-of-00101.safetensors", + "model.layers.74.mlp.experts.38.up_proj.weight": "model-00081-of-00101.safetensors", + "model.layers.74.mlp.experts.38.down_proj.weight": "model-00081-of-00101.safetensors", + "model.layers.74.mlp.experts.39.gate_proj.weight": "model-00081-of-00101.safetensors", + "model.layers.74.mlp.experts.39.up_proj.weight": "model-00081-of-00101.safetensors", + "model.layers.74.mlp.experts.39.down_proj.weight": "model-00081-of-00101.safetensors", + "model.layers.74.mlp.experts.40.gate_proj.weight": "model-00081-of-00101.safetensors", + "model.layers.74.mlp.experts.40.up_proj.weight": "model-00081-of-00101.safetensors", + "model.layers.74.mlp.experts.40.down_proj.weight": "model-00081-of-00101.safetensors", + "model.layers.74.mlp.experts.41.gate_proj.weight": "model-00081-of-00101.safetensors", + "model.layers.74.mlp.experts.41.up_proj.weight": "model-00081-of-00101.safetensors", + "model.layers.74.mlp.experts.41.down_proj.weight": "model-00081-of-00101.safetensors", + "model.layers.74.mlp.experts.42.gate_proj.weight": "model-00081-of-00101.safetensors", + "model.layers.74.mlp.experts.42.up_proj.weight": "model-00081-of-00101.safetensors", + "model.layers.74.mlp.experts.42.down_proj.weight": "model-00081-of-00101.safetensors", + "model.layers.74.mlp.experts.43.gate_proj.weight": "model-00081-of-00101.safetensors", + "model.layers.74.mlp.experts.43.up_proj.weight": "model-00081-of-00101.safetensors", + "model.layers.74.mlp.experts.43.down_proj.weight": "model-00081-of-00101.safetensors", + "model.layers.74.mlp.experts.44.gate_proj.weight": "model-00081-of-00101.safetensors", + "model.layers.74.mlp.experts.44.up_proj.weight": "model-00081-of-00101.safetensors", + "model.layers.74.mlp.experts.44.down_proj.weight": "model-00081-of-00101.safetensors", + "model.layers.74.mlp.experts.45.gate_proj.weight": "model-00081-of-00101.safetensors", + "model.layers.74.mlp.experts.45.up_proj.weight": "model-00081-of-00101.safetensors", + "model.layers.74.mlp.experts.45.down_proj.weight": "model-00081-of-00101.safetensors", + "model.layers.74.mlp.experts.46.gate_proj.weight": "model-00081-of-00101.safetensors", + "model.layers.74.mlp.experts.46.up_proj.weight": "model-00081-of-00101.safetensors", + "model.layers.74.mlp.experts.46.down_proj.weight": "model-00081-of-00101.safetensors", + "model.layers.74.mlp.experts.47.gate_proj.weight": "model-00081-of-00101.safetensors", + "model.layers.74.mlp.experts.47.up_proj.weight": "model-00081-of-00101.safetensors", + "model.layers.74.mlp.experts.47.down_proj.weight": "model-00081-of-00101.safetensors", + "model.layers.74.mlp.experts.48.gate_proj.weight": "model-00081-of-00101.safetensors", + "model.layers.74.mlp.experts.48.up_proj.weight": "model-00081-of-00101.safetensors", + "model.layers.74.mlp.experts.48.down_proj.weight": "model-00081-of-00101.safetensors", + "model.layers.74.mlp.experts.49.gate_proj.weight": "model-00081-of-00101.safetensors", + "model.layers.74.mlp.experts.49.up_proj.weight": "model-00081-of-00101.safetensors", + "model.layers.74.mlp.experts.49.down_proj.weight": "model-00081-of-00101.safetensors", + "model.layers.74.mlp.experts.50.gate_proj.weight": "model-00081-of-00101.safetensors", + "model.layers.74.mlp.experts.50.up_proj.weight": "model-00081-of-00101.safetensors", + "model.layers.74.mlp.experts.50.down_proj.weight": "model-00081-of-00101.safetensors", + "model.layers.74.mlp.experts.51.gate_proj.weight": "model-00081-of-00101.safetensors", + "model.layers.74.mlp.experts.51.up_proj.weight": "model-00081-of-00101.safetensors", + "model.layers.74.mlp.experts.51.down_proj.weight": "model-00081-of-00101.safetensors", + "model.layers.74.mlp.experts.52.gate_proj.weight": "model-00081-of-00101.safetensors", + "model.layers.74.mlp.experts.52.up_proj.weight": "model-00081-of-00101.safetensors", + "model.layers.74.mlp.experts.52.down_proj.weight": "model-00081-of-00101.safetensors", + "model.layers.74.mlp.experts.53.gate_proj.weight": "model-00081-of-00101.safetensors", + "model.layers.74.mlp.experts.53.up_proj.weight": "model-00081-of-00101.safetensors", + "model.layers.74.mlp.experts.53.down_proj.weight": "model-00081-of-00101.safetensors", + "model.layers.74.mlp.experts.54.gate_proj.weight": "model-00081-of-00101.safetensors", + "model.layers.74.mlp.experts.54.up_proj.weight": "model-00081-of-00101.safetensors", + "model.layers.74.mlp.experts.54.down_proj.weight": "model-00081-of-00101.safetensors", + "model.layers.74.mlp.experts.55.gate_proj.weight": "model-00081-of-00101.safetensors", + "model.layers.74.mlp.experts.55.up_proj.weight": "model-00081-of-00101.safetensors", + "model.layers.74.mlp.experts.55.down_proj.weight": "model-00081-of-00101.safetensors", + "model.layers.74.mlp.experts.56.gate_proj.weight": "model-00081-of-00101.safetensors", + "model.layers.74.mlp.experts.56.up_proj.weight": "model-00081-of-00101.safetensors", + "model.layers.74.mlp.experts.56.down_proj.weight": "model-00081-of-00101.safetensors", + "model.layers.74.mlp.experts.57.gate_proj.weight": "model-00081-of-00101.safetensors", + "model.layers.74.mlp.experts.57.up_proj.weight": "model-00081-of-00101.safetensors", + "model.layers.74.mlp.experts.57.down_proj.weight": "model-00081-of-00101.safetensors", + "model.layers.74.mlp.experts.58.gate_proj.weight": "model-00081-of-00101.safetensors", + "model.layers.74.mlp.experts.58.up_proj.weight": "model-00081-of-00101.safetensors", + "model.layers.74.mlp.experts.58.down_proj.weight": "model-00081-of-00101.safetensors", + "model.layers.74.mlp.experts.59.gate_proj.weight": "model-00081-of-00101.safetensors", + "model.layers.74.mlp.experts.59.up_proj.weight": "model-00081-of-00101.safetensors", + "model.layers.74.mlp.experts.59.down_proj.weight": "model-00081-of-00101.safetensors", + "model.layers.74.mlp.experts.60.gate_proj.weight": "model-00081-of-00101.safetensors", + "model.layers.74.mlp.experts.60.up_proj.weight": "model-00081-of-00101.safetensors", + "model.layers.74.mlp.experts.60.down_proj.weight": "model-00081-of-00101.safetensors", + "model.layers.74.mlp.experts.61.gate_proj.weight": "model-00081-of-00101.safetensors", + "model.layers.74.mlp.experts.61.up_proj.weight": "model-00081-of-00101.safetensors", + "model.layers.74.mlp.experts.61.down_proj.weight": "model-00081-of-00101.safetensors", + "model.layers.74.mlp.experts.62.gate_proj.weight": "model-00081-of-00101.safetensors", + "model.layers.74.mlp.experts.62.up_proj.weight": "model-00081-of-00101.safetensors", + "model.layers.74.mlp.experts.62.down_proj.weight": "model-00081-of-00101.safetensors", + "model.layers.74.mlp.experts.63.gate_proj.weight": "model-00081-of-00101.safetensors", + "model.layers.74.mlp.experts.63.up_proj.weight": "model-00081-of-00101.safetensors", + "model.layers.74.mlp.experts.63.down_proj.weight": "model-00081-of-00101.safetensors", + "model.layers.74.mlp.experts.64.gate_proj.weight": "model-00081-of-00101.safetensors", + "model.layers.74.mlp.experts.64.up_proj.weight": "model-00081-of-00101.safetensors", + "model.layers.74.mlp.experts.64.down_proj.weight": "model-00081-of-00101.safetensors", + "model.layers.74.mlp.experts.65.gate_proj.weight": "model-00081-of-00101.safetensors", + "model.layers.74.mlp.experts.65.up_proj.weight": "model-00081-of-00101.safetensors", + "model.layers.74.mlp.experts.65.down_proj.weight": "model-00081-of-00101.safetensors", + "model.layers.74.mlp.experts.66.gate_proj.weight": "model-00081-of-00101.safetensors", + "model.layers.74.mlp.experts.66.up_proj.weight": "model-00081-of-00101.safetensors", + "model.layers.74.mlp.experts.66.down_proj.weight": "model-00081-of-00101.safetensors", + "model.layers.74.mlp.experts.67.gate_proj.weight": "model-00081-of-00101.safetensors", + "model.layers.74.mlp.experts.67.up_proj.weight": "model-00081-of-00101.safetensors", + "model.layers.74.mlp.experts.67.down_proj.weight": "model-00081-of-00101.safetensors", + "model.layers.74.mlp.experts.68.gate_proj.weight": "model-00081-of-00101.safetensors", + "model.layers.74.mlp.experts.68.up_proj.weight": "model-00081-of-00101.safetensors", + "model.layers.74.mlp.experts.68.down_proj.weight": "model-00081-of-00101.safetensors", + "model.layers.74.mlp.experts.69.gate_proj.weight": "model-00081-of-00101.safetensors", + "model.layers.74.mlp.experts.69.up_proj.weight": "model-00081-of-00101.safetensors", + "model.layers.74.mlp.experts.69.down_proj.weight": "model-00081-of-00101.safetensors", + "model.layers.74.mlp.experts.70.gate_proj.weight": "model-00081-of-00101.safetensors", + "model.layers.74.mlp.experts.70.up_proj.weight": "model-00081-of-00101.safetensors", + "model.layers.74.mlp.experts.70.down_proj.weight": "model-00081-of-00101.safetensors", + "model.layers.74.mlp.experts.71.gate_proj.weight": "model-00081-of-00101.safetensors", + "model.layers.74.mlp.experts.71.up_proj.weight": "model-00081-of-00101.safetensors", + "model.layers.74.mlp.experts.71.down_proj.weight": "model-00081-of-00101.safetensors", + "model.layers.74.mlp.experts.72.gate_proj.weight": "model-00081-of-00101.safetensors", + "model.layers.74.mlp.experts.72.up_proj.weight": "model-00081-of-00101.safetensors", + "model.layers.74.mlp.experts.72.down_proj.weight": "model-00081-of-00101.safetensors", + "model.layers.74.mlp.experts.73.gate_proj.weight": "model-00081-of-00101.safetensors", + "model.layers.74.mlp.experts.73.up_proj.weight": "model-00081-of-00101.safetensors", + "model.layers.74.mlp.experts.73.down_proj.weight": "model-00081-of-00101.safetensors", + "model.layers.74.mlp.experts.74.gate_proj.weight": "model-00081-of-00101.safetensors", + "model.layers.74.mlp.experts.74.up_proj.weight": "model-00081-of-00101.safetensors", + "model.layers.74.mlp.experts.74.down_proj.weight": "model-00081-of-00101.safetensors", + "model.layers.74.mlp.experts.75.gate_proj.weight": "model-00081-of-00101.safetensors", + "model.layers.74.mlp.experts.75.up_proj.weight": "model-00081-of-00101.safetensors", + "model.layers.74.mlp.experts.75.down_proj.weight": "model-00081-of-00101.safetensors", + "model.layers.74.mlp.experts.76.gate_proj.weight": "model-00081-of-00101.safetensors", + "model.layers.74.mlp.experts.76.up_proj.weight": "model-00081-of-00101.safetensors", + "model.layers.74.mlp.experts.76.down_proj.weight": "model-00081-of-00101.safetensors", + "model.layers.74.mlp.experts.77.gate_proj.weight": "model-00081-of-00101.safetensors", + "model.layers.74.mlp.experts.77.up_proj.weight": "model-00081-of-00101.safetensors", + "model.layers.74.mlp.experts.77.down_proj.weight": "model-00081-of-00101.safetensors", + "model.layers.74.mlp.experts.78.gate_proj.weight": "model-00081-of-00101.safetensors", + "model.layers.74.mlp.experts.78.up_proj.weight": "model-00081-of-00101.safetensors", + "model.layers.74.mlp.experts.78.down_proj.weight": "model-00081-of-00101.safetensors", + "model.layers.74.mlp.experts.79.gate_proj.weight": "model-00081-of-00101.safetensors", + "model.layers.74.mlp.experts.79.up_proj.weight": "model-00081-of-00101.safetensors", + "model.layers.74.mlp.experts.79.down_proj.weight": "model-00081-of-00101.safetensors", + "model.layers.74.mlp.experts.80.gate_proj.weight": "model-00081-of-00101.safetensors", + "model.layers.74.mlp.experts.80.up_proj.weight": "model-00081-of-00101.safetensors", + "model.layers.74.mlp.experts.80.down_proj.weight": "model-00081-of-00101.safetensors", + "model.layers.74.mlp.experts.81.gate_proj.weight": "model-00081-of-00101.safetensors", + "model.layers.74.mlp.experts.81.up_proj.weight": "model-00081-of-00101.safetensors", + "model.layers.74.mlp.experts.81.down_proj.weight": "model-00081-of-00101.safetensors", + "model.layers.74.mlp.experts.82.gate_proj.weight": "model-00081-of-00101.safetensors", + "model.layers.74.mlp.experts.82.up_proj.weight": "model-00081-of-00101.safetensors", + "model.layers.74.mlp.experts.82.down_proj.weight": "model-00081-of-00101.safetensors", + "model.layers.74.mlp.experts.83.gate_proj.weight": "model-00081-of-00101.safetensors", + "model.layers.74.mlp.experts.83.up_proj.weight": "model-00081-of-00101.safetensors", + "model.layers.74.mlp.experts.83.down_proj.weight": "model-00081-of-00101.safetensors", + "model.layers.74.mlp.experts.84.gate_proj.weight": "model-00081-of-00101.safetensors", + "model.layers.74.mlp.experts.84.up_proj.weight": "model-00081-of-00101.safetensors", + "model.layers.74.mlp.experts.84.down_proj.weight": "model-00081-of-00101.safetensors", + "model.layers.74.mlp.experts.85.gate_proj.weight": "model-00081-of-00101.safetensors", + "model.layers.74.mlp.experts.85.up_proj.weight": "model-00081-of-00101.safetensors", + "model.layers.74.mlp.experts.85.down_proj.weight": "model-00081-of-00101.safetensors", + "model.layers.74.mlp.experts.86.gate_proj.weight": "model-00081-of-00101.safetensors", + "model.layers.74.mlp.experts.86.up_proj.weight": "model-00081-of-00101.safetensors", + "model.layers.74.mlp.experts.86.down_proj.weight": "model-00081-of-00101.safetensors", + "model.layers.74.mlp.experts.87.gate_proj.weight": "model-00081-of-00101.safetensors", + "model.layers.74.mlp.experts.87.up_proj.weight": "model-00081-of-00101.safetensors", + "model.layers.74.mlp.experts.87.down_proj.weight": "model-00081-of-00101.safetensors", + "model.layers.74.mlp.experts.88.gate_proj.weight": "model-00081-of-00101.safetensors", + "model.layers.74.mlp.experts.88.up_proj.weight": "model-00081-of-00101.safetensors", + "model.layers.74.mlp.experts.88.down_proj.weight": "model-00081-of-00101.safetensors", + "model.layers.74.mlp.experts.89.gate_proj.weight": "model-00081-of-00101.safetensors", + "model.layers.74.mlp.experts.89.up_proj.weight": "model-00081-of-00101.safetensors", + "model.layers.74.mlp.experts.89.down_proj.weight": "model-00081-of-00101.safetensors", + "model.layers.74.mlp.experts.90.gate_proj.weight": "model-00081-of-00101.safetensors", + "model.layers.74.mlp.experts.90.up_proj.weight": "model-00081-of-00101.safetensors", + "model.layers.74.mlp.experts.90.down_proj.weight": "model-00081-of-00101.safetensors", + "model.layers.74.mlp.experts.91.gate_proj.weight": "model-00081-of-00101.safetensors", + "model.layers.74.mlp.experts.91.up_proj.weight": "model-00081-of-00101.safetensors", + "model.layers.74.mlp.experts.91.down_proj.weight": "model-00081-of-00101.safetensors", + "model.layers.74.mlp.experts.92.gate_proj.weight": "model-00081-of-00101.safetensors", + "model.layers.74.mlp.experts.92.up_proj.weight": "model-00081-of-00101.safetensors", + "model.layers.74.mlp.experts.92.down_proj.weight": "model-00081-of-00101.safetensors", + "model.layers.74.mlp.experts.93.gate_proj.weight": "model-00081-of-00101.safetensors", + "model.layers.74.mlp.experts.93.up_proj.weight": "model-00081-of-00101.safetensors", + "model.layers.74.mlp.experts.93.down_proj.weight": "model-00081-of-00101.safetensors", + "model.layers.74.mlp.experts.94.gate_proj.weight": "model-00081-of-00101.safetensors", + "model.layers.74.mlp.experts.94.up_proj.weight": "model-00081-of-00101.safetensors", + "model.layers.74.mlp.experts.94.down_proj.weight": "model-00081-of-00101.safetensors", + "model.layers.74.mlp.experts.95.gate_proj.weight": "model-00081-of-00101.safetensors", + "model.layers.74.mlp.experts.95.up_proj.weight": "model-00081-of-00101.safetensors", + "model.layers.74.mlp.experts.95.down_proj.weight": "model-00081-of-00101.safetensors", + "model.layers.74.mlp.experts.96.gate_proj.weight": "model-00081-of-00101.safetensors", + "model.layers.74.mlp.experts.96.up_proj.weight": "model-00081-of-00101.safetensors", + "model.layers.74.mlp.experts.96.down_proj.weight": "model-00081-of-00101.safetensors", + "model.layers.74.mlp.experts.97.gate_proj.weight": "model-00081-of-00101.safetensors", + "model.layers.74.mlp.experts.97.up_proj.weight": "model-00081-of-00101.safetensors", + "model.layers.74.mlp.experts.97.down_proj.weight": "model-00081-of-00101.safetensors", + "model.layers.74.mlp.experts.98.gate_proj.weight": "model-00081-of-00101.safetensors", + "model.layers.74.mlp.experts.98.up_proj.weight": "model-00081-of-00101.safetensors", + "model.layers.74.mlp.experts.98.down_proj.weight": "model-00081-of-00101.safetensors", + "model.layers.74.mlp.experts.99.gate_proj.weight": "model-00081-of-00101.safetensors", + "model.layers.74.mlp.experts.99.up_proj.weight": "model-00081-of-00101.safetensors", + "model.layers.74.mlp.experts.99.down_proj.weight": "model-00081-of-00101.safetensors", + "model.layers.74.mlp.experts.100.gate_proj.weight": "model-00081-of-00101.safetensors", + "model.layers.74.mlp.experts.100.up_proj.weight": "model-00081-of-00101.safetensors", + "model.layers.74.mlp.experts.100.down_proj.weight": "model-00081-of-00101.safetensors", + "model.layers.74.mlp.experts.101.gate_proj.weight": "model-00081-of-00101.safetensors", + "model.layers.74.mlp.experts.101.up_proj.weight": "model-00081-of-00101.safetensors", + "model.layers.74.mlp.experts.101.down_proj.weight": "model-00081-of-00101.safetensors", + "model.layers.74.mlp.experts.102.gate_proj.weight": "model-00081-of-00101.safetensors", + "model.layers.74.mlp.experts.102.up_proj.weight": "model-00081-of-00101.safetensors", + "model.layers.74.mlp.experts.102.down_proj.weight": "model-00081-of-00101.safetensors", + "model.layers.74.mlp.experts.103.gate_proj.weight": "model-00081-of-00101.safetensors", + "model.layers.74.mlp.experts.103.up_proj.weight": "model-00081-of-00101.safetensors", + "model.layers.74.mlp.experts.103.down_proj.weight": "model-00081-of-00101.safetensors", + "model.layers.74.mlp.experts.104.gate_proj.weight": "model-00081-of-00101.safetensors", + "model.layers.74.mlp.experts.104.up_proj.weight": "model-00081-of-00101.safetensors", + "model.layers.74.mlp.experts.104.down_proj.weight": "model-00081-of-00101.safetensors", + "model.layers.74.mlp.experts.105.gate_proj.weight": "model-00081-of-00101.safetensors", + "model.layers.74.mlp.experts.105.up_proj.weight": "model-00081-of-00101.safetensors", + "model.layers.74.mlp.experts.105.down_proj.weight": "model-00081-of-00101.safetensors", + "model.layers.74.mlp.experts.106.gate_proj.weight": "model-00081-of-00101.safetensors", + "model.layers.74.mlp.experts.106.up_proj.weight": "model-00081-of-00101.safetensors", + "model.layers.74.mlp.experts.106.down_proj.weight": "model-00081-of-00101.safetensors", + "model.layers.74.mlp.experts.107.gate_proj.weight": "model-00081-of-00101.safetensors", + "model.layers.74.mlp.experts.107.up_proj.weight": "model-00081-of-00101.safetensors", + "model.layers.74.mlp.experts.107.down_proj.weight": "model-00082-of-00101.safetensors", + "model.layers.74.mlp.experts.108.gate_proj.weight": "model-00082-of-00101.safetensors", + "model.layers.74.mlp.experts.108.up_proj.weight": "model-00082-of-00101.safetensors", + "model.layers.74.mlp.experts.108.down_proj.weight": "model-00082-of-00101.safetensors", + "model.layers.74.mlp.experts.109.gate_proj.weight": "model-00082-of-00101.safetensors", + "model.layers.74.mlp.experts.109.up_proj.weight": "model-00082-of-00101.safetensors", + "model.layers.74.mlp.experts.109.down_proj.weight": "model-00082-of-00101.safetensors", + "model.layers.74.mlp.experts.110.gate_proj.weight": "model-00082-of-00101.safetensors", + "model.layers.74.mlp.experts.110.up_proj.weight": "model-00082-of-00101.safetensors", + "model.layers.74.mlp.experts.110.down_proj.weight": "model-00082-of-00101.safetensors", + "model.layers.74.mlp.experts.111.gate_proj.weight": "model-00082-of-00101.safetensors", + "model.layers.74.mlp.experts.111.up_proj.weight": "model-00082-of-00101.safetensors", + "model.layers.74.mlp.experts.111.down_proj.weight": "model-00082-of-00101.safetensors", + "model.layers.74.mlp.experts.112.gate_proj.weight": "model-00082-of-00101.safetensors", + "model.layers.74.mlp.experts.112.up_proj.weight": "model-00082-of-00101.safetensors", + "model.layers.74.mlp.experts.112.down_proj.weight": "model-00082-of-00101.safetensors", + "model.layers.74.mlp.experts.113.gate_proj.weight": "model-00082-of-00101.safetensors", + "model.layers.74.mlp.experts.113.up_proj.weight": "model-00082-of-00101.safetensors", + "model.layers.74.mlp.experts.113.down_proj.weight": "model-00082-of-00101.safetensors", + "model.layers.74.mlp.experts.114.gate_proj.weight": "model-00082-of-00101.safetensors", + "model.layers.74.mlp.experts.114.up_proj.weight": "model-00082-of-00101.safetensors", + "model.layers.74.mlp.experts.114.down_proj.weight": "model-00082-of-00101.safetensors", + "model.layers.74.mlp.experts.115.gate_proj.weight": "model-00082-of-00101.safetensors", + "model.layers.74.mlp.experts.115.up_proj.weight": "model-00082-of-00101.safetensors", + "model.layers.74.mlp.experts.115.down_proj.weight": "model-00082-of-00101.safetensors", + "model.layers.74.mlp.experts.116.gate_proj.weight": "model-00082-of-00101.safetensors", + "model.layers.74.mlp.experts.116.up_proj.weight": "model-00082-of-00101.safetensors", + "model.layers.74.mlp.experts.116.down_proj.weight": "model-00082-of-00101.safetensors", + "model.layers.74.mlp.experts.117.gate_proj.weight": "model-00082-of-00101.safetensors", + "model.layers.74.mlp.experts.117.up_proj.weight": "model-00082-of-00101.safetensors", + "model.layers.74.mlp.experts.117.down_proj.weight": "model-00082-of-00101.safetensors", + "model.layers.74.mlp.experts.118.gate_proj.weight": "model-00082-of-00101.safetensors", + "model.layers.74.mlp.experts.118.up_proj.weight": "model-00082-of-00101.safetensors", + "model.layers.74.mlp.experts.118.down_proj.weight": "model-00082-of-00101.safetensors", + "model.layers.74.mlp.experts.119.gate_proj.weight": "model-00082-of-00101.safetensors", + "model.layers.74.mlp.experts.119.up_proj.weight": "model-00082-of-00101.safetensors", + "model.layers.74.mlp.experts.119.down_proj.weight": "model-00082-of-00101.safetensors", + "model.layers.74.mlp.gate.weight": "model-00082-of-00101.safetensors", + "model.layers.74.mlp.gate.e_score_correction_bias": "model-00082-of-00101.safetensors", + "model.layers.74.mlp.shared_experts.gate_proj.weight": "model-00082-of-00101.safetensors", + "model.layers.74.mlp.shared_experts.up_proj.weight": "model-00082-of-00101.safetensors", + "model.layers.74.mlp.shared_experts.down_proj.weight": "model-00082-of-00101.safetensors", + "model.layers.74.input_layernorm.weight": "model-00082-of-00101.safetensors", + "model.layers.74.post_attention_layernorm.weight": "model-00082-of-00101.safetensors", + "model.layers.75.self_attn.q_proj.weight": "model-00082-of-00101.safetensors", + "model.layers.75.self_attn.q_proj.bias": "model-00082-of-00101.safetensors", + "model.layers.75.self_attn.k_proj.weight": "model-00082-of-00101.safetensors", + "model.layers.75.self_attn.k_proj.bias": "model-00082-of-00101.safetensors", + "model.layers.75.self_attn.v_proj.weight": "model-00082-of-00101.safetensors", + "model.layers.75.self_attn.v_proj.bias": "model-00082-of-00101.safetensors", + "model.layers.75.self_attn.o_proj.weight": "model-00082-of-00101.safetensors", + "model.layers.75.self_attn.q_norm.weight": "model-00082-of-00101.safetensors", + "model.layers.75.self_attn.k_norm.weight": "model-00082-of-00101.safetensors", + "model.layers.75.mlp.experts.0.gate_proj.weight": "model-00082-of-00101.safetensors", + "model.layers.75.mlp.experts.0.up_proj.weight": "model-00082-of-00101.safetensors", + "model.layers.75.mlp.experts.0.down_proj.weight": "model-00082-of-00101.safetensors", + "model.layers.75.mlp.experts.1.gate_proj.weight": "model-00082-of-00101.safetensors", + "model.layers.75.mlp.experts.1.up_proj.weight": "model-00082-of-00101.safetensors", + "model.layers.75.mlp.experts.1.down_proj.weight": "model-00082-of-00101.safetensors", + "model.layers.75.mlp.experts.2.gate_proj.weight": "model-00082-of-00101.safetensors", + "model.layers.75.mlp.experts.2.up_proj.weight": "model-00082-of-00101.safetensors", + "model.layers.75.mlp.experts.2.down_proj.weight": "model-00082-of-00101.safetensors", + "model.layers.75.mlp.experts.3.gate_proj.weight": "model-00082-of-00101.safetensors", + "model.layers.75.mlp.experts.3.up_proj.weight": "model-00082-of-00101.safetensors", + "model.layers.75.mlp.experts.3.down_proj.weight": "model-00082-of-00101.safetensors", + "model.layers.75.mlp.experts.4.gate_proj.weight": "model-00082-of-00101.safetensors", + "model.layers.75.mlp.experts.4.up_proj.weight": "model-00082-of-00101.safetensors", + "model.layers.75.mlp.experts.4.down_proj.weight": "model-00082-of-00101.safetensors", + "model.layers.75.mlp.experts.5.gate_proj.weight": "model-00082-of-00101.safetensors", + "model.layers.75.mlp.experts.5.up_proj.weight": "model-00082-of-00101.safetensors", + "model.layers.75.mlp.experts.5.down_proj.weight": "model-00082-of-00101.safetensors", + "model.layers.75.mlp.experts.6.gate_proj.weight": "model-00082-of-00101.safetensors", + "model.layers.75.mlp.experts.6.up_proj.weight": "model-00082-of-00101.safetensors", + "model.layers.75.mlp.experts.6.down_proj.weight": "model-00082-of-00101.safetensors", + "model.layers.75.mlp.experts.7.gate_proj.weight": "model-00082-of-00101.safetensors", + "model.layers.75.mlp.experts.7.up_proj.weight": "model-00082-of-00101.safetensors", + "model.layers.75.mlp.experts.7.down_proj.weight": "model-00082-of-00101.safetensors", + "model.layers.75.mlp.experts.8.gate_proj.weight": "model-00082-of-00101.safetensors", + "model.layers.75.mlp.experts.8.up_proj.weight": "model-00082-of-00101.safetensors", + "model.layers.75.mlp.experts.8.down_proj.weight": "model-00082-of-00101.safetensors", + "model.layers.75.mlp.experts.9.gate_proj.weight": "model-00082-of-00101.safetensors", + "model.layers.75.mlp.experts.9.up_proj.weight": "model-00082-of-00101.safetensors", + "model.layers.75.mlp.experts.9.down_proj.weight": "model-00082-of-00101.safetensors", + "model.layers.75.mlp.experts.10.gate_proj.weight": "model-00082-of-00101.safetensors", + "model.layers.75.mlp.experts.10.up_proj.weight": "model-00082-of-00101.safetensors", + "model.layers.75.mlp.experts.10.down_proj.weight": "model-00082-of-00101.safetensors", + "model.layers.75.mlp.experts.11.gate_proj.weight": "model-00082-of-00101.safetensors", + "model.layers.75.mlp.experts.11.up_proj.weight": "model-00082-of-00101.safetensors", + "model.layers.75.mlp.experts.11.down_proj.weight": "model-00082-of-00101.safetensors", + "model.layers.75.mlp.experts.12.gate_proj.weight": "model-00082-of-00101.safetensors", + "model.layers.75.mlp.experts.12.up_proj.weight": "model-00082-of-00101.safetensors", + "model.layers.75.mlp.experts.12.down_proj.weight": "model-00082-of-00101.safetensors", + "model.layers.75.mlp.experts.13.gate_proj.weight": "model-00082-of-00101.safetensors", + "model.layers.75.mlp.experts.13.up_proj.weight": "model-00082-of-00101.safetensors", + "model.layers.75.mlp.experts.13.down_proj.weight": "model-00082-of-00101.safetensors", + "model.layers.75.mlp.experts.14.gate_proj.weight": "model-00082-of-00101.safetensors", + "model.layers.75.mlp.experts.14.up_proj.weight": "model-00082-of-00101.safetensors", + "model.layers.75.mlp.experts.14.down_proj.weight": "model-00082-of-00101.safetensors", + "model.layers.75.mlp.experts.15.gate_proj.weight": "model-00082-of-00101.safetensors", + "model.layers.75.mlp.experts.15.up_proj.weight": "model-00082-of-00101.safetensors", + "model.layers.75.mlp.experts.15.down_proj.weight": "model-00082-of-00101.safetensors", + "model.layers.75.mlp.experts.16.gate_proj.weight": "model-00082-of-00101.safetensors", + "model.layers.75.mlp.experts.16.up_proj.weight": "model-00082-of-00101.safetensors", + "model.layers.75.mlp.experts.16.down_proj.weight": "model-00082-of-00101.safetensors", + "model.layers.75.mlp.experts.17.gate_proj.weight": "model-00082-of-00101.safetensors", + "model.layers.75.mlp.experts.17.up_proj.weight": "model-00082-of-00101.safetensors", + "model.layers.75.mlp.experts.17.down_proj.weight": "model-00082-of-00101.safetensors", + "model.layers.75.mlp.experts.18.gate_proj.weight": "model-00082-of-00101.safetensors", + "model.layers.75.mlp.experts.18.up_proj.weight": "model-00082-of-00101.safetensors", + "model.layers.75.mlp.experts.18.down_proj.weight": "model-00082-of-00101.safetensors", + "model.layers.75.mlp.experts.19.gate_proj.weight": "model-00082-of-00101.safetensors", + "model.layers.75.mlp.experts.19.up_proj.weight": "model-00082-of-00101.safetensors", + "model.layers.75.mlp.experts.19.down_proj.weight": "model-00082-of-00101.safetensors", + "model.layers.75.mlp.experts.20.gate_proj.weight": "model-00082-of-00101.safetensors", + "model.layers.75.mlp.experts.20.up_proj.weight": "model-00082-of-00101.safetensors", + "model.layers.75.mlp.experts.20.down_proj.weight": "model-00082-of-00101.safetensors", + "model.layers.75.mlp.experts.21.gate_proj.weight": "model-00082-of-00101.safetensors", + "model.layers.75.mlp.experts.21.up_proj.weight": "model-00082-of-00101.safetensors", + "model.layers.75.mlp.experts.21.down_proj.weight": "model-00082-of-00101.safetensors", + "model.layers.75.mlp.experts.22.gate_proj.weight": "model-00082-of-00101.safetensors", + "model.layers.75.mlp.experts.22.up_proj.weight": "model-00082-of-00101.safetensors", + "model.layers.75.mlp.experts.22.down_proj.weight": "model-00082-of-00101.safetensors", + "model.layers.75.mlp.experts.23.gate_proj.weight": "model-00082-of-00101.safetensors", + "model.layers.75.mlp.experts.23.up_proj.weight": "model-00082-of-00101.safetensors", + "model.layers.75.mlp.experts.23.down_proj.weight": "model-00082-of-00101.safetensors", + "model.layers.75.mlp.experts.24.gate_proj.weight": "model-00082-of-00101.safetensors", + "model.layers.75.mlp.experts.24.up_proj.weight": "model-00082-of-00101.safetensors", + "model.layers.75.mlp.experts.24.down_proj.weight": "model-00082-of-00101.safetensors", + "model.layers.75.mlp.experts.25.gate_proj.weight": "model-00082-of-00101.safetensors", + "model.layers.75.mlp.experts.25.up_proj.weight": "model-00082-of-00101.safetensors", + "model.layers.75.mlp.experts.25.down_proj.weight": "model-00082-of-00101.safetensors", + "model.layers.75.mlp.experts.26.gate_proj.weight": "model-00082-of-00101.safetensors", + "model.layers.75.mlp.experts.26.up_proj.weight": "model-00082-of-00101.safetensors", + "model.layers.75.mlp.experts.26.down_proj.weight": "model-00082-of-00101.safetensors", + "model.layers.75.mlp.experts.27.gate_proj.weight": "model-00082-of-00101.safetensors", + "model.layers.75.mlp.experts.27.up_proj.weight": "model-00082-of-00101.safetensors", + "model.layers.75.mlp.experts.27.down_proj.weight": "model-00082-of-00101.safetensors", + "model.layers.75.mlp.experts.28.gate_proj.weight": "model-00082-of-00101.safetensors", + "model.layers.75.mlp.experts.28.up_proj.weight": "model-00082-of-00101.safetensors", + "model.layers.75.mlp.experts.28.down_proj.weight": "model-00082-of-00101.safetensors", + "model.layers.75.mlp.experts.29.gate_proj.weight": "model-00082-of-00101.safetensors", + "model.layers.75.mlp.experts.29.up_proj.weight": "model-00082-of-00101.safetensors", + "model.layers.75.mlp.experts.29.down_proj.weight": "model-00082-of-00101.safetensors", + "model.layers.75.mlp.experts.30.gate_proj.weight": "model-00082-of-00101.safetensors", + "model.layers.75.mlp.experts.30.up_proj.weight": "model-00082-of-00101.safetensors", + "model.layers.75.mlp.experts.30.down_proj.weight": "model-00082-of-00101.safetensors", + "model.layers.75.mlp.experts.31.gate_proj.weight": "model-00082-of-00101.safetensors", + "model.layers.75.mlp.experts.31.up_proj.weight": "model-00082-of-00101.safetensors", + "model.layers.75.mlp.experts.31.down_proj.weight": "model-00082-of-00101.safetensors", + "model.layers.75.mlp.experts.32.gate_proj.weight": "model-00082-of-00101.safetensors", + "model.layers.75.mlp.experts.32.up_proj.weight": "model-00082-of-00101.safetensors", + "model.layers.75.mlp.experts.32.down_proj.weight": "model-00082-of-00101.safetensors", + "model.layers.75.mlp.experts.33.gate_proj.weight": "model-00082-of-00101.safetensors", + "model.layers.75.mlp.experts.33.up_proj.weight": "model-00082-of-00101.safetensors", + "model.layers.75.mlp.experts.33.down_proj.weight": "model-00082-of-00101.safetensors", + "model.layers.75.mlp.experts.34.gate_proj.weight": "model-00082-of-00101.safetensors", + "model.layers.75.mlp.experts.34.up_proj.weight": "model-00082-of-00101.safetensors", + "model.layers.75.mlp.experts.34.down_proj.weight": "model-00082-of-00101.safetensors", + "model.layers.75.mlp.experts.35.gate_proj.weight": "model-00082-of-00101.safetensors", + "model.layers.75.mlp.experts.35.up_proj.weight": "model-00082-of-00101.safetensors", + "model.layers.75.mlp.experts.35.down_proj.weight": "model-00082-of-00101.safetensors", + "model.layers.75.mlp.experts.36.gate_proj.weight": "model-00082-of-00101.safetensors", + "model.layers.75.mlp.experts.36.up_proj.weight": "model-00082-of-00101.safetensors", + "model.layers.75.mlp.experts.36.down_proj.weight": "model-00082-of-00101.safetensors", + "model.layers.75.mlp.experts.37.gate_proj.weight": "model-00082-of-00101.safetensors", + "model.layers.75.mlp.experts.37.up_proj.weight": "model-00082-of-00101.safetensors", + "model.layers.75.mlp.experts.37.down_proj.weight": "model-00082-of-00101.safetensors", + "model.layers.75.mlp.experts.38.gate_proj.weight": "model-00082-of-00101.safetensors", + "model.layers.75.mlp.experts.38.up_proj.weight": "model-00082-of-00101.safetensors", + "model.layers.75.mlp.experts.38.down_proj.weight": "model-00082-of-00101.safetensors", + "model.layers.75.mlp.experts.39.gate_proj.weight": "model-00082-of-00101.safetensors", + "model.layers.75.mlp.experts.39.up_proj.weight": "model-00082-of-00101.safetensors", + "model.layers.75.mlp.experts.39.down_proj.weight": "model-00082-of-00101.safetensors", + "model.layers.75.mlp.experts.40.gate_proj.weight": "model-00082-of-00101.safetensors", + "model.layers.75.mlp.experts.40.up_proj.weight": "model-00082-of-00101.safetensors", + "model.layers.75.mlp.experts.40.down_proj.weight": "model-00082-of-00101.safetensors", + "model.layers.75.mlp.experts.41.gate_proj.weight": "model-00082-of-00101.safetensors", + "model.layers.75.mlp.experts.41.up_proj.weight": "model-00082-of-00101.safetensors", + "model.layers.75.mlp.experts.41.down_proj.weight": "model-00082-of-00101.safetensors", + "model.layers.75.mlp.experts.42.gate_proj.weight": "model-00082-of-00101.safetensors", + "model.layers.75.mlp.experts.42.up_proj.weight": "model-00082-of-00101.safetensors", + "model.layers.75.mlp.experts.42.down_proj.weight": "model-00082-of-00101.safetensors", + "model.layers.75.mlp.experts.43.gate_proj.weight": "model-00082-of-00101.safetensors", + "model.layers.75.mlp.experts.43.up_proj.weight": "model-00082-of-00101.safetensors", + "model.layers.75.mlp.experts.43.down_proj.weight": "model-00082-of-00101.safetensors", + "model.layers.75.mlp.experts.44.gate_proj.weight": "model-00082-of-00101.safetensors", + "model.layers.75.mlp.experts.44.up_proj.weight": "model-00082-of-00101.safetensors", + "model.layers.75.mlp.experts.44.down_proj.weight": "model-00082-of-00101.safetensors", + "model.layers.75.mlp.experts.45.gate_proj.weight": "model-00082-of-00101.safetensors", + "model.layers.75.mlp.experts.45.up_proj.weight": "model-00082-of-00101.safetensors", + "model.layers.75.mlp.experts.45.down_proj.weight": "model-00082-of-00101.safetensors", + "model.layers.75.mlp.experts.46.gate_proj.weight": "model-00082-of-00101.safetensors", + "model.layers.75.mlp.experts.46.up_proj.weight": "model-00082-of-00101.safetensors", + "model.layers.75.mlp.experts.46.down_proj.weight": "model-00082-of-00101.safetensors", + "model.layers.75.mlp.experts.47.gate_proj.weight": "model-00082-of-00101.safetensors", + "model.layers.75.mlp.experts.47.up_proj.weight": "model-00082-of-00101.safetensors", + "model.layers.75.mlp.experts.47.down_proj.weight": "model-00082-of-00101.safetensors", + "model.layers.75.mlp.experts.48.gate_proj.weight": "model-00082-of-00101.safetensors", + "model.layers.75.mlp.experts.48.up_proj.weight": "model-00082-of-00101.safetensors", + "model.layers.75.mlp.experts.48.down_proj.weight": "model-00082-of-00101.safetensors", + "model.layers.75.mlp.experts.49.gate_proj.weight": "model-00082-of-00101.safetensors", + "model.layers.75.mlp.experts.49.up_proj.weight": "model-00082-of-00101.safetensors", + "model.layers.75.mlp.experts.49.down_proj.weight": "model-00082-of-00101.safetensors", + "model.layers.75.mlp.experts.50.gate_proj.weight": "model-00082-of-00101.safetensors", + "model.layers.75.mlp.experts.50.up_proj.weight": "model-00082-of-00101.safetensors", + "model.layers.75.mlp.experts.50.down_proj.weight": "model-00082-of-00101.safetensors", + "model.layers.75.mlp.experts.51.gate_proj.weight": "model-00082-of-00101.safetensors", + "model.layers.75.mlp.experts.51.up_proj.weight": "model-00082-of-00101.safetensors", + "model.layers.75.mlp.experts.51.down_proj.weight": "model-00082-of-00101.safetensors", + "model.layers.75.mlp.experts.52.gate_proj.weight": "model-00082-of-00101.safetensors", + "model.layers.75.mlp.experts.52.up_proj.weight": "model-00082-of-00101.safetensors", + "model.layers.75.mlp.experts.52.down_proj.weight": "model-00082-of-00101.safetensors", + "model.layers.75.mlp.experts.53.gate_proj.weight": "model-00082-of-00101.safetensors", + "model.layers.75.mlp.experts.53.up_proj.weight": "model-00082-of-00101.safetensors", + "model.layers.75.mlp.experts.53.down_proj.weight": "model-00082-of-00101.safetensors", + "model.layers.75.mlp.experts.54.gate_proj.weight": "model-00082-of-00101.safetensors", + "model.layers.75.mlp.experts.54.up_proj.weight": "model-00082-of-00101.safetensors", + "model.layers.75.mlp.experts.54.down_proj.weight": "model-00082-of-00101.safetensors", + "model.layers.75.mlp.experts.55.gate_proj.weight": "model-00082-of-00101.safetensors", + "model.layers.75.mlp.experts.55.up_proj.weight": "model-00082-of-00101.safetensors", + "model.layers.75.mlp.experts.55.down_proj.weight": "model-00082-of-00101.safetensors", + "model.layers.75.mlp.experts.56.gate_proj.weight": "model-00082-of-00101.safetensors", + "model.layers.75.mlp.experts.56.up_proj.weight": "model-00082-of-00101.safetensors", + "model.layers.75.mlp.experts.56.down_proj.weight": "model-00082-of-00101.safetensors", + "model.layers.75.mlp.experts.57.gate_proj.weight": "model-00082-of-00101.safetensors", + "model.layers.75.mlp.experts.57.up_proj.weight": "model-00082-of-00101.safetensors", + "model.layers.75.mlp.experts.57.down_proj.weight": "model-00082-of-00101.safetensors", + "model.layers.75.mlp.experts.58.gate_proj.weight": "model-00082-of-00101.safetensors", + "model.layers.75.mlp.experts.58.up_proj.weight": "model-00082-of-00101.safetensors", + "model.layers.75.mlp.experts.58.down_proj.weight": "model-00082-of-00101.safetensors", + "model.layers.75.mlp.experts.59.gate_proj.weight": "model-00082-of-00101.safetensors", + "model.layers.75.mlp.experts.59.up_proj.weight": "model-00082-of-00101.safetensors", + "model.layers.75.mlp.experts.59.down_proj.weight": "model-00082-of-00101.safetensors", + "model.layers.75.mlp.experts.60.gate_proj.weight": "model-00082-of-00101.safetensors", + "model.layers.75.mlp.experts.60.up_proj.weight": "model-00082-of-00101.safetensors", + "model.layers.75.mlp.experts.60.down_proj.weight": "model-00082-of-00101.safetensors", + "model.layers.75.mlp.experts.61.gate_proj.weight": "model-00082-of-00101.safetensors", + "model.layers.75.mlp.experts.61.up_proj.weight": "model-00082-of-00101.safetensors", + "model.layers.75.mlp.experts.61.down_proj.weight": "model-00082-of-00101.safetensors", + "model.layers.75.mlp.experts.62.gate_proj.weight": "model-00082-of-00101.safetensors", + "model.layers.75.mlp.experts.62.up_proj.weight": "model-00082-of-00101.safetensors", + "model.layers.75.mlp.experts.62.down_proj.weight": "model-00082-of-00101.safetensors", + "model.layers.75.mlp.experts.63.gate_proj.weight": "model-00082-of-00101.safetensors", + "model.layers.75.mlp.experts.63.up_proj.weight": "model-00082-of-00101.safetensors", + "model.layers.75.mlp.experts.63.down_proj.weight": "model-00082-of-00101.safetensors", + "model.layers.75.mlp.experts.64.gate_proj.weight": "model-00082-of-00101.safetensors", + "model.layers.75.mlp.experts.64.up_proj.weight": "model-00082-of-00101.safetensors", + "model.layers.75.mlp.experts.64.down_proj.weight": "model-00082-of-00101.safetensors", + "model.layers.75.mlp.experts.65.gate_proj.weight": "model-00082-of-00101.safetensors", + "model.layers.75.mlp.experts.65.up_proj.weight": "model-00082-of-00101.safetensors", + "model.layers.75.mlp.experts.65.down_proj.weight": "model-00082-of-00101.safetensors", + "model.layers.75.mlp.experts.66.gate_proj.weight": "model-00082-of-00101.safetensors", + "model.layers.75.mlp.experts.66.up_proj.weight": "model-00082-of-00101.safetensors", + "model.layers.75.mlp.experts.66.down_proj.weight": "model-00082-of-00101.safetensors", + "model.layers.75.mlp.experts.67.gate_proj.weight": "model-00082-of-00101.safetensors", + "model.layers.75.mlp.experts.67.up_proj.weight": "model-00082-of-00101.safetensors", + "model.layers.75.mlp.experts.67.down_proj.weight": "model-00082-of-00101.safetensors", + "model.layers.75.mlp.experts.68.gate_proj.weight": "model-00082-of-00101.safetensors", + "model.layers.75.mlp.experts.68.up_proj.weight": "model-00082-of-00101.safetensors", + "model.layers.75.mlp.experts.68.down_proj.weight": "model-00082-of-00101.safetensors", + "model.layers.75.mlp.experts.69.gate_proj.weight": "model-00082-of-00101.safetensors", + "model.layers.75.mlp.experts.69.up_proj.weight": "model-00082-of-00101.safetensors", + "model.layers.75.mlp.experts.69.down_proj.weight": "model-00082-of-00101.safetensors", + "model.layers.75.mlp.experts.70.gate_proj.weight": "model-00082-of-00101.safetensors", + "model.layers.75.mlp.experts.70.up_proj.weight": "model-00082-of-00101.safetensors", + "model.layers.75.mlp.experts.70.down_proj.weight": "model-00082-of-00101.safetensors", + "model.layers.75.mlp.experts.71.gate_proj.weight": "model-00082-of-00101.safetensors", + "model.layers.75.mlp.experts.71.up_proj.weight": "model-00082-of-00101.safetensors", + "model.layers.75.mlp.experts.71.down_proj.weight": "model-00082-of-00101.safetensors", + "model.layers.75.mlp.experts.72.gate_proj.weight": "model-00082-of-00101.safetensors", + "model.layers.75.mlp.experts.72.up_proj.weight": "model-00082-of-00101.safetensors", + "model.layers.75.mlp.experts.72.down_proj.weight": "model-00082-of-00101.safetensors", + "model.layers.75.mlp.experts.73.gate_proj.weight": "model-00082-of-00101.safetensors", + "model.layers.75.mlp.experts.73.up_proj.weight": "model-00082-of-00101.safetensors", + "model.layers.75.mlp.experts.73.down_proj.weight": "model-00082-of-00101.safetensors", + "model.layers.75.mlp.experts.74.gate_proj.weight": "model-00082-of-00101.safetensors", + "model.layers.75.mlp.experts.74.up_proj.weight": "model-00082-of-00101.safetensors", + "model.layers.75.mlp.experts.74.down_proj.weight": "model-00082-of-00101.safetensors", + "model.layers.75.mlp.experts.75.gate_proj.weight": "model-00082-of-00101.safetensors", + "model.layers.75.mlp.experts.75.up_proj.weight": "model-00082-of-00101.safetensors", + "model.layers.75.mlp.experts.75.down_proj.weight": "model-00082-of-00101.safetensors", + "model.layers.75.mlp.experts.76.gate_proj.weight": "model-00082-of-00101.safetensors", + "model.layers.75.mlp.experts.76.up_proj.weight": "model-00082-of-00101.safetensors", + "model.layers.75.mlp.experts.76.down_proj.weight": "model-00082-of-00101.safetensors", + "model.layers.75.mlp.experts.77.gate_proj.weight": "model-00082-of-00101.safetensors", + "model.layers.75.mlp.experts.77.up_proj.weight": "model-00082-of-00101.safetensors", + "model.layers.75.mlp.experts.77.down_proj.weight": "model-00082-of-00101.safetensors", + "model.layers.75.mlp.experts.78.gate_proj.weight": "model-00082-of-00101.safetensors", + "model.layers.75.mlp.experts.78.up_proj.weight": "model-00082-of-00101.safetensors", + "model.layers.75.mlp.experts.78.down_proj.weight": "model-00082-of-00101.safetensors", + "model.layers.75.mlp.experts.79.gate_proj.weight": "model-00082-of-00101.safetensors", + "model.layers.75.mlp.experts.79.up_proj.weight": "model-00082-of-00101.safetensors", + "model.layers.75.mlp.experts.79.down_proj.weight": "model-00082-of-00101.safetensors", + "model.layers.75.mlp.experts.80.gate_proj.weight": "model-00082-of-00101.safetensors", + "model.layers.75.mlp.experts.80.up_proj.weight": "model-00082-of-00101.safetensors", + "model.layers.75.mlp.experts.80.down_proj.weight": "model-00082-of-00101.safetensors", + "model.layers.75.mlp.experts.81.gate_proj.weight": "model-00082-of-00101.safetensors", + "model.layers.75.mlp.experts.81.up_proj.weight": "model-00082-of-00101.safetensors", + "model.layers.75.mlp.experts.81.down_proj.weight": "model-00082-of-00101.safetensors", + "model.layers.75.mlp.experts.82.gate_proj.weight": "model-00082-of-00101.safetensors", + "model.layers.75.mlp.experts.82.up_proj.weight": "model-00082-of-00101.safetensors", + "model.layers.75.mlp.experts.82.down_proj.weight": "model-00082-of-00101.safetensors", + "model.layers.75.mlp.experts.83.gate_proj.weight": "model-00082-of-00101.safetensors", + "model.layers.75.mlp.experts.83.up_proj.weight": "model-00082-of-00101.safetensors", + "model.layers.75.mlp.experts.83.down_proj.weight": "model-00082-of-00101.safetensors", + "model.layers.75.mlp.experts.84.gate_proj.weight": "model-00082-of-00101.safetensors", + "model.layers.75.mlp.experts.84.up_proj.weight": "model-00082-of-00101.safetensors", + "model.layers.75.mlp.experts.84.down_proj.weight": "model-00082-of-00101.safetensors", + "model.layers.75.mlp.experts.85.gate_proj.weight": "model-00082-of-00101.safetensors", + "model.layers.75.mlp.experts.85.up_proj.weight": "model-00082-of-00101.safetensors", + "model.layers.75.mlp.experts.85.down_proj.weight": "model-00082-of-00101.safetensors", + "model.layers.75.mlp.experts.86.gate_proj.weight": "model-00082-of-00101.safetensors", + "model.layers.75.mlp.experts.86.up_proj.weight": "model-00082-of-00101.safetensors", + "model.layers.75.mlp.experts.86.down_proj.weight": "model-00082-of-00101.safetensors", + "model.layers.75.mlp.experts.87.gate_proj.weight": "model-00082-of-00101.safetensors", + "model.layers.75.mlp.experts.87.up_proj.weight": "model-00082-of-00101.safetensors", + "model.layers.75.mlp.experts.87.down_proj.weight": "model-00082-of-00101.safetensors", + "model.layers.75.mlp.experts.88.gate_proj.weight": "model-00082-of-00101.safetensors", + "model.layers.75.mlp.experts.88.up_proj.weight": "model-00082-of-00101.safetensors", + "model.layers.75.mlp.experts.88.down_proj.weight": "model-00082-of-00101.safetensors", + "model.layers.75.mlp.experts.89.gate_proj.weight": "model-00082-of-00101.safetensors", + "model.layers.75.mlp.experts.89.up_proj.weight": "model-00082-of-00101.safetensors", + "model.layers.75.mlp.experts.89.down_proj.weight": "model-00082-of-00101.safetensors", + "model.layers.75.mlp.experts.90.gate_proj.weight": "model-00082-of-00101.safetensors", + "model.layers.75.mlp.experts.90.up_proj.weight": "model-00082-of-00101.safetensors", + "model.layers.75.mlp.experts.90.down_proj.weight": "model-00082-of-00101.safetensors", + "model.layers.75.mlp.experts.91.gate_proj.weight": "model-00082-of-00101.safetensors", + "model.layers.75.mlp.experts.91.up_proj.weight": "model-00082-of-00101.safetensors", + "model.layers.75.mlp.experts.91.down_proj.weight": "model-00082-of-00101.safetensors", + "model.layers.75.mlp.experts.92.gate_proj.weight": "model-00082-of-00101.safetensors", + "model.layers.75.mlp.experts.92.up_proj.weight": "model-00082-of-00101.safetensors", + "model.layers.75.mlp.experts.92.down_proj.weight": "model-00082-of-00101.safetensors", + "model.layers.75.mlp.experts.93.gate_proj.weight": "model-00082-of-00101.safetensors", + "model.layers.75.mlp.experts.93.up_proj.weight": "model-00082-of-00101.safetensors", + "model.layers.75.mlp.experts.93.down_proj.weight": "model-00082-of-00101.safetensors", + "model.layers.75.mlp.experts.94.gate_proj.weight": "model-00082-of-00101.safetensors", + "model.layers.75.mlp.experts.94.up_proj.weight": "model-00083-of-00101.safetensors", + "model.layers.75.mlp.experts.94.down_proj.weight": "model-00083-of-00101.safetensors", + "model.layers.75.mlp.experts.95.gate_proj.weight": "model-00083-of-00101.safetensors", + "model.layers.75.mlp.experts.95.up_proj.weight": "model-00083-of-00101.safetensors", + "model.layers.75.mlp.experts.95.down_proj.weight": "model-00083-of-00101.safetensors", + "model.layers.75.mlp.experts.96.gate_proj.weight": "model-00083-of-00101.safetensors", + "model.layers.75.mlp.experts.96.up_proj.weight": "model-00083-of-00101.safetensors", + "model.layers.75.mlp.experts.96.down_proj.weight": "model-00083-of-00101.safetensors", + "model.layers.75.mlp.experts.97.gate_proj.weight": "model-00083-of-00101.safetensors", + "model.layers.75.mlp.experts.97.up_proj.weight": "model-00083-of-00101.safetensors", + "model.layers.75.mlp.experts.97.down_proj.weight": "model-00083-of-00101.safetensors", + "model.layers.75.mlp.experts.98.gate_proj.weight": "model-00083-of-00101.safetensors", + "model.layers.75.mlp.experts.98.up_proj.weight": "model-00083-of-00101.safetensors", + "model.layers.75.mlp.experts.98.down_proj.weight": "model-00083-of-00101.safetensors", + "model.layers.75.mlp.experts.99.gate_proj.weight": "model-00083-of-00101.safetensors", + "model.layers.75.mlp.experts.99.up_proj.weight": "model-00083-of-00101.safetensors", + "model.layers.75.mlp.experts.99.down_proj.weight": "model-00083-of-00101.safetensors", + "model.layers.75.mlp.experts.100.gate_proj.weight": "model-00083-of-00101.safetensors", + "model.layers.75.mlp.experts.100.up_proj.weight": "model-00083-of-00101.safetensors", + "model.layers.75.mlp.experts.100.down_proj.weight": "model-00083-of-00101.safetensors", + "model.layers.75.mlp.experts.101.gate_proj.weight": "model-00083-of-00101.safetensors", + "model.layers.75.mlp.experts.101.up_proj.weight": "model-00083-of-00101.safetensors", + "model.layers.75.mlp.experts.101.down_proj.weight": "model-00083-of-00101.safetensors", + "model.layers.75.mlp.experts.102.gate_proj.weight": "model-00083-of-00101.safetensors", + "model.layers.75.mlp.experts.102.up_proj.weight": "model-00083-of-00101.safetensors", + "model.layers.75.mlp.experts.102.down_proj.weight": "model-00083-of-00101.safetensors", + "model.layers.75.mlp.experts.103.gate_proj.weight": "model-00083-of-00101.safetensors", + "model.layers.75.mlp.experts.103.up_proj.weight": "model-00083-of-00101.safetensors", + "model.layers.75.mlp.experts.103.down_proj.weight": "model-00083-of-00101.safetensors", + "model.layers.75.mlp.experts.104.gate_proj.weight": "model-00083-of-00101.safetensors", + "model.layers.75.mlp.experts.104.up_proj.weight": "model-00083-of-00101.safetensors", + "model.layers.75.mlp.experts.104.down_proj.weight": "model-00083-of-00101.safetensors", + "model.layers.75.mlp.experts.105.gate_proj.weight": "model-00083-of-00101.safetensors", + "model.layers.75.mlp.experts.105.up_proj.weight": "model-00083-of-00101.safetensors", + "model.layers.75.mlp.experts.105.down_proj.weight": "model-00083-of-00101.safetensors", + "model.layers.75.mlp.experts.106.gate_proj.weight": "model-00083-of-00101.safetensors", + "model.layers.75.mlp.experts.106.up_proj.weight": "model-00083-of-00101.safetensors", + "model.layers.75.mlp.experts.106.down_proj.weight": "model-00083-of-00101.safetensors", + "model.layers.75.mlp.experts.107.gate_proj.weight": "model-00083-of-00101.safetensors", + "model.layers.75.mlp.experts.107.up_proj.weight": "model-00083-of-00101.safetensors", + "model.layers.75.mlp.experts.107.down_proj.weight": "model-00083-of-00101.safetensors", + "model.layers.75.mlp.experts.108.gate_proj.weight": "model-00083-of-00101.safetensors", + "model.layers.75.mlp.experts.108.up_proj.weight": "model-00083-of-00101.safetensors", + "model.layers.75.mlp.experts.108.down_proj.weight": "model-00083-of-00101.safetensors", + "model.layers.75.mlp.experts.109.gate_proj.weight": "model-00083-of-00101.safetensors", + "model.layers.75.mlp.experts.109.up_proj.weight": "model-00083-of-00101.safetensors", + "model.layers.75.mlp.experts.109.down_proj.weight": "model-00083-of-00101.safetensors", + "model.layers.75.mlp.experts.110.gate_proj.weight": "model-00083-of-00101.safetensors", + "model.layers.75.mlp.experts.110.up_proj.weight": "model-00083-of-00101.safetensors", + "model.layers.75.mlp.experts.110.down_proj.weight": "model-00083-of-00101.safetensors", + "model.layers.75.mlp.experts.111.gate_proj.weight": "model-00083-of-00101.safetensors", + "model.layers.75.mlp.experts.111.up_proj.weight": "model-00083-of-00101.safetensors", + "model.layers.75.mlp.experts.111.down_proj.weight": "model-00083-of-00101.safetensors", + "model.layers.75.mlp.experts.112.gate_proj.weight": "model-00083-of-00101.safetensors", + "model.layers.75.mlp.experts.112.up_proj.weight": "model-00083-of-00101.safetensors", + "model.layers.75.mlp.experts.112.down_proj.weight": "model-00083-of-00101.safetensors", + "model.layers.75.mlp.experts.113.gate_proj.weight": "model-00083-of-00101.safetensors", + "model.layers.75.mlp.experts.113.up_proj.weight": "model-00083-of-00101.safetensors", + "model.layers.75.mlp.experts.113.down_proj.weight": "model-00083-of-00101.safetensors", + "model.layers.75.mlp.experts.114.gate_proj.weight": "model-00083-of-00101.safetensors", + "model.layers.75.mlp.experts.114.up_proj.weight": "model-00083-of-00101.safetensors", + "model.layers.75.mlp.experts.114.down_proj.weight": "model-00083-of-00101.safetensors", + "model.layers.75.mlp.experts.115.gate_proj.weight": "model-00083-of-00101.safetensors", + "model.layers.75.mlp.experts.115.up_proj.weight": "model-00083-of-00101.safetensors", + "model.layers.75.mlp.experts.115.down_proj.weight": "model-00083-of-00101.safetensors", + "model.layers.75.mlp.experts.116.gate_proj.weight": "model-00083-of-00101.safetensors", + "model.layers.75.mlp.experts.116.up_proj.weight": "model-00083-of-00101.safetensors", + "model.layers.75.mlp.experts.116.down_proj.weight": "model-00083-of-00101.safetensors", + "model.layers.75.mlp.experts.117.gate_proj.weight": "model-00083-of-00101.safetensors", + "model.layers.75.mlp.experts.117.up_proj.weight": "model-00083-of-00101.safetensors", + "model.layers.75.mlp.experts.117.down_proj.weight": "model-00083-of-00101.safetensors", + "model.layers.75.mlp.experts.118.gate_proj.weight": "model-00083-of-00101.safetensors", + "model.layers.75.mlp.experts.118.up_proj.weight": "model-00083-of-00101.safetensors", + "model.layers.75.mlp.experts.118.down_proj.weight": "model-00083-of-00101.safetensors", + "model.layers.75.mlp.experts.119.gate_proj.weight": "model-00083-of-00101.safetensors", + "model.layers.75.mlp.experts.119.up_proj.weight": "model-00083-of-00101.safetensors", + "model.layers.75.mlp.experts.119.down_proj.weight": "model-00083-of-00101.safetensors", + "model.layers.75.mlp.gate.weight": "model-00083-of-00101.safetensors", + "model.layers.75.mlp.gate.e_score_correction_bias": "model-00083-of-00101.safetensors", + "model.layers.75.mlp.shared_experts.gate_proj.weight": "model-00083-of-00101.safetensors", + "model.layers.75.mlp.shared_experts.up_proj.weight": "model-00083-of-00101.safetensors", + "model.layers.75.mlp.shared_experts.down_proj.weight": "model-00083-of-00101.safetensors", + "model.layers.75.input_layernorm.weight": "model-00083-of-00101.safetensors", + "model.layers.75.post_attention_layernorm.weight": "model-00083-of-00101.safetensors", + "model.layers.76.self_attn.q_proj.weight": "model-00083-of-00101.safetensors", + "model.layers.76.self_attn.q_proj.bias": "model-00083-of-00101.safetensors", + "model.layers.76.self_attn.k_proj.weight": "model-00083-of-00101.safetensors", + "model.layers.76.self_attn.k_proj.bias": "model-00083-of-00101.safetensors", + "model.layers.76.self_attn.v_proj.weight": "model-00083-of-00101.safetensors", + "model.layers.76.self_attn.v_proj.bias": "model-00083-of-00101.safetensors", + "model.layers.76.self_attn.o_proj.weight": "model-00083-of-00101.safetensors", + "model.layers.76.self_attn.q_norm.weight": "model-00083-of-00101.safetensors", + "model.layers.76.self_attn.k_norm.weight": "model-00083-of-00101.safetensors", + "model.layers.76.mlp.experts.0.gate_proj.weight": "model-00083-of-00101.safetensors", + "model.layers.76.mlp.experts.0.up_proj.weight": "model-00083-of-00101.safetensors", + "model.layers.76.mlp.experts.0.down_proj.weight": "model-00083-of-00101.safetensors", + "model.layers.76.mlp.experts.1.gate_proj.weight": "model-00083-of-00101.safetensors", + "model.layers.76.mlp.experts.1.up_proj.weight": "model-00083-of-00101.safetensors", + "model.layers.76.mlp.experts.1.down_proj.weight": "model-00083-of-00101.safetensors", + "model.layers.76.mlp.experts.2.gate_proj.weight": "model-00083-of-00101.safetensors", + "model.layers.76.mlp.experts.2.up_proj.weight": "model-00083-of-00101.safetensors", + "model.layers.76.mlp.experts.2.down_proj.weight": "model-00083-of-00101.safetensors", + "model.layers.76.mlp.experts.3.gate_proj.weight": "model-00083-of-00101.safetensors", + "model.layers.76.mlp.experts.3.up_proj.weight": "model-00083-of-00101.safetensors", + "model.layers.76.mlp.experts.3.down_proj.weight": "model-00083-of-00101.safetensors", + "model.layers.76.mlp.experts.4.gate_proj.weight": "model-00083-of-00101.safetensors", + "model.layers.76.mlp.experts.4.up_proj.weight": "model-00083-of-00101.safetensors", + "model.layers.76.mlp.experts.4.down_proj.weight": "model-00083-of-00101.safetensors", + "model.layers.76.mlp.experts.5.gate_proj.weight": "model-00083-of-00101.safetensors", + "model.layers.76.mlp.experts.5.up_proj.weight": "model-00083-of-00101.safetensors", + "model.layers.76.mlp.experts.5.down_proj.weight": "model-00083-of-00101.safetensors", + "model.layers.76.mlp.experts.6.gate_proj.weight": "model-00083-of-00101.safetensors", + "model.layers.76.mlp.experts.6.up_proj.weight": "model-00083-of-00101.safetensors", + "model.layers.76.mlp.experts.6.down_proj.weight": "model-00083-of-00101.safetensors", + "model.layers.76.mlp.experts.7.gate_proj.weight": "model-00083-of-00101.safetensors", + "model.layers.76.mlp.experts.7.up_proj.weight": "model-00083-of-00101.safetensors", + "model.layers.76.mlp.experts.7.down_proj.weight": "model-00083-of-00101.safetensors", + "model.layers.76.mlp.experts.8.gate_proj.weight": "model-00083-of-00101.safetensors", + "model.layers.76.mlp.experts.8.up_proj.weight": "model-00083-of-00101.safetensors", + "model.layers.76.mlp.experts.8.down_proj.weight": "model-00083-of-00101.safetensors", + "model.layers.76.mlp.experts.9.gate_proj.weight": "model-00083-of-00101.safetensors", + "model.layers.76.mlp.experts.9.up_proj.weight": "model-00083-of-00101.safetensors", + "model.layers.76.mlp.experts.9.down_proj.weight": "model-00083-of-00101.safetensors", + "model.layers.76.mlp.experts.10.gate_proj.weight": "model-00083-of-00101.safetensors", + "model.layers.76.mlp.experts.10.up_proj.weight": "model-00083-of-00101.safetensors", + "model.layers.76.mlp.experts.10.down_proj.weight": "model-00083-of-00101.safetensors", + "model.layers.76.mlp.experts.11.gate_proj.weight": "model-00083-of-00101.safetensors", + "model.layers.76.mlp.experts.11.up_proj.weight": "model-00083-of-00101.safetensors", + "model.layers.76.mlp.experts.11.down_proj.weight": "model-00083-of-00101.safetensors", + "model.layers.76.mlp.experts.12.gate_proj.weight": "model-00083-of-00101.safetensors", + "model.layers.76.mlp.experts.12.up_proj.weight": "model-00083-of-00101.safetensors", + "model.layers.76.mlp.experts.12.down_proj.weight": "model-00083-of-00101.safetensors", + "model.layers.76.mlp.experts.13.gate_proj.weight": "model-00083-of-00101.safetensors", + "model.layers.76.mlp.experts.13.up_proj.weight": "model-00083-of-00101.safetensors", + "model.layers.76.mlp.experts.13.down_proj.weight": "model-00083-of-00101.safetensors", + "model.layers.76.mlp.experts.14.gate_proj.weight": "model-00083-of-00101.safetensors", + "model.layers.76.mlp.experts.14.up_proj.weight": "model-00083-of-00101.safetensors", + "model.layers.76.mlp.experts.14.down_proj.weight": "model-00083-of-00101.safetensors", + "model.layers.76.mlp.experts.15.gate_proj.weight": "model-00083-of-00101.safetensors", + "model.layers.76.mlp.experts.15.up_proj.weight": "model-00083-of-00101.safetensors", + "model.layers.76.mlp.experts.15.down_proj.weight": "model-00083-of-00101.safetensors", + "model.layers.76.mlp.experts.16.gate_proj.weight": "model-00083-of-00101.safetensors", + "model.layers.76.mlp.experts.16.up_proj.weight": "model-00083-of-00101.safetensors", + "model.layers.76.mlp.experts.16.down_proj.weight": "model-00083-of-00101.safetensors", + "model.layers.76.mlp.experts.17.gate_proj.weight": "model-00083-of-00101.safetensors", + "model.layers.76.mlp.experts.17.up_proj.weight": "model-00083-of-00101.safetensors", + "model.layers.76.mlp.experts.17.down_proj.weight": "model-00083-of-00101.safetensors", + "model.layers.76.mlp.experts.18.gate_proj.weight": "model-00083-of-00101.safetensors", + "model.layers.76.mlp.experts.18.up_proj.weight": "model-00083-of-00101.safetensors", + "model.layers.76.mlp.experts.18.down_proj.weight": "model-00083-of-00101.safetensors", + "model.layers.76.mlp.experts.19.gate_proj.weight": "model-00083-of-00101.safetensors", + "model.layers.76.mlp.experts.19.up_proj.weight": "model-00083-of-00101.safetensors", + "model.layers.76.mlp.experts.19.down_proj.weight": "model-00083-of-00101.safetensors", + "model.layers.76.mlp.experts.20.gate_proj.weight": "model-00083-of-00101.safetensors", + "model.layers.76.mlp.experts.20.up_proj.weight": "model-00083-of-00101.safetensors", + "model.layers.76.mlp.experts.20.down_proj.weight": "model-00083-of-00101.safetensors", + "model.layers.76.mlp.experts.21.gate_proj.weight": "model-00083-of-00101.safetensors", + "model.layers.76.mlp.experts.21.up_proj.weight": "model-00083-of-00101.safetensors", + "model.layers.76.mlp.experts.21.down_proj.weight": "model-00083-of-00101.safetensors", + "model.layers.76.mlp.experts.22.gate_proj.weight": "model-00083-of-00101.safetensors", + "model.layers.76.mlp.experts.22.up_proj.weight": "model-00083-of-00101.safetensors", + "model.layers.76.mlp.experts.22.down_proj.weight": "model-00083-of-00101.safetensors", + "model.layers.76.mlp.experts.23.gate_proj.weight": "model-00083-of-00101.safetensors", + "model.layers.76.mlp.experts.23.up_proj.weight": "model-00083-of-00101.safetensors", + "model.layers.76.mlp.experts.23.down_proj.weight": "model-00083-of-00101.safetensors", + "model.layers.76.mlp.experts.24.gate_proj.weight": "model-00083-of-00101.safetensors", + "model.layers.76.mlp.experts.24.up_proj.weight": "model-00083-of-00101.safetensors", + "model.layers.76.mlp.experts.24.down_proj.weight": "model-00083-of-00101.safetensors", + "model.layers.76.mlp.experts.25.gate_proj.weight": "model-00083-of-00101.safetensors", + "model.layers.76.mlp.experts.25.up_proj.weight": "model-00083-of-00101.safetensors", + "model.layers.76.mlp.experts.25.down_proj.weight": "model-00083-of-00101.safetensors", + "model.layers.76.mlp.experts.26.gate_proj.weight": "model-00083-of-00101.safetensors", + "model.layers.76.mlp.experts.26.up_proj.weight": "model-00083-of-00101.safetensors", + "model.layers.76.mlp.experts.26.down_proj.weight": "model-00083-of-00101.safetensors", + "model.layers.76.mlp.experts.27.gate_proj.weight": "model-00083-of-00101.safetensors", + "model.layers.76.mlp.experts.27.up_proj.weight": "model-00083-of-00101.safetensors", + "model.layers.76.mlp.experts.27.down_proj.weight": "model-00083-of-00101.safetensors", + "model.layers.76.mlp.experts.28.gate_proj.weight": "model-00083-of-00101.safetensors", + "model.layers.76.mlp.experts.28.up_proj.weight": "model-00083-of-00101.safetensors", + "model.layers.76.mlp.experts.28.down_proj.weight": "model-00083-of-00101.safetensors", + "model.layers.76.mlp.experts.29.gate_proj.weight": "model-00083-of-00101.safetensors", + "model.layers.76.mlp.experts.29.up_proj.weight": "model-00083-of-00101.safetensors", + "model.layers.76.mlp.experts.29.down_proj.weight": "model-00083-of-00101.safetensors", + "model.layers.76.mlp.experts.30.gate_proj.weight": "model-00083-of-00101.safetensors", + "model.layers.76.mlp.experts.30.up_proj.weight": "model-00083-of-00101.safetensors", + "model.layers.76.mlp.experts.30.down_proj.weight": "model-00083-of-00101.safetensors", + "model.layers.76.mlp.experts.31.gate_proj.weight": "model-00083-of-00101.safetensors", + "model.layers.76.mlp.experts.31.up_proj.weight": "model-00083-of-00101.safetensors", + "model.layers.76.mlp.experts.31.down_proj.weight": "model-00083-of-00101.safetensors", + "model.layers.76.mlp.experts.32.gate_proj.weight": "model-00083-of-00101.safetensors", + "model.layers.76.mlp.experts.32.up_proj.weight": "model-00083-of-00101.safetensors", + "model.layers.76.mlp.experts.32.down_proj.weight": "model-00083-of-00101.safetensors", + "model.layers.76.mlp.experts.33.gate_proj.weight": "model-00083-of-00101.safetensors", + "model.layers.76.mlp.experts.33.up_proj.weight": "model-00083-of-00101.safetensors", + "model.layers.76.mlp.experts.33.down_proj.weight": "model-00083-of-00101.safetensors", + "model.layers.76.mlp.experts.34.gate_proj.weight": "model-00083-of-00101.safetensors", + "model.layers.76.mlp.experts.34.up_proj.weight": "model-00083-of-00101.safetensors", + "model.layers.76.mlp.experts.34.down_proj.weight": "model-00083-of-00101.safetensors", + "model.layers.76.mlp.experts.35.gate_proj.weight": "model-00083-of-00101.safetensors", + "model.layers.76.mlp.experts.35.up_proj.weight": "model-00083-of-00101.safetensors", + "model.layers.76.mlp.experts.35.down_proj.weight": "model-00083-of-00101.safetensors", + "model.layers.76.mlp.experts.36.gate_proj.weight": "model-00083-of-00101.safetensors", + "model.layers.76.mlp.experts.36.up_proj.weight": "model-00083-of-00101.safetensors", + "model.layers.76.mlp.experts.36.down_proj.weight": "model-00083-of-00101.safetensors", + "model.layers.76.mlp.experts.37.gate_proj.weight": "model-00083-of-00101.safetensors", + "model.layers.76.mlp.experts.37.up_proj.weight": "model-00083-of-00101.safetensors", + "model.layers.76.mlp.experts.37.down_proj.weight": "model-00083-of-00101.safetensors", + "model.layers.76.mlp.experts.38.gate_proj.weight": "model-00083-of-00101.safetensors", + "model.layers.76.mlp.experts.38.up_proj.weight": "model-00083-of-00101.safetensors", + "model.layers.76.mlp.experts.38.down_proj.weight": "model-00083-of-00101.safetensors", + "model.layers.76.mlp.experts.39.gate_proj.weight": "model-00083-of-00101.safetensors", + "model.layers.76.mlp.experts.39.up_proj.weight": "model-00083-of-00101.safetensors", + "model.layers.76.mlp.experts.39.down_proj.weight": "model-00083-of-00101.safetensors", + "model.layers.76.mlp.experts.40.gate_proj.weight": "model-00083-of-00101.safetensors", + "model.layers.76.mlp.experts.40.up_proj.weight": "model-00083-of-00101.safetensors", + "model.layers.76.mlp.experts.40.down_proj.weight": "model-00083-of-00101.safetensors", + "model.layers.76.mlp.experts.41.gate_proj.weight": "model-00083-of-00101.safetensors", + "model.layers.76.mlp.experts.41.up_proj.weight": "model-00083-of-00101.safetensors", + "model.layers.76.mlp.experts.41.down_proj.weight": "model-00083-of-00101.safetensors", + "model.layers.76.mlp.experts.42.gate_proj.weight": "model-00083-of-00101.safetensors", + "model.layers.76.mlp.experts.42.up_proj.weight": "model-00083-of-00101.safetensors", + "model.layers.76.mlp.experts.42.down_proj.weight": "model-00083-of-00101.safetensors", + "model.layers.76.mlp.experts.43.gate_proj.weight": "model-00083-of-00101.safetensors", + "model.layers.76.mlp.experts.43.up_proj.weight": "model-00083-of-00101.safetensors", + "model.layers.76.mlp.experts.43.down_proj.weight": "model-00083-of-00101.safetensors", + "model.layers.76.mlp.experts.44.gate_proj.weight": "model-00083-of-00101.safetensors", + "model.layers.76.mlp.experts.44.up_proj.weight": "model-00083-of-00101.safetensors", + "model.layers.76.mlp.experts.44.down_proj.weight": "model-00083-of-00101.safetensors", + "model.layers.76.mlp.experts.45.gate_proj.weight": "model-00083-of-00101.safetensors", + "model.layers.76.mlp.experts.45.up_proj.weight": "model-00083-of-00101.safetensors", + "model.layers.76.mlp.experts.45.down_proj.weight": "model-00083-of-00101.safetensors", + "model.layers.76.mlp.experts.46.gate_proj.weight": "model-00083-of-00101.safetensors", + "model.layers.76.mlp.experts.46.up_proj.weight": "model-00083-of-00101.safetensors", + "model.layers.76.mlp.experts.46.down_proj.weight": "model-00083-of-00101.safetensors", + "model.layers.76.mlp.experts.47.gate_proj.weight": "model-00083-of-00101.safetensors", + "model.layers.76.mlp.experts.47.up_proj.weight": "model-00083-of-00101.safetensors", + "model.layers.76.mlp.experts.47.down_proj.weight": "model-00083-of-00101.safetensors", + "model.layers.76.mlp.experts.48.gate_proj.weight": "model-00083-of-00101.safetensors", + "model.layers.76.mlp.experts.48.up_proj.weight": "model-00083-of-00101.safetensors", + "model.layers.76.mlp.experts.48.down_proj.weight": "model-00083-of-00101.safetensors", + "model.layers.76.mlp.experts.49.gate_proj.weight": "model-00083-of-00101.safetensors", + "model.layers.76.mlp.experts.49.up_proj.weight": "model-00083-of-00101.safetensors", + "model.layers.76.mlp.experts.49.down_proj.weight": "model-00083-of-00101.safetensors", + "model.layers.76.mlp.experts.50.gate_proj.weight": "model-00083-of-00101.safetensors", + "model.layers.76.mlp.experts.50.up_proj.weight": "model-00083-of-00101.safetensors", + "model.layers.76.mlp.experts.50.down_proj.weight": "model-00083-of-00101.safetensors", + "model.layers.76.mlp.experts.51.gate_proj.weight": "model-00083-of-00101.safetensors", + "model.layers.76.mlp.experts.51.up_proj.weight": "model-00083-of-00101.safetensors", + "model.layers.76.mlp.experts.51.down_proj.weight": "model-00083-of-00101.safetensors", + "model.layers.76.mlp.experts.52.gate_proj.weight": "model-00083-of-00101.safetensors", + "model.layers.76.mlp.experts.52.up_proj.weight": "model-00083-of-00101.safetensors", + "model.layers.76.mlp.experts.52.down_proj.weight": "model-00083-of-00101.safetensors", + "model.layers.76.mlp.experts.53.gate_proj.weight": "model-00083-of-00101.safetensors", + "model.layers.76.mlp.experts.53.up_proj.weight": "model-00083-of-00101.safetensors", + "model.layers.76.mlp.experts.53.down_proj.weight": "model-00083-of-00101.safetensors", + "model.layers.76.mlp.experts.54.gate_proj.weight": "model-00083-of-00101.safetensors", + "model.layers.76.mlp.experts.54.up_proj.weight": "model-00083-of-00101.safetensors", + "model.layers.76.mlp.experts.54.down_proj.weight": "model-00083-of-00101.safetensors", + "model.layers.76.mlp.experts.55.gate_proj.weight": "model-00083-of-00101.safetensors", + "model.layers.76.mlp.experts.55.up_proj.weight": "model-00083-of-00101.safetensors", + "model.layers.76.mlp.experts.55.down_proj.weight": "model-00083-of-00101.safetensors", + "model.layers.76.mlp.experts.56.gate_proj.weight": "model-00083-of-00101.safetensors", + "model.layers.76.mlp.experts.56.up_proj.weight": "model-00083-of-00101.safetensors", + "model.layers.76.mlp.experts.56.down_proj.weight": "model-00083-of-00101.safetensors", + "model.layers.76.mlp.experts.57.gate_proj.weight": "model-00083-of-00101.safetensors", + "model.layers.76.mlp.experts.57.up_proj.weight": "model-00083-of-00101.safetensors", + "model.layers.76.mlp.experts.57.down_proj.weight": "model-00083-of-00101.safetensors", + "model.layers.76.mlp.experts.58.gate_proj.weight": "model-00083-of-00101.safetensors", + "model.layers.76.mlp.experts.58.up_proj.weight": "model-00083-of-00101.safetensors", + "model.layers.76.mlp.experts.58.down_proj.weight": "model-00083-of-00101.safetensors", + "model.layers.76.mlp.experts.59.gate_proj.weight": "model-00083-of-00101.safetensors", + "model.layers.76.mlp.experts.59.up_proj.weight": "model-00083-of-00101.safetensors", + "model.layers.76.mlp.experts.59.down_proj.weight": "model-00083-of-00101.safetensors", + "model.layers.76.mlp.experts.60.gate_proj.weight": "model-00083-of-00101.safetensors", + "model.layers.76.mlp.experts.60.up_proj.weight": "model-00083-of-00101.safetensors", + "model.layers.76.mlp.experts.60.down_proj.weight": "model-00083-of-00101.safetensors", + "model.layers.76.mlp.experts.61.gate_proj.weight": "model-00083-of-00101.safetensors", + "model.layers.76.mlp.experts.61.up_proj.weight": "model-00083-of-00101.safetensors", + "model.layers.76.mlp.experts.61.down_proj.weight": "model-00083-of-00101.safetensors", + "model.layers.76.mlp.experts.62.gate_proj.weight": "model-00083-of-00101.safetensors", + "model.layers.76.mlp.experts.62.up_proj.weight": "model-00083-of-00101.safetensors", + "model.layers.76.mlp.experts.62.down_proj.weight": "model-00083-of-00101.safetensors", + "model.layers.76.mlp.experts.63.gate_proj.weight": "model-00083-of-00101.safetensors", + "model.layers.76.mlp.experts.63.up_proj.weight": "model-00083-of-00101.safetensors", + "model.layers.76.mlp.experts.63.down_proj.weight": "model-00083-of-00101.safetensors", + "model.layers.76.mlp.experts.64.gate_proj.weight": "model-00083-of-00101.safetensors", + "model.layers.76.mlp.experts.64.up_proj.weight": "model-00083-of-00101.safetensors", + "model.layers.76.mlp.experts.64.down_proj.weight": "model-00083-of-00101.safetensors", + "model.layers.76.mlp.experts.65.gate_proj.weight": "model-00083-of-00101.safetensors", + "model.layers.76.mlp.experts.65.up_proj.weight": "model-00083-of-00101.safetensors", + "model.layers.76.mlp.experts.65.down_proj.weight": "model-00083-of-00101.safetensors", + "model.layers.76.mlp.experts.66.gate_proj.weight": "model-00083-of-00101.safetensors", + "model.layers.76.mlp.experts.66.up_proj.weight": "model-00083-of-00101.safetensors", + "model.layers.76.mlp.experts.66.down_proj.weight": "model-00083-of-00101.safetensors", + "model.layers.76.mlp.experts.67.gate_proj.weight": "model-00083-of-00101.safetensors", + "model.layers.76.mlp.experts.67.up_proj.weight": "model-00083-of-00101.safetensors", + "model.layers.76.mlp.experts.67.down_proj.weight": "model-00083-of-00101.safetensors", + "model.layers.76.mlp.experts.68.gate_proj.weight": "model-00083-of-00101.safetensors", + "model.layers.76.mlp.experts.68.up_proj.weight": "model-00083-of-00101.safetensors", + "model.layers.76.mlp.experts.68.down_proj.weight": "model-00083-of-00101.safetensors", + "model.layers.76.mlp.experts.69.gate_proj.weight": "model-00083-of-00101.safetensors", + "model.layers.76.mlp.experts.69.up_proj.weight": "model-00083-of-00101.safetensors", + "model.layers.76.mlp.experts.69.down_proj.weight": "model-00083-of-00101.safetensors", + "model.layers.76.mlp.experts.70.gate_proj.weight": "model-00083-of-00101.safetensors", + "model.layers.76.mlp.experts.70.up_proj.weight": "model-00083-of-00101.safetensors", + "model.layers.76.mlp.experts.70.down_proj.weight": "model-00083-of-00101.safetensors", + "model.layers.76.mlp.experts.71.gate_proj.weight": "model-00083-of-00101.safetensors", + "model.layers.76.mlp.experts.71.up_proj.weight": "model-00083-of-00101.safetensors", + "model.layers.76.mlp.experts.71.down_proj.weight": "model-00083-of-00101.safetensors", + "model.layers.76.mlp.experts.72.gate_proj.weight": "model-00083-of-00101.safetensors", + "model.layers.76.mlp.experts.72.up_proj.weight": "model-00083-of-00101.safetensors", + "model.layers.76.mlp.experts.72.down_proj.weight": "model-00083-of-00101.safetensors", + "model.layers.76.mlp.experts.73.gate_proj.weight": "model-00083-of-00101.safetensors", + "model.layers.76.mlp.experts.73.up_proj.weight": "model-00083-of-00101.safetensors", + "model.layers.76.mlp.experts.73.down_proj.weight": "model-00083-of-00101.safetensors", + "model.layers.76.mlp.experts.74.gate_proj.weight": "model-00083-of-00101.safetensors", + "model.layers.76.mlp.experts.74.up_proj.weight": "model-00083-of-00101.safetensors", + "model.layers.76.mlp.experts.74.down_proj.weight": "model-00083-of-00101.safetensors", + "model.layers.76.mlp.experts.75.gate_proj.weight": "model-00083-of-00101.safetensors", + "model.layers.76.mlp.experts.75.up_proj.weight": "model-00083-of-00101.safetensors", + "model.layers.76.mlp.experts.75.down_proj.weight": "model-00083-of-00101.safetensors", + "model.layers.76.mlp.experts.76.gate_proj.weight": "model-00083-of-00101.safetensors", + "model.layers.76.mlp.experts.76.up_proj.weight": "model-00083-of-00101.safetensors", + "model.layers.76.mlp.experts.76.down_proj.weight": "model-00083-of-00101.safetensors", + "model.layers.76.mlp.experts.77.gate_proj.weight": "model-00083-of-00101.safetensors", + "model.layers.76.mlp.experts.77.up_proj.weight": "model-00083-of-00101.safetensors", + "model.layers.76.mlp.experts.77.down_proj.weight": "model-00083-of-00101.safetensors", + "model.layers.76.mlp.experts.78.gate_proj.weight": "model-00083-of-00101.safetensors", + "model.layers.76.mlp.experts.78.up_proj.weight": "model-00083-of-00101.safetensors", + "model.layers.76.mlp.experts.78.down_proj.weight": "model-00083-of-00101.safetensors", + "model.layers.76.mlp.experts.79.gate_proj.weight": "model-00083-of-00101.safetensors", + "model.layers.76.mlp.experts.79.up_proj.weight": "model-00083-of-00101.safetensors", + "model.layers.76.mlp.experts.79.down_proj.weight": "model-00083-of-00101.safetensors", + "model.layers.76.mlp.experts.80.gate_proj.weight": "model-00083-of-00101.safetensors", + "model.layers.76.mlp.experts.80.up_proj.weight": "model-00083-of-00101.safetensors", + "model.layers.76.mlp.experts.80.down_proj.weight": "model-00083-of-00101.safetensors", + "model.layers.76.mlp.experts.81.gate_proj.weight": "model-00084-of-00101.safetensors", + "model.layers.76.mlp.experts.81.up_proj.weight": "model-00084-of-00101.safetensors", + "model.layers.76.mlp.experts.81.down_proj.weight": "model-00084-of-00101.safetensors", + "model.layers.76.mlp.experts.82.gate_proj.weight": "model-00084-of-00101.safetensors", + "model.layers.76.mlp.experts.82.up_proj.weight": "model-00084-of-00101.safetensors", + "model.layers.76.mlp.experts.82.down_proj.weight": "model-00084-of-00101.safetensors", + "model.layers.76.mlp.experts.83.gate_proj.weight": "model-00084-of-00101.safetensors", + "model.layers.76.mlp.experts.83.up_proj.weight": "model-00084-of-00101.safetensors", + "model.layers.76.mlp.experts.83.down_proj.weight": "model-00084-of-00101.safetensors", + "model.layers.76.mlp.experts.84.gate_proj.weight": "model-00084-of-00101.safetensors", + "model.layers.76.mlp.experts.84.up_proj.weight": "model-00084-of-00101.safetensors", + "model.layers.76.mlp.experts.84.down_proj.weight": "model-00084-of-00101.safetensors", + "model.layers.76.mlp.experts.85.gate_proj.weight": "model-00084-of-00101.safetensors", + "model.layers.76.mlp.experts.85.up_proj.weight": "model-00084-of-00101.safetensors", + "model.layers.76.mlp.experts.85.down_proj.weight": "model-00084-of-00101.safetensors", + "model.layers.76.mlp.experts.86.gate_proj.weight": "model-00084-of-00101.safetensors", + "model.layers.76.mlp.experts.86.up_proj.weight": "model-00084-of-00101.safetensors", + "model.layers.76.mlp.experts.86.down_proj.weight": "model-00084-of-00101.safetensors", + "model.layers.76.mlp.experts.87.gate_proj.weight": "model-00084-of-00101.safetensors", + "model.layers.76.mlp.experts.87.up_proj.weight": "model-00084-of-00101.safetensors", + "model.layers.76.mlp.experts.87.down_proj.weight": "model-00084-of-00101.safetensors", + "model.layers.76.mlp.experts.88.gate_proj.weight": "model-00084-of-00101.safetensors", + "model.layers.76.mlp.experts.88.up_proj.weight": "model-00084-of-00101.safetensors", + "model.layers.76.mlp.experts.88.down_proj.weight": "model-00084-of-00101.safetensors", + "model.layers.76.mlp.experts.89.gate_proj.weight": "model-00084-of-00101.safetensors", + "model.layers.76.mlp.experts.89.up_proj.weight": "model-00084-of-00101.safetensors", + "model.layers.76.mlp.experts.89.down_proj.weight": "model-00084-of-00101.safetensors", + "model.layers.76.mlp.experts.90.gate_proj.weight": "model-00084-of-00101.safetensors", + "model.layers.76.mlp.experts.90.up_proj.weight": "model-00084-of-00101.safetensors", + "model.layers.76.mlp.experts.90.down_proj.weight": "model-00084-of-00101.safetensors", + "model.layers.76.mlp.experts.91.gate_proj.weight": "model-00084-of-00101.safetensors", + "model.layers.76.mlp.experts.91.up_proj.weight": "model-00084-of-00101.safetensors", + "model.layers.76.mlp.experts.91.down_proj.weight": "model-00084-of-00101.safetensors", + "model.layers.76.mlp.experts.92.gate_proj.weight": "model-00084-of-00101.safetensors", + "model.layers.76.mlp.experts.92.up_proj.weight": "model-00084-of-00101.safetensors", + "model.layers.76.mlp.experts.92.down_proj.weight": "model-00084-of-00101.safetensors", + "model.layers.76.mlp.experts.93.gate_proj.weight": "model-00084-of-00101.safetensors", + "model.layers.76.mlp.experts.93.up_proj.weight": "model-00084-of-00101.safetensors", + "model.layers.76.mlp.experts.93.down_proj.weight": "model-00084-of-00101.safetensors", + "model.layers.76.mlp.experts.94.gate_proj.weight": "model-00084-of-00101.safetensors", + "model.layers.76.mlp.experts.94.up_proj.weight": "model-00084-of-00101.safetensors", + "model.layers.76.mlp.experts.94.down_proj.weight": "model-00084-of-00101.safetensors", + "model.layers.76.mlp.experts.95.gate_proj.weight": "model-00084-of-00101.safetensors", + "model.layers.76.mlp.experts.95.up_proj.weight": "model-00084-of-00101.safetensors", + "model.layers.76.mlp.experts.95.down_proj.weight": "model-00084-of-00101.safetensors", + "model.layers.76.mlp.experts.96.gate_proj.weight": "model-00084-of-00101.safetensors", + "model.layers.76.mlp.experts.96.up_proj.weight": "model-00084-of-00101.safetensors", + "model.layers.76.mlp.experts.96.down_proj.weight": "model-00084-of-00101.safetensors", + "model.layers.76.mlp.experts.97.gate_proj.weight": "model-00084-of-00101.safetensors", + "model.layers.76.mlp.experts.97.up_proj.weight": "model-00084-of-00101.safetensors", + "model.layers.76.mlp.experts.97.down_proj.weight": "model-00084-of-00101.safetensors", + "model.layers.76.mlp.experts.98.gate_proj.weight": "model-00084-of-00101.safetensors", + "model.layers.76.mlp.experts.98.up_proj.weight": "model-00084-of-00101.safetensors", + "model.layers.76.mlp.experts.98.down_proj.weight": "model-00084-of-00101.safetensors", + "model.layers.76.mlp.experts.99.gate_proj.weight": "model-00084-of-00101.safetensors", + "model.layers.76.mlp.experts.99.up_proj.weight": "model-00084-of-00101.safetensors", + "model.layers.76.mlp.experts.99.down_proj.weight": "model-00084-of-00101.safetensors", + "model.layers.76.mlp.experts.100.gate_proj.weight": "model-00084-of-00101.safetensors", + "model.layers.76.mlp.experts.100.up_proj.weight": "model-00084-of-00101.safetensors", + "model.layers.76.mlp.experts.100.down_proj.weight": "model-00084-of-00101.safetensors", + "model.layers.76.mlp.experts.101.gate_proj.weight": "model-00084-of-00101.safetensors", + "model.layers.76.mlp.experts.101.up_proj.weight": "model-00084-of-00101.safetensors", + "model.layers.76.mlp.experts.101.down_proj.weight": "model-00084-of-00101.safetensors", + "model.layers.76.mlp.experts.102.gate_proj.weight": "model-00084-of-00101.safetensors", + "model.layers.76.mlp.experts.102.up_proj.weight": "model-00084-of-00101.safetensors", + "model.layers.76.mlp.experts.102.down_proj.weight": "model-00084-of-00101.safetensors", + "model.layers.76.mlp.experts.103.gate_proj.weight": "model-00084-of-00101.safetensors", + "model.layers.76.mlp.experts.103.up_proj.weight": "model-00084-of-00101.safetensors", + "model.layers.76.mlp.experts.103.down_proj.weight": "model-00084-of-00101.safetensors", + "model.layers.76.mlp.experts.104.gate_proj.weight": "model-00084-of-00101.safetensors", + "model.layers.76.mlp.experts.104.up_proj.weight": "model-00084-of-00101.safetensors", + "model.layers.76.mlp.experts.104.down_proj.weight": "model-00084-of-00101.safetensors", + "model.layers.76.mlp.experts.105.gate_proj.weight": "model-00084-of-00101.safetensors", + "model.layers.76.mlp.experts.105.up_proj.weight": "model-00084-of-00101.safetensors", + "model.layers.76.mlp.experts.105.down_proj.weight": "model-00084-of-00101.safetensors", + "model.layers.76.mlp.experts.106.gate_proj.weight": "model-00084-of-00101.safetensors", + "model.layers.76.mlp.experts.106.up_proj.weight": "model-00084-of-00101.safetensors", + "model.layers.76.mlp.experts.106.down_proj.weight": "model-00084-of-00101.safetensors", + "model.layers.76.mlp.experts.107.gate_proj.weight": "model-00084-of-00101.safetensors", + "model.layers.76.mlp.experts.107.up_proj.weight": "model-00084-of-00101.safetensors", + "model.layers.76.mlp.experts.107.down_proj.weight": "model-00084-of-00101.safetensors", + "model.layers.76.mlp.experts.108.gate_proj.weight": "model-00084-of-00101.safetensors", + "model.layers.76.mlp.experts.108.up_proj.weight": "model-00084-of-00101.safetensors", + "model.layers.76.mlp.experts.108.down_proj.weight": "model-00084-of-00101.safetensors", + "model.layers.76.mlp.experts.109.gate_proj.weight": "model-00084-of-00101.safetensors", + "model.layers.76.mlp.experts.109.up_proj.weight": "model-00084-of-00101.safetensors", + "model.layers.76.mlp.experts.109.down_proj.weight": "model-00084-of-00101.safetensors", + "model.layers.76.mlp.experts.110.gate_proj.weight": "model-00084-of-00101.safetensors", + "model.layers.76.mlp.experts.110.up_proj.weight": "model-00084-of-00101.safetensors", + "model.layers.76.mlp.experts.110.down_proj.weight": "model-00084-of-00101.safetensors", + "model.layers.76.mlp.experts.111.gate_proj.weight": "model-00084-of-00101.safetensors", + "model.layers.76.mlp.experts.111.up_proj.weight": "model-00084-of-00101.safetensors", + "model.layers.76.mlp.experts.111.down_proj.weight": "model-00084-of-00101.safetensors", + "model.layers.76.mlp.experts.112.gate_proj.weight": "model-00084-of-00101.safetensors", + "model.layers.76.mlp.experts.112.up_proj.weight": "model-00084-of-00101.safetensors", + "model.layers.76.mlp.experts.112.down_proj.weight": "model-00084-of-00101.safetensors", + "model.layers.76.mlp.experts.113.gate_proj.weight": "model-00084-of-00101.safetensors", + "model.layers.76.mlp.experts.113.up_proj.weight": "model-00084-of-00101.safetensors", + "model.layers.76.mlp.experts.113.down_proj.weight": "model-00084-of-00101.safetensors", + "model.layers.76.mlp.experts.114.gate_proj.weight": "model-00084-of-00101.safetensors", + "model.layers.76.mlp.experts.114.up_proj.weight": "model-00084-of-00101.safetensors", + "model.layers.76.mlp.experts.114.down_proj.weight": "model-00084-of-00101.safetensors", + "model.layers.76.mlp.experts.115.gate_proj.weight": "model-00084-of-00101.safetensors", + "model.layers.76.mlp.experts.115.up_proj.weight": "model-00084-of-00101.safetensors", + "model.layers.76.mlp.experts.115.down_proj.weight": "model-00084-of-00101.safetensors", + "model.layers.76.mlp.experts.116.gate_proj.weight": "model-00084-of-00101.safetensors", + "model.layers.76.mlp.experts.116.up_proj.weight": "model-00084-of-00101.safetensors", + "model.layers.76.mlp.experts.116.down_proj.weight": "model-00084-of-00101.safetensors", + "model.layers.76.mlp.experts.117.gate_proj.weight": "model-00084-of-00101.safetensors", + "model.layers.76.mlp.experts.117.up_proj.weight": "model-00084-of-00101.safetensors", + "model.layers.76.mlp.experts.117.down_proj.weight": "model-00084-of-00101.safetensors", + "model.layers.76.mlp.experts.118.gate_proj.weight": "model-00084-of-00101.safetensors", + "model.layers.76.mlp.experts.118.up_proj.weight": "model-00084-of-00101.safetensors", + "model.layers.76.mlp.experts.118.down_proj.weight": "model-00084-of-00101.safetensors", + "model.layers.76.mlp.experts.119.gate_proj.weight": "model-00084-of-00101.safetensors", + "model.layers.76.mlp.experts.119.up_proj.weight": "model-00084-of-00101.safetensors", + "model.layers.76.mlp.experts.119.down_proj.weight": "model-00084-of-00101.safetensors", + "model.layers.76.mlp.gate.weight": "model-00084-of-00101.safetensors", + "model.layers.76.mlp.gate.e_score_correction_bias": "model-00084-of-00101.safetensors", + "model.layers.76.mlp.shared_experts.gate_proj.weight": "model-00084-of-00101.safetensors", + "model.layers.76.mlp.shared_experts.up_proj.weight": "model-00084-of-00101.safetensors", + "model.layers.76.mlp.shared_experts.down_proj.weight": "model-00084-of-00101.safetensors", + "model.layers.76.input_layernorm.weight": "model-00084-of-00101.safetensors", + "model.layers.76.post_attention_layernorm.weight": "model-00084-of-00101.safetensors", + "model.layers.77.self_attn.q_proj.weight": "model-00084-of-00101.safetensors", + "model.layers.77.self_attn.q_proj.bias": "model-00084-of-00101.safetensors", + "model.layers.77.self_attn.k_proj.weight": "model-00084-of-00101.safetensors", + "model.layers.77.self_attn.k_proj.bias": "model-00084-of-00101.safetensors", + "model.layers.77.self_attn.v_proj.weight": "model-00084-of-00101.safetensors", + "model.layers.77.self_attn.v_proj.bias": "model-00084-of-00101.safetensors", + "model.layers.77.self_attn.o_proj.weight": "model-00084-of-00101.safetensors", + "model.layers.77.self_attn.q_norm.weight": "model-00084-of-00101.safetensors", + "model.layers.77.self_attn.k_norm.weight": "model-00084-of-00101.safetensors", + "model.layers.77.mlp.experts.0.gate_proj.weight": "model-00084-of-00101.safetensors", + "model.layers.77.mlp.experts.0.up_proj.weight": "model-00084-of-00101.safetensors", + "model.layers.77.mlp.experts.0.down_proj.weight": "model-00084-of-00101.safetensors", + "model.layers.77.mlp.experts.1.gate_proj.weight": "model-00084-of-00101.safetensors", + "model.layers.77.mlp.experts.1.up_proj.weight": "model-00084-of-00101.safetensors", + "model.layers.77.mlp.experts.1.down_proj.weight": "model-00084-of-00101.safetensors", + "model.layers.77.mlp.experts.2.gate_proj.weight": "model-00084-of-00101.safetensors", + "model.layers.77.mlp.experts.2.up_proj.weight": "model-00084-of-00101.safetensors", + "model.layers.77.mlp.experts.2.down_proj.weight": "model-00084-of-00101.safetensors", + "model.layers.77.mlp.experts.3.gate_proj.weight": "model-00084-of-00101.safetensors", + "model.layers.77.mlp.experts.3.up_proj.weight": "model-00084-of-00101.safetensors", + "model.layers.77.mlp.experts.3.down_proj.weight": "model-00084-of-00101.safetensors", + "model.layers.77.mlp.experts.4.gate_proj.weight": "model-00084-of-00101.safetensors", + "model.layers.77.mlp.experts.4.up_proj.weight": "model-00084-of-00101.safetensors", + "model.layers.77.mlp.experts.4.down_proj.weight": "model-00084-of-00101.safetensors", + "model.layers.77.mlp.experts.5.gate_proj.weight": "model-00084-of-00101.safetensors", + "model.layers.77.mlp.experts.5.up_proj.weight": "model-00084-of-00101.safetensors", + "model.layers.77.mlp.experts.5.down_proj.weight": "model-00084-of-00101.safetensors", + "model.layers.77.mlp.experts.6.gate_proj.weight": "model-00084-of-00101.safetensors", + "model.layers.77.mlp.experts.6.up_proj.weight": "model-00084-of-00101.safetensors", + "model.layers.77.mlp.experts.6.down_proj.weight": "model-00084-of-00101.safetensors", + "model.layers.77.mlp.experts.7.gate_proj.weight": "model-00084-of-00101.safetensors", + "model.layers.77.mlp.experts.7.up_proj.weight": "model-00084-of-00101.safetensors", + "model.layers.77.mlp.experts.7.down_proj.weight": "model-00084-of-00101.safetensors", + "model.layers.77.mlp.experts.8.gate_proj.weight": "model-00084-of-00101.safetensors", + "model.layers.77.mlp.experts.8.up_proj.weight": "model-00084-of-00101.safetensors", + "model.layers.77.mlp.experts.8.down_proj.weight": "model-00084-of-00101.safetensors", + "model.layers.77.mlp.experts.9.gate_proj.weight": "model-00084-of-00101.safetensors", + "model.layers.77.mlp.experts.9.up_proj.weight": "model-00084-of-00101.safetensors", + "model.layers.77.mlp.experts.9.down_proj.weight": "model-00084-of-00101.safetensors", + "model.layers.77.mlp.experts.10.gate_proj.weight": "model-00084-of-00101.safetensors", + "model.layers.77.mlp.experts.10.up_proj.weight": "model-00084-of-00101.safetensors", + "model.layers.77.mlp.experts.10.down_proj.weight": "model-00084-of-00101.safetensors", + "model.layers.77.mlp.experts.11.gate_proj.weight": "model-00084-of-00101.safetensors", + "model.layers.77.mlp.experts.11.up_proj.weight": "model-00084-of-00101.safetensors", + "model.layers.77.mlp.experts.11.down_proj.weight": "model-00084-of-00101.safetensors", + "model.layers.77.mlp.experts.12.gate_proj.weight": "model-00084-of-00101.safetensors", + "model.layers.77.mlp.experts.12.up_proj.weight": "model-00084-of-00101.safetensors", + "model.layers.77.mlp.experts.12.down_proj.weight": "model-00084-of-00101.safetensors", + "model.layers.77.mlp.experts.13.gate_proj.weight": "model-00084-of-00101.safetensors", + "model.layers.77.mlp.experts.13.up_proj.weight": "model-00084-of-00101.safetensors", + "model.layers.77.mlp.experts.13.down_proj.weight": "model-00084-of-00101.safetensors", + "model.layers.77.mlp.experts.14.gate_proj.weight": "model-00084-of-00101.safetensors", + "model.layers.77.mlp.experts.14.up_proj.weight": "model-00084-of-00101.safetensors", + "model.layers.77.mlp.experts.14.down_proj.weight": "model-00084-of-00101.safetensors", + "model.layers.77.mlp.experts.15.gate_proj.weight": "model-00084-of-00101.safetensors", + "model.layers.77.mlp.experts.15.up_proj.weight": "model-00084-of-00101.safetensors", + "model.layers.77.mlp.experts.15.down_proj.weight": "model-00084-of-00101.safetensors", + "model.layers.77.mlp.experts.16.gate_proj.weight": "model-00084-of-00101.safetensors", + "model.layers.77.mlp.experts.16.up_proj.weight": "model-00084-of-00101.safetensors", + "model.layers.77.mlp.experts.16.down_proj.weight": "model-00084-of-00101.safetensors", + "model.layers.77.mlp.experts.17.gate_proj.weight": "model-00084-of-00101.safetensors", + "model.layers.77.mlp.experts.17.up_proj.weight": "model-00084-of-00101.safetensors", + "model.layers.77.mlp.experts.17.down_proj.weight": "model-00084-of-00101.safetensors", + "model.layers.77.mlp.experts.18.gate_proj.weight": "model-00084-of-00101.safetensors", + "model.layers.77.mlp.experts.18.up_proj.weight": "model-00084-of-00101.safetensors", + "model.layers.77.mlp.experts.18.down_proj.weight": "model-00084-of-00101.safetensors", + "model.layers.77.mlp.experts.19.gate_proj.weight": "model-00084-of-00101.safetensors", + "model.layers.77.mlp.experts.19.up_proj.weight": "model-00084-of-00101.safetensors", + "model.layers.77.mlp.experts.19.down_proj.weight": "model-00084-of-00101.safetensors", + "model.layers.77.mlp.experts.20.gate_proj.weight": "model-00084-of-00101.safetensors", + "model.layers.77.mlp.experts.20.up_proj.weight": "model-00084-of-00101.safetensors", + "model.layers.77.mlp.experts.20.down_proj.weight": "model-00084-of-00101.safetensors", + "model.layers.77.mlp.experts.21.gate_proj.weight": "model-00084-of-00101.safetensors", + "model.layers.77.mlp.experts.21.up_proj.weight": "model-00084-of-00101.safetensors", + "model.layers.77.mlp.experts.21.down_proj.weight": "model-00084-of-00101.safetensors", + "model.layers.77.mlp.experts.22.gate_proj.weight": "model-00084-of-00101.safetensors", + "model.layers.77.mlp.experts.22.up_proj.weight": "model-00084-of-00101.safetensors", + "model.layers.77.mlp.experts.22.down_proj.weight": "model-00084-of-00101.safetensors", + "model.layers.77.mlp.experts.23.gate_proj.weight": "model-00084-of-00101.safetensors", + "model.layers.77.mlp.experts.23.up_proj.weight": "model-00084-of-00101.safetensors", + "model.layers.77.mlp.experts.23.down_proj.weight": "model-00084-of-00101.safetensors", + "model.layers.77.mlp.experts.24.gate_proj.weight": "model-00084-of-00101.safetensors", + "model.layers.77.mlp.experts.24.up_proj.weight": "model-00084-of-00101.safetensors", + "model.layers.77.mlp.experts.24.down_proj.weight": "model-00084-of-00101.safetensors", + "model.layers.77.mlp.experts.25.gate_proj.weight": "model-00084-of-00101.safetensors", + "model.layers.77.mlp.experts.25.up_proj.weight": "model-00084-of-00101.safetensors", + "model.layers.77.mlp.experts.25.down_proj.weight": "model-00084-of-00101.safetensors", + "model.layers.77.mlp.experts.26.gate_proj.weight": "model-00084-of-00101.safetensors", + "model.layers.77.mlp.experts.26.up_proj.weight": "model-00084-of-00101.safetensors", + "model.layers.77.mlp.experts.26.down_proj.weight": "model-00084-of-00101.safetensors", + "model.layers.77.mlp.experts.27.gate_proj.weight": "model-00084-of-00101.safetensors", + "model.layers.77.mlp.experts.27.up_proj.weight": "model-00084-of-00101.safetensors", + "model.layers.77.mlp.experts.27.down_proj.weight": "model-00084-of-00101.safetensors", + "model.layers.77.mlp.experts.28.gate_proj.weight": "model-00084-of-00101.safetensors", + "model.layers.77.mlp.experts.28.up_proj.weight": "model-00084-of-00101.safetensors", + "model.layers.77.mlp.experts.28.down_proj.weight": "model-00084-of-00101.safetensors", + "model.layers.77.mlp.experts.29.gate_proj.weight": "model-00084-of-00101.safetensors", + "model.layers.77.mlp.experts.29.up_proj.weight": "model-00084-of-00101.safetensors", + "model.layers.77.mlp.experts.29.down_proj.weight": "model-00084-of-00101.safetensors", + "model.layers.77.mlp.experts.30.gate_proj.weight": "model-00084-of-00101.safetensors", + "model.layers.77.mlp.experts.30.up_proj.weight": "model-00084-of-00101.safetensors", + "model.layers.77.mlp.experts.30.down_proj.weight": "model-00084-of-00101.safetensors", + "model.layers.77.mlp.experts.31.gate_proj.weight": "model-00084-of-00101.safetensors", + "model.layers.77.mlp.experts.31.up_proj.weight": "model-00084-of-00101.safetensors", + "model.layers.77.mlp.experts.31.down_proj.weight": "model-00084-of-00101.safetensors", + "model.layers.77.mlp.experts.32.gate_proj.weight": "model-00084-of-00101.safetensors", + "model.layers.77.mlp.experts.32.up_proj.weight": "model-00084-of-00101.safetensors", + "model.layers.77.mlp.experts.32.down_proj.weight": "model-00084-of-00101.safetensors", + "model.layers.77.mlp.experts.33.gate_proj.weight": "model-00084-of-00101.safetensors", + "model.layers.77.mlp.experts.33.up_proj.weight": "model-00084-of-00101.safetensors", + "model.layers.77.mlp.experts.33.down_proj.weight": "model-00084-of-00101.safetensors", + "model.layers.77.mlp.experts.34.gate_proj.weight": "model-00084-of-00101.safetensors", + "model.layers.77.mlp.experts.34.up_proj.weight": "model-00084-of-00101.safetensors", + "model.layers.77.mlp.experts.34.down_proj.weight": "model-00084-of-00101.safetensors", + "model.layers.77.mlp.experts.35.gate_proj.weight": "model-00084-of-00101.safetensors", + "model.layers.77.mlp.experts.35.up_proj.weight": "model-00084-of-00101.safetensors", + "model.layers.77.mlp.experts.35.down_proj.weight": "model-00084-of-00101.safetensors", + "model.layers.77.mlp.experts.36.gate_proj.weight": "model-00084-of-00101.safetensors", + "model.layers.77.mlp.experts.36.up_proj.weight": "model-00084-of-00101.safetensors", + "model.layers.77.mlp.experts.36.down_proj.weight": "model-00084-of-00101.safetensors", + "model.layers.77.mlp.experts.37.gate_proj.weight": "model-00084-of-00101.safetensors", + "model.layers.77.mlp.experts.37.up_proj.weight": "model-00084-of-00101.safetensors", + "model.layers.77.mlp.experts.37.down_proj.weight": "model-00084-of-00101.safetensors", + "model.layers.77.mlp.experts.38.gate_proj.weight": "model-00084-of-00101.safetensors", + "model.layers.77.mlp.experts.38.up_proj.weight": "model-00084-of-00101.safetensors", + "model.layers.77.mlp.experts.38.down_proj.weight": "model-00084-of-00101.safetensors", + "model.layers.77.mlp.experts.39.gate_proj.weight": "model-00084-of-00101.safetensors", + "model.layers.77.mlp.experts.39.up_proj.weight": "model-00084-of-00101.safetensors", + "model.layers.77.mlp.experts.39.down_proj.weight": "model-00084-of-00101.safetensors", + "model.layers.77.mlp.experts.40.gate_proj.weight": "model-00084-of-00101.safetensors", + "model.layers.77.mlp.experts.40.up_proj.weight": "model-00084-of-00101.safetensors", + "model.layers.77.mlp.experts.40.down_proj.weight": "model-00084-of-00101.safetensors", + "model.layers.77.mlp.experts.41.gate_proj.weight": "model-00084-of-00101.safetensors", + "model.layers.77.mlp.experts.41.up_proj.weight": "model-00084-of-00101.safetensors", + "model.layers.77.mlp.experts.41.down_proj.weight": "model-00084-of-00101.safetensors", + "model.layers.77.mlp.experts.42.gate_proj.weight": "model-00084-of-00101.safetensors", + "model.layers.77.mlp.experts.42.up_proj.weight": "model-00084-of-00101.safetensors", + "model.layers.77.mlp.experts.42.down_proj.weight": "model-00084-of-00101.safetensors", + "model.layers.77.mlp.experts.43.gate_proj.weight": "model-00084-of-00101.safetensors", + "model.layers.77.mlp.experts.43.up_proj.weight": "model-00084-of-00101.safetensors", + "model.layers.77.mlp.experts.43.down_proj.weight": "model-00084-of-00101.safetensors", + "model.layers.77.mlp.experts.44.gate_proj.weight": "model-00084-of-00101.safetensors", + "model.layers.77.mlp.experts.44.up_proj.weight": "model-00084-of-00101.safetensors", + "model.layers.77.mlp.experts.44.down_proj.weight": "model-00084-of-00101.safetensors", + "model.layers.77.mlp.experts.45.gate_proj.weight": "model-00084-of-00101.safetensors", + "model.layers.77.mlp.experts.45.up_proj.weight": "model-00084-of-00101.safetensors", + "model.layers.77.mlp.experts.45.down_proj.weight": "model-00084-of-00101.safetensors", + "model.layers.77.mlp.experts.46.gate_proj.weight": "model-00084-of-00101.safetensors", + "model.layers.77.mlp.experts.46.up_proj.weight": "model-00084-of-00101.safetensors", + "model.layers.77.mlp.experts.46.down_proj.weight": "model-00084-of-00101.safetensors", + "model.layers.77.mlp.experts.47.gate_proj.weight": "model-00084-of-00101.safetensors", + "model.layers.77.mlp.experts.47.up_proj.weight": "model-00084-of-00101.safetensors", + "model.layers.77.mlp.experts.47.down_proj.weight": "model-00084-of-00101.safetensors", + "model.layers.77.mlp.experts.48.gate_proj.weight": "model-00084-of-00101.safetensors", + "model.layers.77.mlp.experts.48.up_proj.weight": "model-00084-of-00101.safetensors", + "model.layers.77.mlp.experts.48.down_proj.weight": "model-00084-of-00101.safetensors", + "model.layers.77.mlp.experts.49.gate_proj.weight": "model-00084-of-00101.safetensors", + "model.layers.77.mlp.experts.49.up_proj.weight": "model-00084-of-00101.safetensors", + "model.layers.77.mlp.experts.49.down_proj.weight": "model-00084-of-00101.safetensors", + "model.layers.77.mlp.experts.50.gate_proj.weight": "model-00084-of-00101.safetensors", + "model.layers.77.mlp.experts.50.up_proj.weight": "model-00084-of-00101.safetensors", + "model.layers.77.mlp.experts.50.down_proj.weight": "model-00084-of-00101.safetensors", + "model.layers.77.mlp.experts.51.gate_proj.weight": "model-00084-of-00101.safetensors", + "model.layers.77.mlp.experts.51.up_proj.weight": "model-00084-of-00101.safetensors", + "model.layers.77.mlp.experts.51.down_proj.weight": "model-00084-of-00101.safetensors", + "model.layers.77.mlp.experts.52.gate_proj.weight": "model-00084-of-00101.safetensors", + "model.layers.77.mlp.experts.52.up_proj.weight": "model-00084-of-00101.safetensors", + "model.layers.77.mlp.experts.52.down_proj.weight": "model-00084-of-00101.safetensors", + "model.layers.77.mlp.experts.53.gate_proj.weight": "model-00084-of-00101.safetensors", + "model.layers.77.mlp.experts.53.up_proj.weight": "model-00084-of-00101.safetensors", + "model.layers.77.mlp.experts.53.down_proj.weight": "model-00084-of-00101.safetensors", + "model.layers.77.mlp.experts.54.gate_proj.weight": "model-00084-of-00101.safetensors", + "model.layers.77.mlp.experts.54.up_proj.weight": "model-00084-of-00101.safetensors", + "model.layers.77.mlp.experts.54.down_proj.weight": "model-00084-of-00101.safetensors", + "model.layers.77.mlp.experts.55.gate_proj.weight": "model-00084-of-00101.safetensors", + "model.layers.77.mlp.experts.55.up_proj.weight": "model-00084-of-00101.safetensors", + "model.layers.77.mlp.experts.55.down_proj.weight": "model-00084-of-00101.safetensors", + "model.layers.77.mlp.experts.56.gate_proj.weight": "model-00084-of-00101.safetensors", + "model.layers.77.mlp.experts.56.up_proj.weight": "model-00084-of-00101.safetensors", + "model.layers.77.mlp.experts.56.down_proj.weight": "model-00084-of-00101.safetensors", + "model.layers.77.mlp.experts.57.gate_proj.weight": "model-00084-of-00101.safetensors", + "model.layers.77.mlp.experts.57.up_proj.weight": "model-00084-of-00101.safetensors", + "model.layers.77.mlp.experts.57.down_proj.weight": "model-00084-of-00101.safetensors", + "model.layers.77.mlp.experts.58.gate_proj.weight": "model-00084-of-00101.safetensors", + "model.layers.77.mlp.experts.58.up_proj.weight": "model-00084-of-00101.safetensors", + "model.layers.77.mlp.experts.58.down_proj.weight": "model-00084-of-00101.safetensors", + "model.layers.77.mlp.experts.59.gate_proj.weight": "model-00084-of-00101.safetensors", + "model.layers.77.mlp.experts.59.up_proj.weight": "model-00084-of-00101.safetensors", + "model.layers.77.mlp.experts.59.down_proj.weight": "model-00084-of-00101.safetensors", + "model.layers.77.mlp.experts.60.gate_proj.weight": "model-00084-of-00101.safetensors", + "model.layers.77.mlp.experts.60.up_proj.weight": "model-00084-of-00101.safetensors", + "model.layers.77.mlp.experts.60.down_proj.weight": "model-00084-of-00101.safetensors", + "model.layers.77.mlp.experts.61.gate_proj.weight": "model-00084-of-00101.safetensors", + "model.layers.77.mlp.experts.61.up_proj.weight": "model-00084-of-00101.safetensors", + "model.layers.77.mlp.experts.61.down_proj.weight": "model-00084-of-00101.safetensors", + "model.layers.77.mlp.experts.62.gate_proj.weight": "model-00084-of-00101.safetensors", + "model.layers.77.mlp.experts.62.up_proj.weight": "model-00084-of-00101.safetensors", + "model.layers.77.mlp.experts.62.down_proj.weight": "model-00084-of-00101.safetensors", + "model.layers.77.mlp.experts.63.gate_proj.weight": "model-00084-of-00101.safetensors", + "model.layers.77.mlp.experts.63.up_proj.weight": "model-00084-of-00101.safetensors", + "model.layers.77.mlp.experts.63.down_proj.weight": "model-00084-of-00101.safetensors", + "model.layers.77.mlp.experts.64.gate_proj.weight": "model-00084-of-00101.safetensors", + "model.layers.77.mlp.experts.64.up_proj.weight": "model-00084-of-00101.safetensors", + "model.layers.77.mlp.experts.64.down_proj.weight": "model-00084-of-00101.safetensors", + "model.layers.77.mlp.experts.65.gate_proj.weight": "model-00084-of-00101.safetensors", + "model.layers.77.mlp.experts.65.up_proj.weight": "model-00084-of-00101.safetensors", + "model.layers.77.mlp.experts.65.down_proj.weight": "model-00084-of-00101.safetensors", + "model.layers.77.mlp.experts.66.gate_proj.weight": "model-00084-of-00101.safetensors", + "model.layers.77.mlp.experts.66.up_proj.weight": "model-00084-of-00101.safetensors", + "model.layers.77.mlp.experts.66.down_proj.weight": "model-00084-of-00101.safetensors", + "model.layers.77.mlp.experts.67.gate_proj.weight": "model-00084-of-00101.safetensors", + "model.layers.77.mlp.experts.67.up_proj.weight": "model-00084-of-00101.safetensors", + "model.layers.77.mlp.experts.67.down_proj.weight": "model-00085-of-00101.safetensors", + "model.layers.77.mlp.experts.68.gate_proj.weight": "model-00085-of-00101.safetensors", + "model.layers.77.mlp.experts.68.up_proj.weight": "model-00085-of-00101.safetensors", + "model.layers.77.mlp.experts.68.down_proj.weight": "model-00085-of-00101.safetensors", + "model.layers.77.mlp.experts.69.gate_proj.weight": "model-00085-of-00101.safetensors", + "model.layers.77.mlp.experts.69.up_proj.weight": "model-00085-of-00101.safetensors", + "model.layers.77.mlp.experts.69.down_proj.weight": "model-00085-of-00101.safetensors", + "model.layers.77.mlp.experts.70.gate_proj.weight": "model-00085-of-00101.safetensors", + "model.layers.77.mlp.experts.70.up_proj.weight": "model-00085-of-00101.safetensors", + "model.layers.77.mlp.experts.70.down_proj.weight": "model-00085-of-00101.safetensors", + "model.layers.77.mlp.experts.71.gate_proj.weight": "model-00085-of-00101.safetensors", + "model.layers.77.mlp.experts.71.up_proj.weight": "model-00085-of-00101.safetensors", + "model.layers.77.mlp.experts.71.down_proj.weight": "model-00085-of-00101.safetensors", + "model.layers.77.mlp.experts.72.gate_proj.weight": "model-00085-of-00101.safetensors", + "model.layers.77.mlp.experts.72.up_proj.weight": "model-00085-of-00101.safetensors", + "model.layers.77.mlp.experts.72.down_proj.weight": "model-00085-of-00101.safetensors", + "model.layers.77.mlp.experts.73.gate_proj.weight": "model-00085-of-00101.safetensors", + "model.layers.77.mlp.experts.73.up_proj.weight": "model-00085-of-00101.safetensors", + "model.layers.77.mlp.experts.73.down_proj.weight": "model-00085-of-00101.safetensors", + "model.layers.77.mlp.experts.74.gate_proj.weight": "model-00085-of-00101.safetensors", + "model.layers.77.mlp.experts.74.up_proj.weight": "model-00085-of-00101.safetensors", + "model.layers.77.mlp.experts.74.down_proj.weight": "model-00085-of-00101.safetensors", + "model.layers.77.mlp.experts.75.gate_proj.weight": "model-00085-of-00101.safetensors", + "model.layers.77.mlp.experts.75.up_proj.weight": "model-00085-of-00101.safetensors", + "model.layers.77.mlp.experts.75.down_proj.weight": "model-00085-of-00101.safetensors", + "model.layers.77.mlp.experts.76.gate_proj.weight": "model-00085-of-00101.safetensors", + "model.layers.77.mlp.experts.76.up_proj.weight": "model-00085-of-00101.safetensors", + "model.layers.77.mlp.experts.76.down_proj.weight": "model-00085-of-00101.safetensors", + "model.layers.77.mlp.experts.77.gate_proj.weight": "model-00085-of-00101.safetensors", + "model.layers.77.mlp.experts.77.up_proj.weight": "model-00085-of-00101.safetensors", + "model.layers.77.mlp.experts.77.down_proj.weight": "model-00085-of-00101.safetensors", + "model.layers.77.mlp.experts.78.gate_proj.weight": "model-00085-of-00101.safetensors", + "model.layers.77.mlp.experts.78.up_proj.weight": "model-00085-of-00101.safetensors", + "model.layers.77.mlp.experts.78.down_proj.weight": "model-00085-of-00101.safetensors", + "model.layers.77.mlp.experts.79.gate_proj.weight": "model-00085-of-00101.safetensors", + "model.layers.77.mlp.experts.79.up_proj.weight": "model-00085-of-00101.safetensors", + "model.layers.77.mlp.experts.79.down_proj.weight": "model-00085-of-00101.safetensors", + "model.layers.77.mlp.experts.80.gate_proj.weight": "model-00085-of-00101.safetensors", + "model.layers.77.mlp.experts.80.up_proj.weight": "model-00085-of-00101.safetensors", + "model.layers.77.mlp.experts.80.down_proj.weight": "model-00085-of-00101.safetensors", + "model.layers.77.mlp.experts.81.gate_proj.weight": "model-00085-of-00101.safetensors", + "model.layers.77.mlp.experts.81.up_proj.weight": "model-00085-of-00101.safetensors", + "model.layers.77.mlp.experts.81.down_proj.weight": "model-00085-of-00101.safetensors", + "model.layers.77.mlp.experts.82.gate_proj.weight": "model-00085-of-00101.safetensors", + "model.layers.77.mlp.experts.82.up_proj.weight": "model-00085-of-00101.safetensors", + "model.layers.77.mlp.experts.82.down_proj.weight": "model-00085-of-00101.safetensors", + "model.layers.77.mlp.experts.83.gate_proj.weight": "model-00085-of-00101.safetensors", + "model.layers.77.mlp.experts.83.up_proj.weight": "model-00085-of-00101.safetensors", + "model.layers.77.mlp.experts.83.down_proj.weight": "model-00085-of-00101.safetensors", + "model.layers.77.mlp.experts.84.gate_proj.weight": "model-00085-of-00101.safetensors", + "model.layers.77.mlp.experts.84.up_proj.weight": "model-00085-of-00101.safetensors", + "model.layers.77.mlp.experts.84.down_proj.weight": "model-00085-of-00101.safetensors", + "model.layers.77.mlp.experts.85.gate_proj.weight": "model-00085-of-00101.safetensors", + "model.layers.77.mlp.experts.85.up_proj.weight": "model-00085-of-00101.safetensors", + "model.layers.77.mlp.experts.85.down_proj.weight": "model-00085-of-00101.safetensors", + "model.layers.77.mlp.experts.86.gate_proj.weight": "model-00085-of-00101.safetensors", + "model.layers.77.mlp.experts.86.up_proj.weight": "model-00085-of-00101.safetensors", + "model.layers.77.mlp.experts.86.down_proj.weight": "model-00085-of-00101.safetensors", + "model.layers.77.mlp.experts.87.gate_proj.weight": "model-00085-of-00101.safetensors", + "model.layers.77.mlp.experts.87.up_proj.weight": "model-00085-of-00101.safetensors", + "model.layers.77.mlp.experts.87.down_proj.weight": "model-00085-of-00101.safetensors", + "model.layers.77.mlp.experts.88.gate_proj.weight": "model-00085-of-00101.safetensors", + "model.layers.77.mlp.experts.88.up_proj.weight": "model-00085-of-00101.safetensors", + "model.layers.77.mlp.experts.88.down_proj.weight": "model-00085-of-00101.safetensors", + "model.layers.77.mlp.experts.89.gate_proj.weight": "model-00085-of-00101.safetensors", + "model.layers.77.mlp.experts.89.up_proj.weight": "model-00085-of-00101.safetensors", + "model.layers.77.mlp.experts.89.down_proj.weight": "model-00085-of-00101.safetensors", + "model.layers.77.mlp.experts.90.gate_proj.weight": "model-00085-of-00101.safetensors", + "model.layers.77.mlp.experts.90.up_proj.weight": "model-00085-of-00101.safetensors", + "model.layers.77.mlp.experts.90.down_proj.weight": "model-00085-of-00101.safetensors", + "model.layers.77.mlp.experts.91.gate_proj.weight": "model-00085-of-00101.safetensors", + "model.layers.77.mlp.experts.91.up_proj.weight": "model-00085-of-00101.safetensors", + "model.layers.77.mlp.experts.91.down_proj.weight": "model-00085-of-00101.safetensors", + "model.layers.77.mlp.experts.92.gate_proj.weight": "model-00085-of-00101.safetensors", + "model.layers.77.mlp.experts.92.up_proj.weight": "model-00085-of-00101.safetensors", + "model.layers.77.mlp.experts.92.down_proj.weight": "model-00085-of-00101.safetensors", + "model.layers.77.mlp.experts.93.gate_proj.weight": "model-00085-of-00101.safetensors", + "model.layers.77.mlp.experts.93.up_proj.weight": "model-00085-of-00101.safetensors", + "model.layers.77.mlp.experts.93.down_proj.weight": "model-00085-of-00101.safetensors", + "model.layers.77.mlp.experts.94.gate_proj.weight": "model-00085-of-00101.safetensors", + "model.layers.77.mlp.experts.94.up_proj.weight": "model-00085-of-00101.safetensors", + "model.layers.77.mlp.experts.94.down_proj.weight": "model-00085-of-00101.safetensors", + "model.layers.77.mlp.experts.95.gate_proj.weight": "model-00085-of-00101.safetensors", + "model.layers.77.mlp.experts.95.up_proj.weight": "model-00085-of-00101.safetensors", + "model.layers.77.mlp.experts.95.down_proj.weight": "model-00085-of-00101.safetensors", + "model.layers.77.mlp.experts.96.gate_proj.weight": "model-00085-of-00101.safetensors", + "model.layers.77.mlp.experts.96.up_proj.weight": "model-00085-of-00101.safetensors", + "model.layers.77.mlp.experts.96.down_proj.weight": "model-00085-of-00101.safetensors", + "model.layers.77.mlp.experts.97.gate_proj.weight": "model-00085-of-00101.safetensors", + "model.layers.77.mlp.experts.97.up_proj.weight": "model-00085-of-00101.safetensors", + "model.layers.77.mlp.experts.97.down_proj.weight": "model-00085-of-00101.safetensors", + "model.layers.77.mlp.experts.98.gate_proj.weight": "model-00085-of-00101.safetensors", + "model.layers.77.mlp.experts.98.up_proj.weight": "model-00085-of-00101.safetensors", + "model.layers.77.mlp.experts.98.down_proj.weight": "model-00085-of-00101.safetensors", + "model.layers.77.mlp.experts.99.gate_proj.weight": "model-00085-of-00101.safetensors", + "model.layers.77.mlp.experts.99.up_proj.weight": "model-00085-of-00101.safetensors", + "model.layers.77.mlp.experts.99.down_proj.weight": "model-00085-of-00101.safetensors", + "model.layers.77.mlp.experts.100.gate_proj.weight": "model-00085-of-00101.safetensors", + "model.layers.77.mlp.experts.100.up_proj.weight": "model-00085-of-00101.safetensors", + "model.layers.77.mlp.experts.100.down_proj.weight": "model-00085-of-00101.safetensors", + "model.layers.77.mlp.experts.101.gate_proj.weight": "model-00085-of-00101.safetensors", + "model.layers.77.mlp.experts.101.up_proj.weight": "model-00085-of-00101.safetensors", + "model.layers.77.mlp.experts.101.down_proj.weight": "model-00085-of-00101.safetensors", + "model.layers.77.mlp.experts.102.gate_proj.weight": "model-00085-of-00101.safetensors", + "model.layers.77.mlp.experts.102.up_proj.weight": "model-00085-of-00101.safetensors", + "model.layers.77.mlp.experts.102.down_proj.weight": "model-00085-of-00101.safetensors", + "model.layers.77.mlp.experts.103.gate_proj.weight": "model-00085-of-00101.safetensors", + "model.layers.77.mlp.experts.103.up_proj.weight": "model-00085-of-00101.safetensors", + "model.layers.77.mlp.experts.103.down_proj.weight": "model-00085-of-00101.safetensors", + "model.layers.77.mlp.experts.104.gate_proj.weight": "model-00085-of-00101.safetensors", + "model.layers.77.mlp.experts.104.up_proj.weight": "model-00085-of-00101.safetensors", + "model.layers.77.mlp.experts.104.down_proj.weight": "model-00085-of-00101.safetensors", + "model.layers.77.mlp.experts.105.gate_proj.weight": "model-00085-of-00101.safetensors", + "model.layers.77.mlp.experts.105.up_proj.weight": "model-00085-of-00101.safetensors", + "model.layers.77.mlp.experts.105.down_proj.weight": "model-00085-of-00101.safetensors", + "model.layers.77.mlp.experts.106.gate_proj.weight": "model-00085-of-00101.safetensors", + "model.layers.77.mlp.experts.106.up_proj.weight": "model-00085-of-00101.safetensors", + "model.layers.77.mlp.experts.106.down_proj.weight": "model-00085-of-00101.safetensors", + "model.layers.77.mlp.experts.107.gate_proj.weight": "model-00085-of-00101.safetensors", + "model.layers.77.mlp.experts.107.up_proj.weight": "model-00085-of-00101.safetensors", + "model.layers.77.mlp.experts.107.down_proj.weight": "model-00085-of-00101.safetensors", + "model.layers.77.mlp.experts.108.gate_proj.weight": "model-00085-of-00101.safetensors", + "model.layers.77.mlp.experts.108.up_proj.weight": "model-00085-of-00101.safetensors", + "model.layers.77.mlp.experts.108.down_proj.weight": "model-00085-of-00101.safetensors", + "model.layers.77.mlp.experts.109.gate_proj.weight": "model-00085-of-00101.safetensors", + "model.layers.77.mlp.experts.109.up_proj.weight": "model-00085-of-00101.safetensors", + "model.layers.77.mlp.experts.109.down_proj.weight": "model-00085-of-00101.safetensors", + "model.layers.77.mlp.experts.110.gate_proj.weight": "model-00085-of-00101.safetensors", + "model.layers.77.mlp.experts.110.up_proj.weight": "model-00085-of-00101.safetensors", + "model.layers.77.mlp.experts.110.down_proj.weight": "model-00085-of-00101.safetensors", + "model.layers.77.mlp.experts.111.gate_proj.weight": "model-00085-of-00101.safetensors", + "model.layers.77.mlp.experts.111.up_proj.weight": "model-00085-of-00101.safetensors", + "model.layers.77.mlp.experts.111.down_proj.weight": "model-00085-of-00101.safetensors", + "model.layers.77.mlp.experts.112.gate_proj.weight": "model-00085-of-00101.safetensors", + "model.layers.77.mlp.experts.112.up_proj.weight": "model-00085-of-00101.safetensors", + "model.layers.77.mlp.experts.112.down_proj.weight": "model-00085-of-00101.safetensors", + "model.layers.77.mlp.experts.113.gate_proj.weight": "model-00085-of-00101.safetensors", + "model.layers.77.mlp.experts.113.up_proj.weight": "model-00085-of-00101.safetensors", + "model.layers.77.mlp.experts.113.down_proj.weight": "model-00085-of-00101.safetensors", + "model.layers.77.mlp.experts.114.gate_proj.weight": "model-00085-of-00101.safetensors", + "model.layers.77.mlp.experts.114.up_proj.weight": "model-00085-of-00101.safetensors", + "model.layers.77.mlp.experts.114.down_proj.weight": "model-00085-of-00101.safetensors", + "model.layers.77.mlp.experts.115.gate_proj.weight": "model-00085-of-00101.safetensors", + "model.layers.77.mlp.experts.115.up_proj.weight": "model-00085-of-00101.safetensors", + "model.layers.77.mlp.experts.115.down_proj.weight": "model-00085-of-00101.safetensors", + "model.layers.77.mlp.experts.116.gate_proj.weight": "model-00085-of-00101.safetensors", + "model.layers.77.mlp.experts.116.up_proj.weight": "model-00085-of-00101.safetensors", + "model.layers.77.mlp.experts.116.down_proj.weight": "model-00085-of-00101.safetensors", + "model.layers.77.mlp.experts.117.gate_proj.weight": "model-00085-of-00101.safetensors", + "model.layers.77.mlp.experts.117.up_proj.weight": "model-00085-of-00101.safetensors", + "model.layers.77.mlp.experts.117.down_proj.weight": "model-00085-of-00101.safetensors", + "model.layers.77.mlp.experts.118.gate_proj.weight": "model-00085-of-00101.safetensors", + "model.layers.77.mlp.experts.118.up_proj.weight": "model-00085-of-00101.safetensors", + "model.layers.77.mlp.experts.118.down_proj.weight": "model-00085-of-00101.safetensors", + "model.layers.77.mlp.experts.119.gate_proj.weight": "model-00085-of-00101.safetensors", + "model.layers.77.mlp.experts.119.up_proj.weight": "model-00085-of-00101.safetensors", + "model.layers.77.mlp.experts.119.down_proj.weight": "model-00085-of-00101.safetensors", + "model.layers.77.mlp.gate.weight": "model-00085-of-00101.safetensors", + "model.layers.77.mlp.gate.e_score_correction_bias": "model-00085-of-00101.safetensors", + "model.layers.77.mlp.shared_experts.gate_proj.weight": "model-00085-of-00101.safetensors", + "model.layers.77.mlp.shared_experts.up_proj.weight": "model-00085-of-00101.safetensors", + "model.layers.77.mlp.shared_experts.down_proj.weight": "model-00085-of-00101.safetensors", + "model.layers.77.input_layernorm.weight": "model-00085-of-00101.safetensors", + "model.layers.77.post_attention_layernorm.weight": "model-00085-of-00101.safetensors", + "model.layers.78.self_attn.q_proj.weight": "model-00085-of-00101.safetensors", + "model.layers.78.self_attn.q_proj.bias": "model-00085-of-00101.safetensors", + "model.layers.78.self_attn.k_proj.weight": "model-00085-of-00101.safetensors", + "model.layers.78.self_attn.k_proj.bias": "model-00085-of-00101.safetensors", + "model.layers.78.self_attn.v_proj.weight": "model-00085-of-00101.safetensors", + "model.layers.78.self_attn.v_proj.bias": "model-00085-of-00101.safetensors", + "model.layers.78.self_attn.o_proj.weight": "model-00085-of-00101.safetensors", + "model.layers.78.self_attn.q_norm.weight": "model-00085-of-00101.safetensors", + "model.layers.78.self_attn.k_norm.weight": "model-00085-of-00101.safetensors", + "model.layers.78.mlp.experts.0.gate_proj.weight": "model-00085-of-00101.safetensors", + "model.layers.78.mlp.experts.0.up_proj.weight": "model-00085-of-00101.safetensors", + "model.layers.78.mlp.experts.0.down_proj.weight": "model-00085-of-00101.safetensors", + "model.layers.78.mlp.experts.1.gate_proj.weight": "model-00085-of-00101.safetensors", + "model.layers.78.mlp.experts.1.up_proj.weight": "model-00085-of-00101.safetensors", + "model.layers.78.mlp.experts.1.down_proj.weight": "model-00085-of-00101.safetensors", + "model.layers.78.mlp.experts.2.gate_proj.weight": "model-00085-of-00101.safetensors", + "model.layers.78.mlp.experts.2.up_proj.weight": "model-00085-of-00101.safetensors", + "model.layers.78.mlp.experts.2.down_proj.weight": "model-00085-of-00101.safetensors", + "model.layers.78.mlp.experts.3.gate_proj.weight": "model-00085-of-00101.safetensors", + "model.layers.78.mlp.experts.3.up_proj.weight": "model-00085-of-00101.safetensors", + "model.layers.78.mlp.experts.3.down_proj.weight": "model-00085-of-00101.safetensors", + "model.layers.78.mlp.experts.4.gate_proj.weight": "model-00085-of-00101.safetensors", + "model.layers.78.mlp.experts.4.up_proj.weight": "model-00085-of-00101.safetensors", + "model.layers.78.mlp.experts.4.down_proj.weight": "model-00085-of-00101.safetensors", + "model.layers.78.mlp.experts.5.gate_proj.weight": "model-00085-of-00101.safetensors", + "model.layers.78.mlp.experts.5.up_proj.weight": "model-00085-of-00101.safetensors", + "model.layers.78.mlp.experts.5.down_proj.weight": "model-00085-of-00101.safetensors", + "model.layers.78.mlp.experts.6.gate_proj.weight": "model-00085-of-00101.safetensors", + "model.layers.78.mlp.experts.6.up_proj.weight": "model-00085-of-00101.safetensors", + "model.layers.78.mlp.experts.6.down_proj.weight": "model-00085-of-00101.safetensors", + "model.layers.78.mlp.experts.7.gate_proj.weight": "model-00085-of-00101.safetensors", + "model.layers.78.mlp.experts.7.up_proj.weight": "model-00085-of-00101.safetensors", + "model.layers.78.mlp.experts.7.down_proj.weight": "model-00085-of-00101.safetensors", + "model.layers.78.mlp.experts.8.gate_proj.weight": "model-00085-of-00101.safetensors", + "model.layers.78.mlp.experts.8.up_proj.weight": "model-00085-of-00101.safetensors", + "model.layers.78.mlp.experts.8.down_proj.weight": "model-00085-of-00101.safetensors", + "model.layers.78.mlp.experts.9.gate_proj.weight": "model-00085-of-00101.safetensors", + "model.layers.78.mlp.experts.9.up_proj.weight": "model-00085-of-00101.safetensors", + "model.layers.78.mlp.experts.9.down_proj.weight": "model-00085-of-00101.safetensors", + "model.layers.78.mlp.experts.10.gate_proj.weight": "model-00085-of-00101.safetensors", + "model.layers.78.mlp.experts.10.up_proj.weight": "model-00085-of-00101.safetensors", + "model.layers.78.mlp.experts.10.down_proj.weight": "model-00085-of-00101.safetensors", + "model.layers.78.mlp.experts.11.gate_proj.weight": "model-00085-of-00101.safetensors", + "model.layers.78.mlp.experts.11.up_proj.weight": "model-00085-of-00101.safetensors", + "model.layers.78.mlp.experts.11.down_proj.weight": "model-00085-of-00101.safetensors", + "model.layers.78.mlp.experts.12.gate_proj.weight": "model-00085-of-00101.safetensors", + "model.layers.78.mlp.experts.12.up_proj.weight": "model-00085-of-00101.safetensors", + "model.layers.78.mlp.experts.12.down_proj.weight": "model-00085-of-00101.safetensors", + "model.layers.78.mlp.experts.13.gate_proj.weight": "model-00085-of-00101.safetensors", + "model.layers.78.mlp.experts.13.up_proj.weight": "model-00085-of-00101.safetensors", + "model.layers.78.mlp.experts.13.down_proj.weight": "model-00085-of-00101.safetensors", + "model.layers.78.mlp.experts.14.gate_proj.weight": "model-00085-of-00101.safetensors", + "model.layers.78.mlp.experts.14.up_proj.weight": "model-00085-of-00101.safetensors", + "model.layers.78.mlp.experts.14.down_proj.weight": "model-00085-of-00101.safetensors", + "model.layers.78.mlp.experts.15.gate_proj.weight": "model-00085-of-00101.safetensors", + "model.layers.78.mlp.experts.15.up_proj.weight": "model-00085-of-00101.safetensors", + "model.layers.78.mlp.experts.15.down_proj.weight": "model-00085-of-00101.safetensors", + "model.layers.78.mlp.experts.16.gate_proj.weight": "model-00085-of-00101.safetensors", + "model.layers.78.mlp.experts.16.up_proj.weight": "model-00085-of-00101.safetensors", + "model.layers.78.mlp.experts.16.down_proj.weight": "model-00085-of-00101.safetensors", + "model.layers.78.mlp.experts.17.gate_proj.weight": "model-00085-of-00101.safetensors", + "model.layers.78.mlp.experts.17.up_proj.weight": "model-00085-of-00101.safetensors", + "model.layers.78.mlp.experts.17.down_proj.weight": "model-00085-of-00101.safetensors", + "model.layers.78.mlp.experts.18.gate_proj.weight": "model-00085-of-00101.safetensors", + "model.layers.78.mlp.experts.18.up_proj.weight": "model-00085-of-00101.safetensors", + "model.layers.78.mlp.experts.18.down_proj.weight": "model-00085-of-00101.safetensors", + "model.layers.78.mlp.experts.19.gate_proj.weight": "model-00085-of-00101.safetensors", + "model.layers.78.mlp.experts.19.up_proj.weight": "model-00085-of-00101.safetensors", + "model.layers.78.mlp.experts.19.down_proj.weight": "model-00085-of-00101.safetensors", + "model.layers.78.mlp.experts.20.gate_proj.weight": "model-00085-of-00101.safetensors", + "model.layers.78.mlp.experts.20.up_proj.weight": "model-00085-of-00101.safetensors", + "model.layers.78.mlp.experts.20.down_proj.weight": "model-00085-of-00101.safetensors", + "model.layers.78.mlp.experts.21.gate_proj.weight": "model-00085-of-00101.safetensors", + "model.layers.78.mlp.experts.21.up_proj.weight": "model-00085-of-00101.safetensors", + "model.layers.78.mlp.experts.21.down_proj.weight": "model-00085-of-00101.safetensors", + "model.layers.78.mlp.experts.22.gate_proj.weight": "model-00085-of-00101.safetensors", + "model.layers.78.mlp.experts.22.up_proj.weight": "model-00085-of-00101.safetensors", + "model.layers.78.mlp.experts.22.down_proj.weight": "model-00085-of-00101.safetensors", + "model.layers.78.mlp.experts.23.gate_proj.weight": "model-00085-of-00101.safetensors", + "model.layers.78.mlp.experts.23.up_proj.weight": "model-00085-of-00101.safetensors", + "model.layers.78.mlp.experts.23.down_proj.weight": "model-00085-of-00101.safetensors", + "model.layers.78.mlp.experts.24.gate_proj.weight": "model-00085-of-00101.safetensors", + "model.layers.78.mlp.experts.24.up_proj.weight": "model-00085-of-00101.safetensors", + "model.layers.78.mlp.experts.24.down_proj.weight": "model-00085-of-00101.safetensors", + "model.layers.78.mlp.experts.25.gate_proj.weight": "model-00085-of-00101.safetensors", + "model.layers.78.mlp.experts.25.up_proj.weight": "model-00085-of-00101.safetensors", + "model.layers.78.mlp.experts.25.down_proj.weight": "model-00085-of-00101.safetensors", + "model.layers.78.mlp.experts.26.gate_proj.weight": "model-00085-of-00101.safetensors", + "model.layers.78.mlp.experts.26.up_proj.weight": "model-00085-of-00101.safetensors", + "model.layers.78.mlp.experts.26.down_proj.weight": "model-00085-of-00101.safetensors", + "model.layers.78.mlp.experts.27.gate_proj.weight": "model-00085-of-00101.safetensors", + "model.layers.78.mlp.experts.27.up_proj.weight": "model-00085-of-00101.safetensors", + "model.layers.78.mlp.experts.27.down_proj.weight": "model-00085-of-00101.safetensors", + "model.layers.78.mlp.experts.28.gate_proj.weight": "model-00085-of-00101.safetensors", + "model.layers.78.mlp.experts.28.up_proj.weight": "model-00085-of-00101.safetensors", + "model.layers.78.mlp.experts.28.down_proj.weight": "model-00085-of-00101.safetensors", + "model.layers.78.mlp.experts.29.gate_proj.weight": "model-00085-of-00101.safetensors", + "model.layers.78.mlp.experts.29.up_proj.weight": "model-00085-of-00101.safetensors", + "model.layers.78.mlp.experts.29.down_proj.weight": "model-00085-of-00101.safetensors", + "model.layers.78.mlp.experts.30.gate_proj.weight": "model-00085-of-00101.safetensors", + "model.layers.78.mlp.experts.30.up_proj.weight": "model-00085-of-00101.safetensors", + "model.layers.78.mlp.experts.30.down_proj.weight": "model-00085-of-00101.safetensors", + "model.layers.78.mlp.experts.31.gate_proj.weight": "model-00085-of-00101.safetensors", + "model.layers.78.mlp.experts.31.up_proj.weight": "model-00085-of-00101.safetensors", + "model.layers.78.mlp.experts.31.down_proj.weight": "model-00085-of-00101.safetensors", + "model.layers.78.mlp.experts.32.gate_proj.weight": "model-00085-of-00101.safetensors", + "model.layers.78.mlp.experts.32.up_proj.weight": "model-00085-of-00101.safetensors", + "model.layers.78.mlp.experts.32.down_proj.weight": "model-00085-of-00101.safetensors", + "model.layers.78.mlp.experts.33.gate_proj.weight": "model-00085-of-00101.safetensors", + "model.layers.78.mlp.experts.33.up_proj.weight": "model-00085-of-00101.safetensors", + "model.layers.78.mlp.experts.33.down_proj.weight": "model-00085-of-00101.safetensors", + "model.layers.78.mlp.experts.34.gate_proj.weight": "model-00085-of-00101.safetensors", + "model.layers.78.mlp.experts.34.up_proj.weight": "model-00085-of-00101.safetensors", + "model.layers.78.mlp.experts.34.down_proj.weight": "model-00085-of-00101.safetensors", + "model.layers.78.mlp.experts.35.gate_proj.weight": "model-00085-of-00101.safetensors", + "model.layers.78.mlp.experts.35.up_proj.weight": "model-00085-of-00101.safetensors", + "model.layers.78.mlp.experts.35.down_proj.weight": "model-00085-of-00101.safetensors", + "model.layers.78.mlp.experts.36.gate_proj.weight": "model-00085-of-00101.safetensors", + "model.layers.78.mlp.experts.36.up_proj.weight": "model-00085-of-00101.safetensors", + "model.layers.78.mlp.experts.36.down_proj.weight": "model-00085-of-00101.safetensors", + "model.layers.78.mlp.experts.37.gate_proj.weight": "model-00085-of-00101.safetensors", + "model.layers.78.mlp.experts.37.up_proj.weight": "model-00085-of-00101.safetensors", + "model.layers.78.mlp.experts.37.down_proj.weight": "model-00085-of-00101.safetensors", + "model.layers.78.mlp.experts.38.gate_proj.weight": "model-00085-of-00101.safetensors", + "model.layers.78.mlp.experts.38.up_proj.weight": "model-00085-of-00101.safetensors", + "model.layers.78.mlp.experts.38.down_proj.weight": "model-00085-of-00101.safetensors", + "model.layers.78.mlp.experts.39.gate_proj.weight": "model-00085-of-00101.safetensors", + "model.layers.78.mlp.experts.39.up_proj.weight": "model-00085-of-00101.safetensors", + "model.layers.78.mlp.experts.39.down_proj.weight": "model-00085-of-00101.safetensors", + "model.layers.78.mlp.experts.40.gate_proj.weight": "model-00085-of-00101.safetensors", + "model.layers.78.mlp.experts.40.up_proj.weight": "model-00085-of-00101.safetensors", + "model.layers.78.mlp.experts.40.down_proj.weight": "model-00085-of-00101.safetensors", + "model.layers.78.mlp.experts.41.gate_proj.weight": "model-00085-of-00101.safetensors", + "model.layers.78.mlp.experts.41.up_proj.weight": "model-00085-of-00101.safetensors", + "model.layers.78.mlp.experts.41.down_proj.weight": "model-00085-of-00101.safetensors", + "model.layers.78.mlp.experts.42.gate_proj.weight": "model-00085-of-00101.safetensors", + "model.layers.78.mlp.experts.42.up_proj.weight": "model-00085-of-00101.safetensors", + "model.layers.78.mlp.experts.42.down_proj.weight": "model-00085-of-00101.safetensors", + "model.layers.78.mlp.experts.43.gate_proj.weight": "model-00085-of-00101.safetensors", + "model.layers.78.mlp.experts.43.up_proj.weight": "model-00085-of-00101.safetensors", + "model.layers.78.mlp.experts.43.down_proj.weight": "model-00085-of-00101.safetensors", + "model.layers.78.mlp.experts.44.gate_proj.weight": "model-00085-of-00101.safetensors", + "model.layers.78.mlp.experts.44.up_proj.weight": "model-00085-of-00101.safetensors", + "model.layers.78.mlp.experts.44.down_proj.weight": "model-00085-of-00101.safetensors", + "model.layers.78.mlp.experts.45.gate_proj.weight": "model-00085-of-00101.safetensors", + "model.layers.78.mlp.experts.45.up_proj.weight": "model-00085-of-00101.safetensors", + "model.layers.78.mlp.experts.45.down_proj.weight": "model-00085-of-00101.safetensors", + "model.layers.78.mlp.experts.46.gate_proj.weight": "model-00085-of-00101.safetensors", + "model.layers.78.mlp.experts.46.up_proj.weight": "model-00085-of-00101.safetensors", + "model.layers.78.mlp.experts.46.down_proj.weight": "model-00085-of-00101.safetensors", + "model.layers.78.mlp.experts.47.gate_proj.weight": "model-00085-of-00101.safetensors", + "model.layers.78.mlp.experts.47.up_proj.weight": "model-00085-of-00101.safetensors", + "model.layers.78.mlp.experts.47.down_proj.weight": "model-00085-of-00101.safetensors", + "model.layers.78.mlp.experts.48.gate_proj.weight": "model-00085-of-00101.safetensors", + "model.layers.78.mlp.experts.48.up_proj.weight": "model-00085-of-00101.safetensors", + "model.layers.78.mlp.experts.48.down_proj.weight": "model-00085-of-00101.safetensors", + "model.layers.78.mlp.experts.49.gate_proj.weight": "model-00085-of-00101.safetensors", + "model.layers.78.mlp.experts.49.up_proj.weight": "model-00085-of-00101.safetensors", + "model.layers.78.mlp.experts.49.down_proj.weight": "model-00085-of-00101.safetensors", + "model.layers.78.mlp.experts.50.gate_proj.weight": "model-00085-of-00101.safetensors", + "model.layers.78.mlp.experts.50.up_proj.weight": "model-00085-of-00101.safetensors", + "model.layers.78.mlp.experts.50.down_proj.weight": "model-00085-of-00101.safetensors", + "model.layers.78.mlp.experts.51.gate_proj.weight": "model-00085-of-00101.safetensors", + "model.layers.78.mlp.experts.51.up_proj.weight": "model-00085-of-00101.safetensors", + "model.layers.78.mlp.experts.51.down_proj.weight": "model-00085-of-00101.safetensors", + "model.layers.78.mlp.experts.52.gate_proj.weight": "model-00085-of-00101.safetensors", + "model.layers.78.mlp.experts.52.up_proj.weight": "model-00085-of-00101.safetensors", + "model.layers.78.mlp.experts.52.down_proj.weight": "model-00085-of-00101.safetensors", + "model.layers.78.mlp.experts.53.gate_proj.weight": "model-00085-of-00101.safetensors", + "model.layers.78.mlp.experts.53.up_proj.weight": "model-00085-of-00101.safetensors", + "model.layers.78.mlp.experts.53.down_proj.weight": "model-00085-of-00101.safetensors", + "model.layers.78.mlp.experts.54.gate_proj.weight": "model-00085-of-00101.safetensors", + "model.layers.78.mlp.experts.54.up_proj.weight": "model-00086-of-00101.safetensors", + "model.layers.78.mlp.experts.54.down_proj.weight": "model-00086-of-00101.safetensors", + "model.layers.78.mlp.experts.55.gate_proj.weight": "model-00086-of-00101.safetensors", + "model.layers.78.mlp.experts.55.up_proj.weight": "model-00086-of-00101.safetensors", + "model.layers.78.mlp.experts.55.down_proj.weight": "model-00086-of-00101.safetensors", + "model.layers.78.mlp.experts.56.gate_proj.weight": "model-00086-of-00101.safetensors", + "model.layers.78.mlp.experts.56.up_proj.weight": "model-00086-of-00101.safetensors", + "model.layers.78.mlp.experts.56.down_proj.weight": "model-00086-of-00101.safetensors", + "model.layers.78.mlp.experts.57.gate_proj.weight": "model-00086-of-00101.safetensors", + "model.layers.78.mlp.experts.57.up_proj.weight": "model-00086-of-00101.safetensors", + "model.layers.78.mlp.experts.57.down_proj.weight": "model-00086-of-00101.safetensors", + "model.layers.78.mlp.experts.58.gate_proj.weight": "model-00086-of-00101.safetensors", + "model.layers.78.mlp.experts.58.up_proj.weight": "model-00086-of-00101.safetensors", + "model.layers.78.mlp.experts.58.down_proj.weight": "model-00086-of-00101.safetensors", + "model.layers.78.mlp.experts.59.gate_proj.weight": "model-00086-of-00101.safetensors", + "model.layers.78.mlp.experts.59.up_proj.weight": "model-00086-of-00101.safetensors", + "model.layers.78.mlp.experts.59.down_proj.weight": "model-00086-of-00101.safetensors", + "model.layers.78.mlp.experts.60.gate_proj.weight": "model-00086-of-00101.safetensors", + "model.layers.78.mlp.experts.60.up_proj.weight": "model-00086-of-00101.safetensors", + "model.layers.78.mlp.experts.60.down_proj.weight": "model-00086-of-00101.safetensors", + "model.layers.78.mlp.experts.61.gate_proj.weight": "model-00086-of-00101.safetensors", + "model.layers.78.mlp.experts.61.up_proj.weight": "model-00086-of-00101.safetensors", + "model.layers.78.mlp.experts.61.down_proj.weight": "model-00086-of-00101.safetensors", + "model.layers.78.mlp.experts.62.gate_proj.weight": "model-00086-of-00101.safetensors", + "model.layers.78.mlp.experts.62.up_proj.weight": "model-00086-of-00101.safetensors", + "model.layers.78.mlp.experts.62.down_proj.weight": "model-00086-of-00101.safetensors", + "model.layers.78.mlp.experts.63.gate_proj.weight": "model-00086-of-00101.safetensors", + "model.layers.78.mlp.experts.63.up_proj.weight": "model-00086-of-00101.safetensors", + "model.layers.78.mlp.experts.63.down_proj.weight": "model-00086-of-00101.safetensors", + "model.layers.78.mlp.experts.64.gate_proj.weight": "model-00086-of-00101.safetensors", + "model.layers.78.mlp.experts.64.up_proj.weight": "model-00086-of-00101.safetensors", + "model.layers.78.mlp.experts.64.down_proj.weight": "model-00086-of-00101.safetensors", + "model.layers.78.mlp.experts.65.gate_proj.weight": "model-00086-of-00101.safetensors", + "model.layers.78.mlp.experts.65.up_proj.weight": "model-00086-of-00101.safetensors", + "model.layers.78.mlp.experts.65.down_proj.weight": "model-00086-of-00101.safetensors", + "model.layers.78.mlp.experts.66.gate_proj.weight": "model-00086-of-00101.safetensors", + "model.layers.78.mlp.experts.66.up_proj.weight": "model-00086-of-00101.safetensors", + "model.layers.78.mlp.experts.66.down_proj.weight": "model-00086-of-00101.safetensors", + "model.layers.78.mlp.experts.67.gate_proj.weight": "model-00086-of-00101.safetensors", + "model.layers.78.mlp.experts.67.up_proj.weight": "model-00086-of-00101.safetensors", + "model.layers.78.mlp.experts.67.down_proj.weight": "model-00086-of-00101.safetensors", + "model.layers.78.mlp.experts.68.gate_proj.weight": "model-00086-of-00101.safetensors", + "model.layers.78.mlp.experts.68.up_proj.weight": "model-00086-of-00101.safetensors", + "model.layers.78.mlp.experts.68.down_proj.weight": "model-00086-of-00101.safetensors", + "model.layers.78.mlp.experts.69.gate_proj.weight": "model-00086-of-00101.safetensors", + "model.layers.78.mlp.experts.69.up_proj.weight": "model-00086-of-00101.safetensors", + "model.layers.78.mlp.experts.69.down_proj.weight": "model-00086-of-00101.safetensors", + "model.layers.78.mlp.experts.70.gate_proj.weight": "model-00086-of-00101.safetensors", + "model.layers.78.mlp.experts.70.up_proj.weight": "model-00086-of-00101.safetensors", + "model.layers.78.mlp.experts.70.down_proj.weight": "model-00086-of-00101.safetensors", + "model.layers.78.mlp.experts.71.gate_proj.weight": "model-00086-of-00101.safetensors", + "model.layers.78.mlp.experts.71.up_proj.weight": "model-00086-of-00101.safetensors", + "model.layers.78.mlp.experts.71.down_proj.weight": "model-00086-of-00101.safetensors", + "model.layers.78.mlp.experts.72.gate_proj.weight": "model-00086-of-00101.safetensors", + "model.layers.78.mlp.experts.72.up_proj.weight": "model-00086-of-00101.safetensors", + "model.layers.78.mlp.experts.72.down_proj.weight": "model-00086-of-00101.safetensors", + "model.layers.78.mlp.experts.73.gate_proj.weight": "model-00086-of-00101.safetensors", + "model.layers.78.mlp.experts.73.up_proj.weight": "model-00086-of-00101.safetensors", + "model.layers.78.mlp.experts.73.down_proj.weight": "model-00086-of-00101.safetensors", + "model.layers.78.mlp.experts.74.gate_proj.weight": "model-00086-of-00101.safetensors", + "model.layers.78.mlp.experts.74.up_proj.weight": "model-00086-of-00101.safetensors", + "model.layers.78.mlp.experts.74.down_proj.weight": "model-00086-of-00101.safetensors", + "model.layers.78.mlp.experts.75.gate_proj.weight": "model-00086-of-00101.safetensors", + "model.layers.78.mlp.experts.75.up_proj.weight": "model-00086-of-00101.safetensors", + "model.layers.78.mlp.experts.75.down_proj.weight": "model-00086-of-00101.safetensors", + "model.layers.78.mlp.experts.76.gate_proj.weight": "model-00086-of-00101.safetensors", + "model.layers.78.mlp.experts.76.up_proj.weight": "model-00086-of-00101.safetensors", + "model.layers.78.mlp.experts.76.down_proj.weight": "model-00086-of-00101.safetensors", + "model.layers.78.mlp.experts.77.gate_proj.weight": "model-00086-of-00101.safetensors", + "model.layers.78.mlp.experts.77.up_proj.weight": "model-00086-of-00101.safetensors", + "model.layers.78.mlp.experts.77.down_proj.weight": "model-00086-of-00101.safetensors", + "model.layers.78.mlp.experts.78.gate_proj.weight": "model-00086-of-00101.safetensors", + "model.layers.78.mlp.experts.78.up_proj.weight": "model-00086-of-00101.safetensors", + "model.layers.78.mlp.experts.78.down_proj.weight": "model-00086-of-00101.safetensors", + "model.layers.78.mlp.experts.79.gate_proj.weight": "model-00086-of-00101.safetensors", + "model.layers.78.mlp.experts.79.up_proj.weight": "model-00086-of-00101.safetensors", + "model.layers.78.mlp.experts.79.down_proj.weight": "model-00086-of-00101.safetensors", + "model.layers.78.mlp.experts.80.gate_proj.weight": "model-00086-of-00101.safetensors", + "model.layers.78.mlp.experts.80.up_proj.weight": "model-00086-of-00101.safetensors", + "model.layers.78.mlp.experts.80.down_proj.weight": "model-00086-of-00101.safetensors", + "model.layers.78.mlp.experts.81.gate_proj.weight": "model-00086-of-00101.safetensors", + "model.layers.78.mlp.experts.81.up_proj.weight": "model-00086-of-00101.safetensors", + "model.layers.78.mlp.experts.81.down_proj.weight": "model-00086-of-00101.safetensors", + "model.layers.78.mlp.experts.82.gate_proj.weight": "model-00086-of-00101.safetensors", + "model.layers.78.mlp.experts.82.up_proj.weight": "model-00086-of-00101.safetensors", + "model.layers.78.mlp.experts.82.down_proj.weight": "model-00086-of-00101.safetensors", + "model.layers.78.mlp.experts.83.gate_proj.weight": "model-00086-of-00101.safetensors", + "model.layers.78.mlp.experts.83.up_proj.weight": "model-00086-of-00101.safetensors", + "model.layers.78.mlp.experts.83.down_proj.weight": "model-00086-of-00101.safetensors", + "model.layers.78.mlp.experts.84.gate_proj.weight": "model-00086-of-00101.safetensors", + "model.layers.78.mlp.experts.84.up_proj.weight": "model-00086-of-00101.safetensors", + "model.layers.78.mlp.experts.84.down_proj.weight": "model-00086-of-00101.safetensors", + "model.layers.78.mlp.experts.85.gate_proj.weight": "model-00086-of-00101.safetensors", + "model.layers.78.mlp.experts.85.up_proj.weight": "model-00086-of-00101.safetensors", + "model.layers.78.mlp.experts.85.down_proj.weight": "model-00086-of-00101.safetensors", + "model.layers.78.mlp.experts.86.gate_proj.weight": "model-00086-of-00101.safetensors", + "model.layers.78.mlp.experts.86.up_proj.weight": "model-00086-of-00101.safetensors", + "model.layers.78.mlp.experts.86.down_proj.weight": "model-00086-of-00101.safetensors", + "model.layers.78.mlp.experts.87.gate_proj.weight": "model-00086-of-00101.safetensors", + "model.layers.78.mlp.experts.87.up_proj.weight": "model-00086-of-00101.safetensors", + "model.layers.78.mlp.experts.87.down_proj.weight": "model-00086-of-00101.safetensors", + "model.layers.78.mlp.experts.88.gate_proj.weight": "model-00086-of-00101.safetensors", + "model.layers.78.mlp.experts.88.up_proj.weight": "model-00086-of-00101.safetensors", + "model.layers.78.mlp.experts.88.down_proj.weight": "model-00086-of-00101.safetensors", + "model.layers.78.mlp.experts.89.gate_proj.weight": "model-00086-of-00101.safetensors", + "model.layers.78.mlp.experts.89.up_proj.weight": "model-00086-of-00101.safetensors", + "model.layers.78.mlp.experts.89.down_proj.weight": "model-00086-of-00101.safetensors", + "model.layers.78.mlp.experts.90.gate_proj.weight": "model-00086-of-00101.safetensors", + "model.layers.78.mlp.experts.90.up_proj.weight": "model-00086-of-00101.safetensors", + "model.layers.78.mlp.experts.90.down_proj.weight": "model-00086-of-00101.safetensors", + "model.layers.78.mlp.experts.91.gate_proj.weight": "model-00086-of-00101.safetensors", + "model.layers.78.mlp.experts.91.up_proj.weight": "model-00086-of-00101.safetensors", + "model.layers.78.mlp.experts.91.down_proj.weight": "model-00086-of-00101.safetensors", + "model.layers.78.mlp.experts.92.gate_proj.weight": "model-00086-of-00101.safetensors", + "model.layers.78.mlp.experts.92.up_proj.weight": "model-00086-of-00101.safetensors", + "model.layers.78.mlp.experts.92.down_proj.weight": "model-00086-of-00101.safetensors", + "model.layers.78.mlp.experts.93.gate_proj.weight": "model-00086-of-00101.safetensors", + "model.layers.78.mlp.experts.93.up_proj.weight": "model-00086-of-00101.safetensors", + "model.layers.78.mlp.experts.93.down_proj.weight": "model-00086-of-00101.safetensors", + "model.layers.78.mlp.experts.94.gate_proj.weight": "model-00086-of-00101.safetensors", + "model.layers.78.mlp.experts.94.up_proj.weight": "model-00086-of-00101.safetensors", + "model.layers.78.mlp.experts.94.down_proj.weight": "model-00086-of-00101.safetensors", + "model.layers.78.mlp.experts.95.gate_proj.weight": "model-00086-of-00101.safetensors", + "model.layers.78.mlp.experts.95.up_proj.weight": "model-00086-of-00101.safetensors", + "model.layers.78.mlp.experts.95.down_proj.weight": "model-00086-of-00101.safetensors", + "model.layers.78.mlp.experts.96.gate_proj.weight": "model-00086-of-00101.safetensors", + "model.layers.78.mlp.experts.96.up_proj.weight": "model-00086-of-00101.safetensors", + "model.layers.78.mlp.experts.96.down_proj.weight": "model-00086-of-00101.safetensors", + "model.layers.78.mlp.experts.97.gate_proj.weight": "model-00086-of-00101.safetensors", + "model.layers.78.mlp.experts.97.up_proj.weight": "model-00086-of-00101.safetensors", + "model.layers.78.mlp.experts.97.down_proj.weight": "model-00086-of-00101.safetensors", + "model.layers.78.mlp.experts.98.gate_proj.weight": "model-00086-of-00101.safetensors", + "model.layers.78.mlp.experts.98.up_proj.weight": "model-00086-of-00101.safetensors", + "model.layers.78.mlp.experts.98.down_proj.weight": "model-00086-of-00101.safetensors", + "model.layers.78.mlp.experts.99.gate_proj.weight": "model-00086-of-00101.safetensors", + "model.layers.78.mlp.experts.99.up_proj.weight": "model-00086-of-00101.safetensors", + "model.layers.78.mlp.experts.99.down_proj.weight": "model-00086-of-00101.safetensors", + "model.layers.78.mlp.experts.100.gate_proj.weight": "model-00086-of-00101.safetensors", + "model.layers.78.mlp.experts.100.up_proj.weight": "model-00086-of-00101.safetensors", + "model.layers.78.mlp.experts.100.down_proj.weight": "model-00086-of-00101.safetensors", + "model.layers.78.mlp.experts.101.gate_proj.weight": "model-00086-of-00101.safetensors", + "model.layers.78.mlp.experts.101.up_proj.weight": "model-00086-of-00101.safetensors", + "model.layers.78.mlp.experts.101.down_proj.weight": "model-00086-of-00101.safetensors", + "model.layers.78.mlp.experts.102.gate_proj.weight": "model-00086-of-00101.safetensors", + "model.layers.78.mlp.experts.102.up_proj.weight": "model-00086-of-00101.safetensors", + "model.layers.78.mlp.experts.102.down_proj.weight": "model-00086-of-00101.safetensors", + "model.layers.78.mlp.experts.103.gate_proj.weight": "model-00086-of-00101.safetensors", + "model.layers.78.mlp.experts.103.up_proj.weight": "model-00086-of-00101.safetensors", + "model.layers.78.mlp.experts.103.down_proj.weight": "model-00086-of-00101.safetensors", + "model.layers.78.mlp.experts.104.gate_proj.weight": "model-00086-of-00101.safetensors", + "model.layers.78.mlp.experts.104.up_proj.weight": "model-00086-of-00101.safetensors", + "model.layers.78.mlp.experts.104.down_proj.weight": "model-00086-of-00101.safetensors", + "model.layers.78.mlp.experts.105.gate_proj.weight": "model-00086-of-00101.safetensors", + "model.layers.78.mlp.experts.105.up_proj.weight": "model-00086-of-00101.safetensors", + "model.layers.78.mlp.experts.105.down_proj.weight": "model-00086-of-00101.safetensors", + "model.layers.78.mlp.experts.106.gate_proj.weight": "model-00086-of-00101.safetensors", + "model.layers.78.mlp.experts.106.up_proj.weight": "model-00086-of-00101.safetensors", + "model.layers.78.mlp.experts.106.down_proj.weight": "model-00086-of-00101.safetensors", + "model.layers.78.mlp.experts.107.gate_proj.weight": "model-00086-of-00101.safetensors", + "model.layers.78.mlp.experts.107.up_proj.weight": "model-00086-of-00101.safetensors", + "model.layers.78.mlp.experts.107.down_proj.weight": "model-00086-of-00101.safetensors", + "model.layers.78.mlp.experts.108.gate_proj.weight": "model-00086-of-00101.safetensors", + "model.layers.78.mlp.experts.108.up_proj.weight": "model-00086-of-00101.safetensors", + "model.layers.78.mlp.experts.108.down_proj.weight": "model-00086-of-00101.safetensors", + "model.layers.78.mlp.experts.109.gate_proj.weight": "model-00086-of-00101.safetensors", + "model.layers.78.mlp.experts.109.up_proj.weight": "model-00086-of-00101.safetensors", + "model.layers.78.mlp.experts.109.down_proj.weight": "model-00086-of-00101.safetensors", + "model.layers.78.mlp.experts.110.gate_proj.weight": "model-00086-of-00101.safetensors", + "model.layers.78.mlp.experts.110.up_proj.weight": "model-00086-of-00101.safetensors", + "model.layers.78.mlp.experts.110.down_proj.weight": "model-00086-of-00101.safetensors", + "model.layers.78.mlp.experts.111.gate_proj.weight": "model-00086-of-00101.safetensors", + "model.layers.78.mlp.experts.111.up_proj.weight": "model-00086-of-00101.safetensors", + "model.layers.78.mlp.experts.111.down_proj.weight": "model-00086-of-00101.safetensors", + "model.layers.78.mlp.experts.112.gate_proj.weight": "model-00086-of-00101.safetensors", + "model.layers.78.mlp.experts.112.up_proj.weight": "model-00086-of-00101.safetensors", + "model.layers.78.mlp.experts.112.down_proj.weight": "model-00086-of-00101.safetensors", + "model.layers.78.mlp.experts.113.gate_proj.weight": "model-00086-of-00101.safetensors", + "model.layers.78.mlp.experts.113.up_proj.weight": "model-00086-of-00101.safetensors", + "model.layers.78.mlp.experts.113.down_proj.weight": "model-00086-of-00101.safetensors", + "model.layers.78.mlp.experts.114.gate_proj.weight": "model-00086-of-00101.safetensors", + "model.layers.78.mlp.experts.114.up_proj.weight": "model-00086-of-00101.safetensors", + "model.layers.78.mlp.experts.114.down_proj.weight": "model-00086-of-00101.safetensors", + "model.layers.78.mlp.experts.115.gate_proj.weight": "model-00086-of-00101.safetensors", + "model.layers.78.mlp.experts.115.up_proj.weight": "model-00086-of-00101.safetensors", + "model.layers.78.mlp.experts.115.down_proj.weight": "model-00086-of-00101.safetensors", + "model.layers.78.mlp.experts.116.gate_proj.weight": "model-00086-of-00101.safetensors", + "model.layers.78.mlp.experts.116.up_proj.weight": "model-00086-of-00101.safetensors", + "model.layers.78.mlp.experts.116.down_proj.weight": "model-00086-of-00101.safetensors", + "model.layers.78.mlp.experts.117.gate_proj.weight": "model-00086-of-00101.safetensors", + "model.layers.78.mlp.experts.117.up_proj.weight": "model-00086-of-00101.safetensors", + "model.layers.78.mlp.experts.117.down_proj.weight": "model-00086-of-00101.safetensors", + "model.layers.78.mlp.experts.118.gate_proj.weight": "model-00086-of-00101.safetensors", + "model.layers.78.mlp.experts.118.up_proj.weight": "model-00086-of-00101.safetensors", + "model.layers.78.mlp.experts.118.down_proj.weight": "model-00086-of-00101.safetensors", + "model.layers.78.mlp.experts.119.gate_proj.weight": "model-00086-of-00101.safetensors", + "model.layers.78.mlp.experts.119.up_proj.weight": "model-00086-of-00101.safetensors", + "model.layers.78.mlp.experts.119.down_proj.weight": "model-00086-of-00101.safetensors", + "model.layers.78.mlp.gate.weight": "model-00086-of-00101.safetensors", + "model.layers.78.mlp.gate.e_score_correction_bias": "model-00086-of-00101.safetensors", + "model.layers.78.mlp.shared_experts.gate_proj.weight": "model-00086-of-00101.safetensors", + "model.layers.78.mlp.shared_experts.up_proj.weight": "model-00086-of-00101.safetensors", + "model.layers.78.mlp.shared_experts.down_proj.weight": "model-00086-of-00101.safetensors", + "model.layers.78.input_layernorm.weight": "model-00086-of-00101.safetensors", + "model.layers.78.post_attention_layernorm.weight": "model-00086-of-00101.safetensors", + "model.layers.79.self_attn.q_proj.weight": "model-00086-of-00101.safetensors", + "model.layers.79.self_attn.q_proj.bias": "model-00086-of-00101.safetensors", + "model.layers.79.self_attn.k_proj.weight": "model-00086-of-00101.safetensors", + "model.layers.79.self_attn.k_proj.bias": "model-00086-of-00101.safetensors", + "model.layers.79.self_attn.v_proj.weight": "model-00086-of-00101.safetensors", + "model.layers.79.self_attn.v_proj.bias": "model-00086-of-00101.safetensors", + "model.layers.79.self_attn.o_proj.weight": "model-00086-of-00101.safetensors", + "model.layers.79.self_attn.q_norm.weight": "model-00086-of-00101.safetensors", + "model.layers.79.self_attn.k_norm.weight": "model-00086-of-00101.safetensors", + "model.layers.79.mlp.experts.0.gate_proj.weight": "model-00086-of-00101.safetensors", + "model.layers.79.mlp.experts.0.up_proj.weight": "model-00086-of-00101.safetensors", + "model.layers.79.mlp.experts.0.down_proj.weight": "model-00086-of-00101.safetensors", + "model.layers.79.mlp.experts.1.gate_proj.weight": "model-00086-of-00101.safetensors", + "model.layers.79.mlp.experts.1.up_proj.weight": "model-00086-of-00101.safetensors", + "model.layers.79.mlp.experts.1.down_proj.weight": "model-00086-of-00101.safetensors", + "model.layers.79.mlp.experts.2.gate_proj.weight": "model-00086-of-00101.safetensors", + "model.layers.79.mlp.experts.2.up_proj.weight": "model-00086-of-00101.safetensors", + "model.layers.79.mlp.experts.2.down_proj.weight": "model-00086-of-00101.safetensors", + "model.layers.79.mlp.experts.3.gate_proj.weight": "model-00086-of-00101.safetensors", + "model.layers.79.mlp.experts.3.up_proj.weight": "model-00086-of-00101.safetensors", + "model.layers.79.mlp.experts.3.down_proj.weight": "model-00086-of-00101.safetensors", + "model.layers.79.mlp.experts.4.gate_proj.weight": "model-00086-of-00101.safetensors", + "model.layers.79.mlp.experts.4.up_proj.weight": "model-00086-of-00101.safetensors", + "model.layers.79.mlp.experts.4.down_proj.weight": "model-00086-of-00101.safetensors", + "model.layers.79.mlp.experts.5.gate_proj.weight": "model-00086-of-00101.safetensors", + "model.layers.79.mlp.experts.5.up_proj.weight": "model-00086-of-00101.safetensors", + "model.layers.79.mlp.experts.5.down_proj.weight": "model-00086-of-00101.safetensors", + "model.layers.79.mlp.experts.6.gate_proj.weight": "model-00086-of-00101.safetensors", + "model.layers.79.mlp.experts.6.up_proj.weight": "model-00086-of-00101.safetensors", + "model.layers.79.mlp.experts.6.down_proj.weight": "model-00086-of-00101.safetensors", + "model.layers.79.mlp.experts.7.gate_proj.weight": "model-00086-of-00101.safetensors", + "model.layers.79.mlp.experts.7.up_proj.weight": "model-00086-of-00101.safetensors", + "model.layers.79.mlp.experts.7.down_proj.weight": "model-00086-of-00101.safetensors", + "model.layers.79.mlp.experts.8.gate_proj.weight": "model-00086-of-00101.safetensors", + "model.layers.79.mlp.experts.8.up_proj.weight": "model-00086-of-00101.safetensors", + "model.layers.79.mlp.experts.8.down_proj.weight": "model-00086-of-00101.safetensors", + "model.layers.79.mlp.experts.9.gate_proj.weight": "model-00086-of-00101.safetensors", + "model.layers.79.mlp.experts.9.up_proj.weight": "model-00086-of-00101.safetensors", + "model.layers.79.mlp.experts.9.down_proj.weight": "model-00086-of-00101.safetensors", + "model.layers.79.mlp.experts.10.gate_proj.weight": "model-00086-of-00101.safetensors", + "model.layers.79.mlp.experts.10.up_proj.weight": "model-00086-of-00101.safetensors", + "model.layers.79.mlp.experts.10.down_proj.weight": "model-00086-of-00101.safetensors", + "model.layers.79.mlp.experts.11.gate_proj.weight": "model-00086-of-00101.safetensors", + "model.layers.79.mlp.experts.11.up_proj.weight": "model-00086-of-00101.safetensors", + "model.layers.79.mlp.experts.11.down_proj.weight": "model-00086-of-00101.safetensors", + "model.layers.79.mlp.experts.12.gate_proj.weight": "model-00086-of-00101.safetensors", + "model.layers.79.mlp.experts.12.up_proj.weight": "model-00086-of-00101.safetensors", + "model.layers.79.mlp.experts.12.down_proj.weight": "model-00086-of-00101.safetensors", + "model.layers.79.mlp.experts.13.gate_proj.weight": "model-00086-of-00101.safetensors", + "model.layers.79.mlp.experts.13.up_proj.weight": "model-00086-of-00101.safetensors", + "model.layers.79.mlp.experts.13.down_proj.weight": "model-00086-of-00101.safetensors", + "model.layers.79.mlp.experts.14.gate_proj.weight": "model-00086-of-00101.safetensors", + "model.layers.79.mlp.experts.14.up_proj.weight": "model-00086-of-00101.safetensors", + "model.layers.79.mlp.experts.14.down_proj.weight": "model-00086-of-00101.safetensors", + "model.layers.79.mlp.experts.15.gate_proj.weight": "model-00086-of-00101.safetensors", + "model.layers.79.mlp.experts.15.up_proj.weight": "model-00086-of-00101.safetensors", + "model.layers.79.mlp.experts.15.down_proj.weight": "model-00086-of-00101.safetensors", + "model.layers.79.mlp.experts.16.gate_proj.weight": "model-00086-of-00101.safetensors", + "model.layers.79.mlp.experts.16.up_proj.weight": "model-00086-of-00101.safetensors", + "model.layers.79.mlp.experts.16.down_proj.weight": "model-00086-of-00101.safetensors", + "model.layers.79.mlp.experts.17.gate_proj.weight": "model-00086-of-00101.safetensors", + "model.layers.79.mlp.experts.17.up_proj.weight": "model-00086-of-00101.safetensors", + "model.layers.79.mlp.experts.17.down_proj.weight": "model-00086-of-00101.safetensors", + "model.layers.79.mlp.experts.18.gate_proj.weight": "model-00086-of-00101.safetensors", + "model.layers.79.mlp.experts.18.up_proj.weight": "model-00086-of-00101.safetensors", + "model.layers.79.mlp.experts.18.down_proj.weight": "model-00086-of-00101.safetensors", + "model.layers.79.mlp.experts.19.gate_proj.weight": "model-00086-of-00101.safetensors", + "model.layers.79.mlp.experts.19.up_proj.weight": "model-00086-of-00101.safetensors", + "model.layers.79.mlp.experts.19.down_proj.weight": "model-00086-of-00101.safetensors", + "model.layers.79.mlp.experts.20.gate_proj.weight": "model-00086-of-00101.safetensors", + "model.layers.79.mlp.experts.20.up_proj.weight": "model-00086-of-00101.safetensors", + "model.layers.79.mlp.experts.20.down_proj.weight": "model-00086-of-00101.safetensors", + "model.layers.79.mlp.experts.21.gate_proj.weight": "model-00086-of-00101.safetensors", + "model.layers.79.mlp.experts.21.up_proj.weight": "model-00086-of-00101.safetensors", + "model.layers.79.mlp.experts.21.down_proj.weight": "model-00086-of-00101.safetensors", + "model.layers.79.mlp.experts.22.gate_proj.weight": "model-00086-of-00101.safetensors", + "model.layers.79.mlp.experts.22.up_proj.weight": "model-00086-of-00101.safetensors", + "model.layers.79.mlp.experts.22.down_proj.weight": "model-00086-of-00101.safetensors", + "model.layers.79.mlp.experts.23.gate_proj.weight": "model-00086-of-00101.safetensors", + "model.layers.79.mlp.experts.23.up_proj.weight": "model-00086-of-00101.safetensors", + "model.layers.79.mlp.experts.23.down_proj.weight": "model-00086-of-00101.safetensors", + "model.layers.79.mlp.experts.24.gate_proj.weight": "model-00086-of-00101.safetensors", + "model.layers.79.mlp.experts.24.up_proj.weight": "model-00086-of-00101.safetensors", + "model.layers.79.mlp.experts.24.down_proj.weight": "model-00086-of-00101.safetensors", + "model.layers.79.mlp.experts.25.gate_proj.weight": "model-00086-of-00101.safetensors", + "model.layers.79.mlp.experts.25.up_proj.weight": "model-00086-of-00101.safetensors", + "model.layers.79.mlp.experts.25.down_proj.weight": "model-00086-of-00101.safetensors", + "model.layers.79.mlp.experts.26.gate_proj.weight": "model-00086-of-00101.safetensors", + "model.layers.79.mlp.experts.26.up_proj.weight": "model-00086-of-00101.safetensors", + "model.layers.79.mlp.experts.26.down_proj.weight": "model-00086-of-00101.safetensors", + "model.layers.79.mlp.experts.27.gate_proj.weight": "model-00086-of-00101.safetensors", + "model.layers.79.mlp.experts.27.up_proj.weight": "model-00086-of-00101.safetensors", + "model.layers.79.mlp.experts.27.down_proj.weight": "model-00086-of-00101.safetensors", + "model.layers.79.mlp.experts.28.gate_proj.weight": "model-00086-of-00101.safetensors", + "model.layers.79.mlp.experts.28.up_proj.weight": "model-00086-of-00101.safetensors", + "model.layers.79.mlp.experts.28.down_proj.weight": "model-00086-of-00101.safetensors", + "model.layers.79.mlp.experts.29.gate_proj.weight": "model-00086-of-00101.safetensors", + "model.layers.79.mlp.experts.29.up_proj.weight": "model-00086-of-00101.safetensors", + "model.layers.79.mlp.experts.29.down_proj.weight": "model-00086-of-00101.safetensors", + "model.layers.79.mlp.experts.30.gate_proj.weight": "model-00086-of-00101.safetensors", + "model.layers.79.mlp.experts.30.up_proj.weight": "model-00086-of-00101.safetensors", + "model.layers.79.mlp.experts.30.down_proj.weight": "model-00086-of-00101.safetensors", + "model.layers.79.mlp.experts.31.gate_proj.weight": "model-00086-of-00101.safetensors", + "model.layers.79.mlp.experts.31.up_proj.weight": "model-00086-of-00101.safetensors", + "model.layers.79.mlp.experts.31.down_proj.weight": "model-00086-of-00101.safetensors", + "model.layers.79.mlp.experts.32.gate_proj.weight": "model-00086-of-00101.safetensors", + "model.layers.79.mlp.experts.32.up_proj.weight": "model-00086-of-00101.safetensors", + "model.layers.79.mlp.experts.32.down_proj.weight": "model-00086-of-00101.safetensors", + "model.layers.79.mlp.experts.33.gate_proj.weight": "model-00086-of-00101.safetensors", + "model.layers.79.mlp.experts.33.up_proj.weight": "model-00086-of-00101.safetensors", + "model.layers.79.mlp.experts.33.down_proj.weight": "model-00086-of-00101.safetensors", + "model.layers.79.mlp.experts.34.gate_proj.weight": "model-00086-of-00101.safetensors", + "model.layers.79.mlp.experts.34.up_proj.weight": "model-00086-of-00101.safetensors", + "model.layers.79.mlp.experts.34.down_proj.weight": "model-00086-of-00101.safetensors", + "model.layers.79.mlp.experts.35.gate_proj.weight": "model-00086-of-00101.safetensors", + "model.layers.79.mlp.experts.35.up_proj.weight": "model-00086-of-00101.safetensors", + "model.layers.79.mlp.experts.35.down_proj.weight": "model-00086-of-00101.safetensors", + "model.layers.79.mlp.experts.36.gate_proj.weight": "model-00086-of-00101.safetensors", + "model.layers.79.mlp.experts.36.up_proj.weight": "model-00086-of-00101.safetensors", + "model.layers.79.mlp.experts.36.down_proj.weight": "model-00086-of-00101.safetensors", + "model.layers.79.mlp.experts.37.gate_proj.weight": "model-00086-of-00101.safetensors", + "model.layers.79.mlp.experts.37.up_proj.weight": "model-00086-of-00101.safetensors", + "model.layers.79.mlp.experts.37.down_proj.weight": "model-00086-of-00101.safetensors", + "model.layers.79.mlp.experts.38.gate_proj.weight": "model-00086-of-00101.safetensors", + "model.layers.79.mlp.experts.38.up_proj.weight": "model-00086-of-00101.safetensors", + "model.layers.79.mlp.experts.38.down_proj.weight": "model-00086-of-00101.safetensors", + "model.layers.79.mlp.experts.39.gate_proj.weight": "model-00086-of-00101.safetensors", + "model.layers.79.mlp.experts.39.up_proj.weight": "model-00086-of-00101.safetensors", + "model.layers.79.mlp.experts.39.down_proj.weight": "model-00086-of-00101.safetensors", + "model.layers.79.mlp.experts.40.gate_proj.weight": "model-00086-of-00101.safetensors", + "model.layers.79.mlp.experts.40.up_proj.weight": "model-00086-of-00101.safetensors", + "model.layers.79.mlp.experts.40.down_proj.weight": "model-00086-of-00101.safetensors", + "model.layers.79.mlp.experts.41.gate_proj.weight": "model-00087-of-00101.safetensors", + "model.layers.79.mlp.experts.41.up_proj.weight": "model-00087-of-00101.safetensors", + "model.layers.79.mlp.experts.41.down_proj.weight": "model-00087-of-00101.safetensors", + "model.layers.79.mlp.experts.42.gate_proj.weight": "model-00087-of-00101.safetensors", + "model.layers.79.mlp.experts.42.up_proj.weight": "model-00087-of-00101.safetensors", + "model.layers.79.mlp.experts.42.down_proj.weight": "model-00087-of-00101.safetensors", + "model.layers.79.mlp.experts.43.gate_proj.weight": "model-00087-of-00101.safetensors", + "model.layers.79.mlp.experts.43.up_proj.weight": "model-00087-of-00101.safetensors", + "model.layers.79.mlp.experts.43.down_proj.weight": "model-00087-of-00101.safetensors", + "model.layers.79.mlp.experts.44.gate_proj.weight": "model-00087-of-00101.safetensors", + "model.layers.79.mlp.experts.44.up_proj.weight": "model-00087-of-00101.safetensors", + "model.layers.79.mlp.experts.44.down_proj.weight": "model-00087-of-00101.safetensors", + "model.layers.79.mlp.experts.45.gate_proj.weight": "model-00087-of-00101.safetensors", + "model.layers.79.mlp.experts.45.up_proj.weight": "model-00087-of-00101.safetensors", + "model.layers.79.mlp.experts.45.down_proj.weight": "model-00087-of-00101.safetensors", + "model.layers.79.mlp.experts.46.gate_proj.weight": "model-00087-of-00101.safetensors", + "model.layers.79.mlp.experts.46.up_proj.weight": "model-00087-of-00101.safetensors", + "model.layers.79.mlp.experts.46.down_proj.weight": "model-00087-of-00101.safetensors", + "model.layers.79.mlp.experts.47.gate_proj.weight": "model-00087-of-00101.safetensors", + "model.layers.79.mlp.experts.47.up_proj.weight": "model-00087-of-00101.safetensors", + "model.layers.79.mlp.experts.47.down_proj.weight": "model-00087-of-00101.safetensors", + "model.layers.79.mlp.experts.48.gate_proj.weight": "model-00087-of-00101.safetensors", + "model.layers.79.mlp.experts.48.up_proj.weight": "model-00087-of-00101.safetensors", + "model.layers.79.mlp.experts.48.down_proj.weight": "model-00087-of-00101.safetensors", + "model.layers.79.mlp.experts.49.gate_proj.weight": "model-00087-of-00101.safetensors", + "model.layers.79.mlp.experts.49.up_proj.weight": "model-00087-of-00101.safetensors", + "model.layers.79.mlp.experts.49.down_proj.weight": "model-00087-of-00101.safetensors", + "model.layers.79.mlp.experts.50.gate_proj.weight": "model-00087-of-00101.safetensors", + "model.layers.79.mlp.experts.50.up_proj.weight": "model-00087-of-00101.safetensors", + "model.layers.79.mlp.experts.50.down_proj.weight": "model-00087-of-00101.safetensors", + "model.layers.79.mlp.experts.51.gate_proj.weight": "model-00087-of-00101.safetensors", + "model.layers.79.mlp.experts.51.up_proj.weight": "model-00087-of-00101.safetensors", + "model.layers.79.mlp.experts.51.down_proj.weight": "model-00087-of-00101.safetensors", + "model.layers.79.mlp.experts.52.gate_proj.weight": "model-00087-of-00101.safetensors", + "model.layers.79.mlp.experts.52.up_proj.weight": "model-00087-of-00101.safetensors", + "model.layers.79.mlp.experts.52.down_proj.weight": "model-00087-of-00101.safetensors", + "model.layers.79.mlp.experts.53.gate_proj.weight": "model-00087-of-00101.safetensors", + "model.layers.79.mlp.experts.53.up_proj.weight": "model-00087-of-00101.safetensors", + "model.layers.79.mlp.experts.53.down_proj.weight": "model-00087-of-00101.safetensors", + "model.layers.79.mlp.experts.54.gate_proj.weight": "model-00087-of-00101.safetensors", + "model.layers.79.mlp.experts.54.up_proj.weight": "model-00087-of-00101.safetensors", + "model.layers.79.mlp.experts.54.down_proj.weight": "model-00087-of-00101.safetensors", + "model.layers.79.mlp.experts.55.gate_proj.weight": "model-00087-of-00101.safetensors", + "model.layers.79.mlp.experts.55.up_proj.weight": "model-00087-of-00101.safetensors", + "model.layers.79.mlp.experts.55.down_proj.weight": "model-00087-of-00101.safetensors", + "model.layers.79.mlp.experts.56.gate_proj.weight": "model-00087-of-00101.safetensors", + "model.layers.79.mlp.experts.56.up_proj.weight": "model-00087-of-00101.safetensors", + "model.layers.79.mlp.experts.56.down_proj.weight": "model-00087-of-00101.safetensors", + "model.layers.79.mlp.experts.57.gate_proj.weight": "model-00087-of-00101.safetensors", + "model.layers.79.mlp.experts.57.up_proj.weight": "model-00087-of-00101.safetensors", + "model.layers.79.mlp.experts.57.down_proj.weight": "model-00087-of-00101.safetensors", + "model.layers.79.mlp.experts.58.gate_proj.weight": "model-00087-of-00101.safetensors", + "model.layers.79.mlp.experts.58.up_proj.weight": "model-00087-of-00101.safetensors", + "model.layers.79.mlp.experts.58.down_proj.weight": "model-00087-of-00101.safetensors", + "model.layers.79.mlp.experts.59.gate_proj.weight": "model-00087-of-00101.safetensors", + "model.layers.79.mlp.experts.59.up_proj.weight": "model-00087-of-00101.safetensors", + "model.layers.79.mlp.experts.59.down_proj.weight": "model-00087-of-00101.safetensors", + "model.layers.79.mlp.experts.60.gate_proj.weight": "model-00087-of-00101.safetensors", + "model.layers.79.mlp.experts.60.up_proj.weight": "model-00087-of-00101.safetensors", + "model.layers.79.mlp.experts.60.down_proj.weight": "model-00087-of-00101.safetensors", + "model.layers.79.mlp.experts.61.gate_proj.weight": "model-00087-of-00101.safetensors", + "model.layers.79.mlp.experts.61.up_proj.weight": "model-00087-of-00101.safetensors", + "model.layers.79.mlp.experts.61.down_proj.weight": "model-00087-of-00101.safetensors", + "model.layers.79.mlp.experts.62.gate_proj.weight": "model-00087-of-00101.safetensors", + "model.layers.79.mlp.experts.62.up_proj.weight": "model-00087-of-00101.safetensors", + "model.layers.79.mlp.experts.62.down_proj.weight": "model-00087-of-00101.safetensors", + "model.layers.79.mlp.experts.63.gate_proj.weight": "model-00087-of-00101.safetensors", + "model.layers.79.mlp.experts.63.up_proj.weight": "model-00087-of-00101.safetensors", + "model.layers.79.mlp.experts.63.down_proj.weight": "model-00087-of-00101.safetensors", + "model.layers.79.mlp.experts.64.gate_proj.weight": "model-00087-of-00101.safetensors", + "model.layers.79.mlp.experts.64.up_proj.weight": "model-00087-of-00101.safetensors", + "model.layers.79.mlp.experts.64.down_proj.weight": "model-00087-of-00101.safetensors", + "model.layers.79.mlp.experts.65.gate_proj.weight": "model-00087-of-00101.safetensors", + "model.layers.79.mlp.experts.65.up_proj.weight": "model-00087-of-00101.safetensors", + "model.layers.79.mlp.experts.65.down_proj.weight": "model-00087-of-00101.safetensors", + "model.layers.79.mlp.experts.66.gate_proj.weight": "model-00087-of-00101.safetensors", + "model.layers.79.mlp.experts.66.up_proj.weight": "model-00087-of-00101.safetensors", + "model.layers.79.mlp.experts.66.down_proj.weight": "model-00087-of-00101.safetensors", + "model.layers.79.mlp.experts.67.gate_proj.weight": "model-00087-of-00101.safetensors", + "model.layers.79.mlp.experts.67.up_proj.weight": "model-00087-of-00101.safetensors", + "model.layers.79.mlp.experts.67.down_proj.weight": "model-00087-of-00101.safetensors", + "model.layers.79.mlp.experts.68.gate_proj.weight": "model-00087-of-00101.safetensors", + "model.layers.79.mlp.experts.68.up_proj.weight": "model-00087-of-00101.safetensors", + "model.layers.79.mlp.experts.68.down_proj.weight": "model-00087-of-00101.safetensors", + "model.layers.79.mlp.experts.69.gate_proj.weight": "model-00087-of-00101.safetensors", + "model.layers.79.mlp.experts.69.up_proj.weight": "model-00087-of-00101.safetensors", + "model.layers.79.mlp.experts.69.down_proj.weight": "model-00087-of-00101.safetensors", + "model.layers.79.mlp.experts.70.gate_proj.weight": "model-00087-of-00101.safetensors", + "model.layers.79.mlp.experts.70.up_proj.weight": "model-00087-of-00101.safetensors", + "model.layers.79.mlp.experts.70.down_proj.weight": "model-00087-of-00101.safetensors", + "model.layers.79.mlp.experts.71.gate_proj.weight": "model-00087-of-00101.safetensors", + "model.layers.79.mlp.experts.71.up_proj.weight": "model-00087-of-00101.safetensors", + "model.layers.79.mlp.experts.71.down_proj.weight": "model-00087-of-00101.safetensors", + "model.layers.79.mlp.experts.72.gate_proj.weight": "model-00087-of-00101.safetensors", + "model.layers.79.mlp.experts.72.up_proj.weight": "model-00087-of-00101.safetensors", + "model.layers.79.mlp.experts.72.down_proj.weight": "model-00087-of-00101.safetensors", + "model.layers.79.mlp.experts.73.gate_proj.weight": "model-00087-of-00101.safetensors", + "model.layers.79.mlp.experts.73.up_proj.weight": "model-00087-of-00101.safetensors", + "model.layers.79.mlp.experts.73.down_proj.weight": "model-00087-of-00101.safetensors", + "model.layers.79.mlp.experts.74.gate_proj.weight": "model-00087-of-00101.safetensors", + "model.layers.79.mlp.experts.74.up_proj.weight": "model-00087-of-00101.safetensors", + "model.layers.79.mlp.experts.74.down_proj.weight": "model-00087-of-00101.safetensors", + "model.layers.79.mlp.experts.75.gate_proj.weight": "model-00087-of-00101.safetensors", + "model.layers.79.mlp.experts.75.up_proj.weight": "model-00087-of-00101.safetensors", + "model.layers.79.mlp.experts.75.down_proj.weight": "model-00087-of-00101.safetensors", + "model.layers.79.mlp.experts.76.gate_proj.weight": "model-00087-of-00101.safetensors", + "model.layers.79.mlp.experts.76.up_proj.weight": "model-00087-of-00101.safetensors", + "model.layers.79.mlp.experts.76.down_proj.weight": "model-00087-of-00101.safetensors", + "model.layers.79.mlp.experts.77.gate_proj.weight": "model-00087-of-00101.safetensors", + "model.layers.79.mlp.experts.77.up_proj.weight": "model-00087-of-00101.safetensors", + "model.layers.79.mlp.experts.77.down_proj.weight": "model-00087-of-00101.safetensors", + "model.layers.79.mlp.experts.78.gate_proj.weight": "model-00087-of-00101.safetensors", + "model.layers.79.mlp.experts.78.up_proj.weight": "model-00087-of-00101.safetensors", + "model.layers.79.mlp.experts.78.down_proj.weight": "model-00087-of-00101.safetensors", + "model.layers.79.mlp.experts.79.gate_proj.weight": "model-00087-of-00101.safetensors", + "model.layers.79.mlp.experts.79.up_proj.weight": "model-00087-of-00101.safetensors", + "model.layers.79.mlp.experts.79.down_proj.weight": "model-00087-of-00101.safetensors", + "model.layers.79.mlp.experts.80.gate_proj.weight": "model-00087-of-00101.safetensors", + "model.layers.79.mlp.experts.80.up_proj.weight": "model-00087-of-00101.safetensors", + "model.layers.79.mlp.experts.80.down_proj.weight": "model-00087-of-00101.safetensors", + "model.layers.79.mlp.experts.81.gate_proj.weight": "model-00087-of-00101.safetensors", + "model.layers.79.mlp.experts.81.up_proj.weight": "model-00087-of-00101.safetensors", + "model.layers.79.mlp.experts.81.down_proj.weight": "model-00087-of-00101.safetensors", + "model.layers.79.mlp.experts.82.gate_proj.weight": "model-00087-of-00101.safetensors", + "model.layers.79.mlp.experts.82.up_proj.weight": "model-00087-of-00101.safetensors", + "model.layers.79.mlp.experts.82.down_proj.weight": "model-00087-of-00101.safetensors", + "model.layers.79.mlp.experts.83.gate_proj.weight": "model-00087-of-00101.safetensors", + "model.layers.79.mlp.experts.83.up_proj.weight": "model-00087-of-00101.safetensors", + "model.layers.79.mlp.experts.83.down_proj.weight": "model-00087-of-00101.safetensors", + "model.layers.79.mlp.experts.84.gate_proj.weight": "model-00087-of-00101.safetensors", + "model.layers.79.mlp.experts.84.up_proj.weight": "model-00087-of-00101.safetensors", + "model.layers.79.mlp.experts.84.down_proj.weight": "model-00087-of-00101.safetensors", + "model.layers.79.mlp.experts.85.gate_proj.weight": "model-00087-of-00101.safetensors", + "model.layers.79.mlp.experts.85.up_proj.weight": "model-00087-of-00101.safetensors", + "model.layers.79.mlp.experts.85.down_proj.weight": "model-00087-of-00101.safetensors", + "model.layers.79.mlp.experts.86.gate_proj.weight": "model-00087-of-00101.safetensors", + "model.layers.79.mlp.experts.86.up_proj.weight": "model-00087-of-00101.safetensors", + "model.layers.79.mlp.experts.86.down_proj.weight": "model-00087-of-00101.safetensors", + "model.layers.79.mlp.experts.87.gate_proj.weight": "model-00087-of-00101.safetensors", + "model.layers.79.mlp.experts.87.up_proj.weight": "model-00087-of-00101.safetensors", + "model.layers.79.mlp.experts.87.down_proj.weight": "model-00087-of-00101.safetensors", + "model.layers.79.mlp.experts.88.gate_proj.weight": "model-00087-of-00101.safetensors", + "model.layers.79.mlp.experts.88.up_proj.weight": "model-00087-of-00101.safetensors", + "model.layers.79.mlp.experts.88.down_proj.weight": "model-00087-of-00101.safetensors", + "model.layers.79.mlp.experts.89.gate_proj.weight": "model-00087-of-00101.safetensors", + "model.layers.79.mlp.experts.89.up_proj.weight": "model-00087-of-00101.safetensors", + "model.layers.79.mlp.experts.89.down_proj.weight": "model-00087-of-00101.safetensors", + "model.layers.79.mlp.experts.90.gate_proj.weight": "model-00087-of-00101.safetensors", + "model.layers.79.mlp.experts.90.up_proj.weight": "model-00087-of-00101.safetensors", + "model.layers.79.mlp.experts.90.down_proj.weight": "model-00087-of-00101.safetensors", + "model.layers.79.mlp.experts.91.gate_proj.weight": "model-00087-of-00101.safetensors", + "model.layers.79.mlp.experts.91.up_proj.weight": "model-00087-of-00101.safetensors", + "model.layers.79.mlp.experts.91.down_proj.weight": "model-00087-of-00101.safetensors", + "model.layers.79.mlp.experts.92.gate_proj.weight": "model-00087-of-00101.safetensors", + "model.layers.79.mlp.experts.92.up_proj.weight": "model-00087-of-00101.safetensors", + "model.layers.79.mlp.experts.92.down_proj.weight": "model-00087-of-00101.safetensors", + "model.layers.79.mlp.experts.93.gate_proj.weight": "model-00087-of-00101.safetensors", + "model.layers.79.mlp.experts.93.up_proj.weight": "model-00087-of-00101.safetensors", + "model.layers.79.mlp.experts.93.down_proj.weight": "model-00087-of-00101.safetensors", + "model.layers.79.mlp.experts.94.gate_proj.weight": "model-00087-of-00101.safetensors", + "model.layers.79.mlp.experts.94.up_proj.weight": "model-00087-of-00101.safetensors", + "model.layers.79.mlp.experts.94.down_proj.weight": "model-00087-of-00101.safetensors", + "model.layers.79.mlp.experts.95.gate_proj.weight": "model-00087-of-00101.safetensors", + "model.layers.79.mlp.experts.95.up_proj.weight": "model-00087-of-00101.safetensors", + "model.layers.79.mlp.experts.95.down_proj.weight": "model-00087-of-00101.safetensors", + "model.layers.79.mlp.experts.96.gate_proj.weight": "model-00087-of-00101.safetensors", + "model.layers.79.mlp.experts.96.up_proj.weight": "model-00087-of-00101.safetensors", + "model.layers.79.mlp.experts.96.down_proj.weight": "model-00087-of-00101.safetensors", + "model.layers.79.mlp.experts.97.gate_proj.weight": "model-00087-of-00101.safetensors", + "model.layers.79.mlp.experts.97.up_proj.weight": "model-00087-of-00101.safetensors", + "model.layers.79.mlp.experts.97.down_proj.weight": "model-00087-of-00101.safetensors", + "model.layers.79.mlp.experts.98.gate_proj.weight": "model-00087-of-00101.safetensors", + "model.layers.79.mlp.experts.98.up_proj.weight": "model-00087-of-00101.safetensors", + "model.layers.79.mlp.experts.98.down_proj.weight": "model-00087-of-00101.safetensors", + "model.layers.79.mlp.experts.99.gate_proj.weight": "model-00087-of-00101.safetensors", + "model.layers.79.mlp.experts.99.up_proj.weight": "model-00087-of-00101.safetensors", + "model.layers.79.mlp.experts.99.down_proj.weight": "model-00087-of-00101.safetensors", + "model.layers.79.mlp.experts.100.gate_proj.weight": "model-00087-of-00101.safetensors", + "model.layers.79.mlp.experts.100.up_proj.weight": "model-00087-of-00101.safetensors", + "model.layers.79.mlp.experts.100.down_proj.weight": "model-00087-of-00101.safetensors", + "model.layers.79.mlp.experts.101.gate_proj.weight": "model-00087-of-00101.safetensors", + "model.layers.79.mlp.experts.101.up_proj.weight": "model-00087-of-00101.safetensors", + "model.layers.79.mlp.experts.101.down_proj.weight": "model-00087-of-00101.safetensors", + "model.layers.79.mlp.experts.102.gate_proj.weight": "model-00087-of-00101.safetensors", + "model.layers.79.mlp.experts.102.up_proj.weight": "model-00087-of-00101.safetensors", + "model.layers.79.mlp.experts.102.down_proj.weight": "model-00087-of-00101.safetensors", + "model.layers.79.mlp.experts.103.gate_proj.weight": "model-00087-of-00101.safetensors", + "model.layers.79.mlp.experts.103.up_proj.weight": "model-00087-of-00101.safetensors", + "model.layers.79.mlp.experts.103.down_proj.weight": "model-00087-of-00101.safetensors", + "model.layers.79.mlp.experts.104.gate_proj.weight": "model-00087-of-00101.safetensors", + "model.layers.79.mlp.experts.104.up_proj.weight": "model-00087-of-00101.safetensors", + "model.layers.79.mlp.experts.104.down_proj.weight": "model-00087-of-00101.safetensors", + "model.layers.79.mlp.experts.105.gate_proj.weight": "model-00087-of-00101.safetensors", + "model.layers.79.mlp.experts.105.up_proj.weight": "model-00087-of-00101.safetensors", + "model.layers.79.mlp.experts.105.down_proj.weight": "model-00087-of-00101.safetensors", + "model.layers.79.mlp.experts.106.gate_proj.weight": "model-00087-of-00101.safetensors", + "model.layers.79.mlp.experts.106.up_proj.weight": "model-00087-of-00101.safetensors", + "model.layers.79.mlp.experts.106.down_proj.weight": "model-00087-of-00101.safetensors", + "model.layers.79.mlp.experts.107.gate_proj.weight": "model-00087-of-00101.safetensors", + "model.layers.79.mlp.experts.107.up_proj.weight": "model-00087-of-00101.safetensors", + "model.layers.79.mlp.experts.107.down_proj.weight": "model-00087-of-00101.safetensors", + "model.layers.79.mlp.experts.108.gate_proj.weight": "model-00087-of-00101.safetensors", + "model.layers.79.mlp.experts.108.up_proj.weight": "model-00087-of-00101.safetensors", + "model.layers.79.mlp.experts.108.down_proj.weight": "model-00087-of-00101.safetensors", + "model.layers.79.mlp.experts.109.gate_proj.weight": "model-00087-of-00101.safetensors", + "model.layers.79.mlp.experts.109.up_proj.weight": "model-00087-of-00101.safetensors", + "model.layers.79.mlp.experts.109.down_proj.weight": "model-00087-of-00101.safetensors", + "model.layers.79.mlp.experts.110.gate_proj.weight": "model-00087-of-00101.safetensors", + "model.layers.79.mlp.experts.110.up_proj.weight": "model-00087-of-00101.safetensors", + "model.layers.79.mlp.experts.110.down_proj.weight": "model-00087-of-00101.safetensors", + "model.layers.79.mlp.experts.111.gate_proj.weight": "model-00087-of-00101.safetensors", + "model.layers.79.mlp.experts.111.up_proj.weight": "model-00087-of-00101.safetensors", + "model.layers.79.mlp.experts.111.down_proj.weight": "model-00087-of-00101.safetensors", + "model.layers.79.mlp.experts.112.gate_proj.weight": "model-00087-of-00101.safetensors", + "model.layers.79.mlp.experts.112.up_proj.weight": "model-00087-of-00101.safetensors", + "model.layers.79.mlp.experts.112.down_proj.weight": "model-00087-of-00101.safetensors", + "model.layers.79.mlp.experts.113.gate_proj.weight": "model-00087-of-00101.safetensors", + "model.layers.79.mlp.experts.113.up_proj.weight": "model-00087-of-00101.safetensors", + "model.layers.79.mlp.experts.113.down_proj.weight": "model-00087-of-00101.safetensors", + "model.layers.79.mlp.experts.114.gate_proj.weight": "model-00087-of-00101.safetensors", + "model.layers.79.mlp.experts.114.up_proj.weight": "model-00087-of-00101.safetensors", + "model.layers.79.mlp.experts.114.down_proj.weight": "model-00087-of-00101.safetensors", + "model.layers.79.mlp.experts.115.gate_proj.weight": "model-00087-of-00101.safetensors", + "model.layers.79.mlp.experts.115.up_proj.weight": "model-00087-of-00101.safetensors", + "model.layers.79.mlp.experts.115.down_proj.weight": "model-00087-of-00101.safetensors", + "model.layers.79.mlp.experts.116.gate_proj.weight": "model-00087-of-00101.safetensors", + "model.layers.79.mlp.experts.116.up_proj.weight": "model-00087-of-00101.safetensors", + "model.layers.79.mlp.experts.116.down_proj.weight": "model-00087-of-00101.safetensors", + "model.layers.79.mlp.experts.117.gate_proj.weight": "model-00087-of-00101.safetensors", + "model.layers.79.mlp.experts.117.up_proj.weight": "model-00087-of-00101.safetensors", + "model.layers.79.mlp.experts.117.down_proj.weight": "model-00087-of-00101.safetensors", + "model.layers.79.mlp.experts.118.gate_proj.weight": "model-00087-of-00101.safetensors", + "model.layers.79.mlp.experts.118.up_proj.weight": "model-00087-of-00101.safetensors", + "model.layers.79.mlp.experts.118.down_proj.weight": "model-00087-of-00101.safetensors", + "model.layers.79.mlp.experts.119.gate_proj.weight": "model-00087-of-00101.safetensors", + "model.layers.79.mlp.experts.119.up_proj.weight": "model-00087-of-00101.safetensors", + "model.layers.79.mlp.experts.119.down_proj.weight": "model-00087-of-00101.safetensors", + "model.layers.79.mlp.gate.weight": "model-00087-of-00101.safetensors", + "model.layers.79.mlp.gate.e_score_correction_bias": "model-00087-of-00101.safetensors", + "model.layers.79.mlp.shared_experts.gate_proj.weight": "model-00087-of-00101.safetensors", + "model.layers.79.mlp.shared_experts.up_proj.weight": "model-00087-of-00101.safetensors", + "model.layers.79.mlp.shared_experts.down_proj.weight": "model-00087-of-00101.safetensors", + "model.layers.79.input_layernorm.weight": "model-00087-of-00101.safetensors", + "model.layers.79.post_attention_layernorm.weight": "model-00087-of-00101.safetensors", + "model.layers.80.self_attn.q_proj.weight": "model-00087-of-00101.safetensors", + "model.layers.80.self_attn.q_proj.bias": "model-00087-of-00101.safetensors", + "model.layers.80.self_attn.k_proj.weight": "model-00087-of-00101.safetensors", + "model.layers.80.self_attn.k_proj.bias": "model-00087-of-00101.safetensors", + "model.layers.80.self_attn.v_proj.weight": "model-00087-of-00101.safetensors", + "model.layers.80.self_attn.v_proj.bias": "model-00087-of-00101.safetensors", + "model.layers.80.self_attn.o_proj.weight": "model-00087-of-00101.safetensors", + "model.layers.80.self_attn.q_norm.weight": "model-00087-of-00101.safetensors", + "model.layers.80.self_attn.k_norm.weight": "model-00087-of-00101.safetensors", + "model.layers.80.mlp.experts.0.gate_proj.weight": "model-00087-of-00101.safetensors", + "model.layers.80.mlp.experts.0.up_proj.weight": "model-00087-of-00101.safetensors", + "model.layers.80.mlp.experts.0.down_proj.weight": "model-00087-of-00101.safetensors", + "model.layers.80.mlp.experts.1.gate_proj.weight": "model-00087-of-00101.safetensors", + "model.layers.80.mlp.experts.1.up_proj.weight": "model-00087-of-00101.safetensors", + "model.layers.80.mlp.experts.1.down_proj.weight": "model-00087-of-00101.safetensors", + "model.layers.80.mlp.experts.2.gate_proj.weight": "model-00087-of-00101.safetensors", + "model.layers.80.mlp.experts.2.up_proj.weight": "model-00087-of-00101.safetensors", + "model.layers.80.mlp.experts.2.down_proj.weight": "model-00087-of-00101.safetensors", + "model.layers.80.mlp.experts.3.gate_proj.weight": "model-00087-of-00101.safetensors", + "model.layers.80.mlp.experts.3.up_proj.weight": "model-00087-of-00101.safetensors", + "model.layers.80.mlp.experts.3.down_proj.weight": "model-00087-of-00101.safetensors", + "model.layers.80.mlp.experts.4.gate_proj.weight": "model-00087-of-00101.safetensors", + "model.layers.80.mlp.experts.4.up_proj.weight": "model-00087-of-00101.safetensors", + "model.layers.80.mlp.experts.4.down_proj.weight": "model-00087-of-00101.safetensors", + "model.layers.80.mlp.experts.5.gate_proj.weight": "model-00087-of-00101.safetensors", + "model.layers.80.mlp.experts.5.up_proj.weight": "model-00087-of-00101.safetensors", + "model.layers.80.mlp.experts.5.down_proj.weight": "model-00087-of-00101.safetensors", + "model.layers.80.mlp.experts.6.gate_proj.weight": "model-00087-of-00101.safetensors", + "model.layers.80.mlp.experts.6.up_proj.weight": "model-00087-of-00101.safetensors", + "model.layers.80.mlp.experts.6.down_proj.weight": "model-00087-of-00101.safetensors", + "model.layers.80.mlp.experts.7.gate_proj.weight": "model-00087-of-00101.safetensors", + "model.layers.80.mlp.experts.7.up_proj.weight": "model-00087-of-00101.safetensors", + "model.layers.80.mlp.experts.7.down_proj.weight": "model-00087-of-00101.safetensors", + "model.layers.80.mlp.experts.8.gate_proj.weight": "model-00087-of-00101.safetensors", + "model.layers.80.mlp.experts.8.up_proj.weight": "model-00087-of-00101.safetensors", + "model.layers.80.mlp.experts.8.down_proj.weight": "model-00087-of-00101.safetensors", + "model.layers.80.mlp.experts.9.gate_proj.weight": "model-00087-of-00101.safetensors", + "model.layers.80.mlp.experts.9.up_proj.weight": "model-00087-of-00101.safetensors", + "model.layers.80.mlp.experts.9.down_proj.weight": "model-00087-of-00101.safetensors", + "model.layers.80.mlp.experts.10.gate_proj.weight": "model-00087-of-00101.safetensors", + "model.layers.80.mlp.experts.10.up_proj.weight": "model-00087-of-00101.safetensors", + "model.layers.80.mlp.experts.10.down_proj.weight": "model-00087-of-00101.safetensors", + "model.layers.80.mlp.experts.11.gate_proj.weight": "model-00087-of-00101.safetensors", + "model.layers.80.mlp.experts.11.up_proj.weight": "model-00087-of-00101.safetensors", + "model.layers.80.mlp.experts.11.down_proj.weight": "model-00087-of-00101.safetensors", + "model.layers.80.mlp.experts.12.gate_proj.weight": "model-00087-of-00101.safetensors", + "model.layers.80.mlp.experts.12.up_proj.weight": "model-00087-of-00101.safetensors", + "model.layers.80.mlp.experts.12.down_proj.weight": "model-00087-of-00101.safetensors", + "model.layers.80.mlp.experts.13.gate_proj.weight": "model-00087-of-00101.safetensors", + "model.layers.80.mlp.experts.13.up_proj.weight": "model-00087-of-00101.safetensors", + "model.layers.80.mlp.experts.13.down_proj.weight": "model-00087-of-00101.safetensors", + "model.layers.80.mlp.experts.14.gate_proj.weight": "model-00087-of-00101.safetensors", + "model.layers.80.mlp.experts.14.up_proj.weight": "model-00087-of-00101.safetensors", + "model.layers.80.mlp.experts.14.down_proj.weight": "model-00087-of-00101.safetensors", + "model.layers.80.mlp.experts.15.gate_proj.weight": "model-00087-of-00101.safetensors", + "model.layers.80.mlp.experts.15.up_proj.weight": "model-00087-of-00101.safetensors", + "model.layers.80.mlp.experts.15.down_proj.weight": "model-00087-of-00101.safetensors", + "model.layers.80.mlp.experts.16.gate_proj.weight": "model-00087-of-00101.safetensors", + "model.layers.80.mlp.experts.16.up_proj.weight": "model-00087-of-00101.safetensors", + "model.layers.80.mlp.experts.16.down_proj.weight": "model-00087-of-00101.safetensors", + "model.layers.80.mlp.experts.17.gate_proj.weight": "model-00087-of-00101.safetensors", + "model.layers.80.mlp.experts.17.up_proj.weight": "model-00087-of-00101.safetensors", + "model.layers.80.mlp.experts.17.down_proj.weight": "model-00087-of-00101.safetensors", + "model.layers.80.mlp.experts.18.gate_proj.weight": "model-00087-of-00101.safetensors", + "model.layers.80.mlp.experts.18.up_proj.weight": "model-00087-of-00101.safetensors", + "model.layers.80.mlp.experts.18.down_proj.weight": "model-00087-of-00101.safetensors", + "model.layers.80.mlp.experts.19.gate_proj.weight": "model-00087-of-00101.safetensors", + "model.layers.80.mlp.experts.19.up_proj.weight": "model-00087-of-00101.safetensors", + "model.layers.80.mlp.experts.19.down_proj.weight": "model-00087-of-00101.safetensors", + "model.layers.80.mlp.experts.20.gate_proj.weight": "model-00087-of-00101.safetensors", + "model.layers.80.mlp.experts.20.up_proj.weight": "model-00087-of-00101.safetensors", + "model.layers.80.mlp.experts.20.down_proj.weight": "model-00087-of-00101.safetensors", + "model.layers.80.mlp.experts.21.gate_proj.weight": "model-00087-of-00101.safetensors", + "model.layers.80.mlp.experts.21.up_proj.weight": "model-00087-of-00101.safetensors", + "model.layers.80.mlp.experts.21.down_proj.weight": "model-00087-of-00101.safetensors", + "model.layers.80.mlp.experts.22.gate_proj.weight": "model-00087-of-00101.safetensors", + "model.layers.80.mlp.experts.22.up_proj.weight": "model-00087-of-00101.safetensors", + "model.layers.80.mlp.experts.22.down_proj.weight": "model-00087-of-00101.safetensors", + "model.layers.80.mlp.experts.23.gate_proj.weight": "model-00087-of-00101.safetensors", + "model.layers.80.mlp.experts.23.up_proj.weight": "model-00087-of-00101.safetensors", + "model.layers.80.mlp.experts.23.down_proj.weight": "model-00087-of-00101.safetensors", + "model.layers.80.mlp.experts.24.gate_proj.weight": "model-00087-of-00101.safetensors", + "model.layers.80.mlp.experts.24.up_proj.weight": "model-00087-of-00101.safetensors", + "model.layers.80.mlp.experts.24.down_proj.weight": "model-00087-of-00101.safetensors", + "model.layers.80.mlp.experts.25.gate_proj.weight": "model-00087-of-00101.safetensors", + "model.layers.80.mlp.experts.25.up_proj.weight": "model-00087-of-00101.safetensors", + "model.layers.80.mlp.experts.25.down_proj.weight": "model-00087-of-00101.safetensors", + "model.layers.80.mlp.experts.26.gate_proj.weight": "model-00087-of-00101.safetensors", + "model.layers.80.mlp.experts.26.up_proj.weight": "model-00087-of-00101.safetensors", + "model.layers.80.mlp.experts.26.down_proj.weight": "model-00087-of-00101.safetensors", + "model.layers.80.mlp.experts.27.gate_proj.weight": "model-00087-of-00101.safetensors", + "model.layers.80.mlp.experts.27.up_proj.weight": "model-00087-of-00101.safetensors", + "model.layers.80.mlp.experts.27.down_proj.weight": "model-00088-of-00101.safetensors", + "model.layers.80.mlp.experts.28.gate_proj.weight": "model-00088-of-00101.safetensors", + "model.layers.80.mlp.experts.28.up_proj.weight": "model-00088-of-00101.safetensors", + "model.layers.80.mlp.experts.28.down_proj.weight": "model-00088-of-00101.safetensors", + "model.layers.80.mlp.experts.29.gate_proj.weight": "model-00088-of-00101.safetensors", + "model.layers.80.mlp.experts.29.up_proj.weight": "model-00088-of-00101.safetensors", + "model.layers.80.mlp.experts.29.down_proj.weight": "model-00088-of-00101.safetensors", + "model.layers.80.mlp.experts.30.gate_proj.weight": "model-00088-of-00101.safetensors", + "model.layers.80.mlp.experts.30.up_proj.weight": "model-00088-of-00101.safetensors", + "model.layers.80.mlp.experts.30.down_proj.weight": "model-00088-of-00101.safetensors", + "model.layers.80.mlp.experts.31.gate_proj.weight": "model-00088-of-00101.safetensors", + "model.layers.80.mlp.experts.31.up_proj.weight": "model-00088-of-00101.safetensors", + "model.layers.80.mlp.experts.31.down_proj.weight": "model-00088-of-00101.safetensors", + "model.layers.80.mlp.experts.32.gate_proj.weight": "model-00088-of-00101.safetensors", + "model.layers.80.mlp.experts.32.up_proj.weight": "model-00088-of-00101.safetensors", + "model.layers.80.mlp.experts.32.down_proj.weight": "model-00088-of-00101.safetensors", + "model.layers.80.mlp.experts.33.gate_proj.weight": "model-00088-of-00101.safetensors", + "model.layers.80.mlp.experts.33.up_proj.weight": "model-00088-of-00101.safetensors", + "model.layers.80.mlp.experts.33.down_proj.weight": "model-00088-of-00101.safetensors", + "model.layers.80.mlp.experts.34.gate_proj.weight": "model-00088-of-00101.safetensors", + "model.layers.80.mlp.experts.34.up_proj.weight": "model-00088-of-00101.safetensors", + "model.layers.80.mlp.experts.34.down_proj.weight": "model-00088-of-00101.safetensors", + "model.layers.80.mlp.experts.35.gate_proj.weight": "model-00088-of-00101.safetensors", + "model.layers.80.mlp.experts.35.up_proj.weight": "model-00088-of-00101.safetensors", + "model.layers.80.mlp.experts.35.down_proj.weight": "model-00088-of-00101.safetensors", + "model.layers.80.mlp.experts.36.gate_proj.weight": "model-00088-of-00101.safetensors", + "model.layers.80.mlp.experts.36.up_proj.weight": "model-00088-of-00101.safetensors", + "model.layers.80.mlp.experts.36.down_proj.weight": "model-00088-of-00101.safetensors", + "model.layers.80.mlp.experts.37.gate_proj.weight": "model-00088-of-00101.safetensors", + "model.layers.80.mlp.experts.37.up_proj.weight": "model-00088-of-00101.safetensors", + "model.layers.80.mlp.experts.37.down_proj.weight": "model-00088-of-00101.safetensors", + "model.layers.80.mlp.experts.38.gate_proj.weight": "model-00088-of-00101.safetensors", + "model.layers.80.mlp.experts.38.up_proj.weight": "model-00088-of-00101.safetensors", + "model.layers.80.mlp.experts.38.down_proj.weight": "model-00088-of-00101.safetensors", + "model.layers.80.mlp.experts.39.gate_proj.weight": "model-00088-of-00101.safetensors", + "model.layers.80.mlp.experts.39.up_proj.weight": "model-00088-of-00101.safetensors", + "model.layers.80.mlp.experts.39.down_proj.weight": "model-00088-of-00101.safetensors", + "model.layers.80.mlp.experts.40.gate_proj.weight": "model-00088-of-00101.safetensors", + "model.layers.80.mlp.experts.40.up_proj.weight": "model-00088-of-00101.safetensors", + "model.layers.80.mlp.experts.40.down_proj.weight": "model-00088-of-00101.safetensors", + "model.layers.80.mlp.experts.41.gate_proj.weight": "model-00088-of-00101.safetensors", + "model.layers.80.mlp.experts.41.up_proj.weight": "model-00088-of-00101.safetensors", + "model.layers.80.mlp.experts.41.down_proj.weight": "model-00088-of-00101.safetensors", + "model.layers.80.mlp.experts.42.gate_proj.weight": "model-00088-of-00101.safetensors", + "model.layers.80.mlp.experts.42.up_proj.weight": "model-00088-of-00101.safetensors", + "model.layers.80.mlp.experts.42.down_proj.weight": "model-00088-of-00101.safetensors", + "model.layers.80.mlp.experts.43.gate_proj.weight": "model-00088-of-00101.safetensors", + "model.layers.80.mlp.experts.43.up_proj.weight": "model-00088-of-00101.safetensors", + "model.layers.80.mlp.experts.43.down_proj.weight": "model-00088-of-00101.safetensors", + "model.layers.80.mlp.experts.44.gate_proj.weight": "model-00088-of-00101.safetensors", + "model.layers.80.mlp.experts.44.up_proj.weight": "model-00088-of-00101.safetensors", + "model.layers.80.mlp.experts.44.down_proj.weight": "model-00088-of-00101.safetensors", + "model.layers.80.mlp.experts.45.gate_proj.weight": "model-00088-of-00101.safetensors", + "model.layers.80.mlp.experts.45.up_proj.weight": "model-00088-of-00101.safetensors", + "model.layers.80.mlp.experts.45.down_proj.weight": "model-00088-of-00101.safetensors", + "model.layers.80.mlp.experts.46.gate_proj.weight": "model-00088-of-00101.safetensors", + "model.layers.80.mlp.experts.46.up_proj.weight": "model-00088-of-00101.safetensors", + "model.layers.80.mlp.experts.46.down_proj.weight": "model-00088-of-00101.safetensors", + "model.layers.80.mlp.experts.47.gate_proj.weight": "model-00088-of-00101.safetensors", + "model.layers.80.mlp.experts.47.up_proj.weight": "model-00088-of-00101.safetensors", + "model.layers.80.mlp.experts.47.down_proj.weight": "model-00088-of-00101.safetensors", + "model.layers.80.mlp.experts.48.gate_proj.weight": "model-00088-of-00101.safetensors", + "model.layers.80.mlp.experts.48.up_proj.weight": "model-00088-of-00101.safetensors", + "model.layers.80.mlp.experts.48.down_proj.weight": "model-00088-of-00101.safetensors", + "model.layers.80.mlp.experts.49.gate_proj.weight": "model-00088-of-00101.safetensors", + "model.layers.80.mlp.experts.49.up_proj.weight": "model-00088-of-00101.safetensors", + "model.layers.80.mlp.experts.49.down_proj.weight": "model-00088-of-00101.safetensors", + "model.layers.80.mlp.experts.50.gate_proj.weight": "model-00088-of-00101.safetensors", + "model.layers.80.mlp.experts.50.up_proj.weight": "model-00088-of-00101.safetensors", + "model.layers.80.mlp.experts.50.down_proj.weight": "model-00088-of-00101.safetensors", + "model.layers.80.mlp.experts.51.gate_proj.weight": "model-00088-of-00101.safetensors", + "model.layers.80.mlp.experts.51.up_proj.weight": "model-00088-of-00101.safetensors", + "model.layers.80.mlp.experts.51.down_proj.weight": "model-00088-of-00101.safetensors", + "model.layers.80.mlp.experts.52.gate_proj.weight": "model-00088-of-00101.safetensors", + "model.layers.80.mlp.experts.52.up_proj.weight": "model-00088-of-00101.safetensors", + "model.layers.80.mlp.experts.52.down_proj.weight": "model-00088-of-00101.safetensors", + "model.layers.80.mlp.experts.53.gate_proj.weight": "model-00088-of-00101.safetensors", + "model.layers.80.mlp.experts.53.up_proj.weight": "model-00088-of-00101.safetensors", + "model.layers.80.mlp.experts.53.down_proj.weight": "model-00088-of-00101.safetensors", + "model.layers.80.mlp.experts.54.gate_proj.weight": "model-00088-of-00101.safetensors", + "model.layers.80.mlp.experts.54.up_proj.weight": "model-00088-of-00101.safetensors", + "model.layers.80.mlp.experts.54.down_proj.weight": "model-00088-of-00101.safetensors", + "model.layers.80.mlp.experts.55.gate_proj.weight": "model-00088-of-00101.safetensors", + "model.layers.80.mlp.experts.55.up_proj.weight": "model-00088-of-00101.safetensors", + "model.layers.80.mlp.experts.55.down_proj.weight": "model-00088-of-00101.safetensors", + "model.layers.80.mlp.experts.56.gate_proj.weight": "model-00088-of-00101.safetensors", + "model.layers.80.mlp.experts.56.up_proj.weight": "model-00088-of-00101.safetensors", + "model.layers.80.mlp.experts.56.down_proj.weight": "model-00088-of-00101.safetensors", + "model.layers.80.mlp.experts.57.gate_proj.weight": "model-00088-of-00101.safetensors", + "model.layers.80.mlp.experts.57.up_proj.weight": "model-00088-of-00101.safetensors", + "model.layers.80.mlp.experts.57.down_proj.weight": "model-00088-of-00101.safetensors", + "model.layers.80.mlp.experts.58.gate_proj.weight": "model-00088-of-00101.safetensors", + "model.layers.80.mlp.experts.58.up_proj.weight": "model-00088-of-00101.safetensors", + "model.layers.80.mlp.experts.58.down_proj.weight": "model-00088-of-00101.safetensors", + "model.layers.80.mlp.experts.59.gate_proj.weight": "model-00088-of-00101.safetensors", + "model.layers.80.mlp.experts.59.up_proj.weight": "model-00088-of-00101.safetensors", + "model.layers.80.mlp.experts.59.down_proj.weight": "model-00088-of-00101.safetensors", + "model.layers.80.mlp.experts.60.gate_proj.weight": "model-00088-of-00101.safetensors", + "model.layers.80.mlp.experts.60.up_proj.weight": "model-00088-of-00101.safetensors", + "model.layers.80.mlp.experts.60.down_proj.weight": "model-00088-of-00101.safetensors", + "model.layers.80.mlp.experts.61.gate_proj.weight": "model-00088-of-00101.safetensors", + "model.layers.80.mlp.experts.61.up_proj.weight": "model-00088-of-00101.safetensors", + "model.layers.80.mlp.experts.61.down_proj.weight": "model-00088-of-00101.safetensors", + "model.layers.80.mlp.experts.62.gate_proj.weight": "model-00088-of-00101.safetensors", + "model.layers.80.mlp.experts.62.up_proj.weight": "model-00088-of-00101.safetensors", + "model.layers.80.mlp.experts.62.down_proj.weight": "model-00088-of-00101.safetensors", + "model.layers.80.mlp.experts.63.gate_proj.weight": "model-00088-of-00101.safetensors", + "model.layers.80.mlp.experts.63.up_proj.weight": "model-00088-of-00101.safetensors", + "model.layers.80.mlp.experts.63.down_proj.weight": "model-00088-of-00101.safetensors", + "model.layers.80.mlp.experts.64.gate_proj.weight": "model-00088-of-00101.safetensors", + "model.layers.80.mlp.experts.64.up_proj.weight": "model-00088-of-00101.safetensors", + "model.layers.80.mlp.experts.64.down_proj.weight": "model-00088-of-00101.safetensors", + "model.layers.80.mlp.experts.65.gate_proj.weight": "model-00088-of-00101.safetensors", + "model.layers.80.mlp.experts.65.up_proj.weight": "model-00088-of-00101.safetensors", + "model.layers.80.mlp.experts.65.down_proj.weight": "model-00088-of-00101.safetensors", + "model.layers.80.mlp.experts.66.gate_proj.weight": "model-00088-of-00101.safetensors", + "model.layers.80.mlp.experts.66.up_proj.weight": "model-00088-of-00101.safetensors", + "model.layers.80.mlp.experts.66.down_proj.weight": "model-00088-of-00101.safetensors", + "model.layers.80.mlp.experts.67.gate_proj.weight": "model-00088-of-00101.safetensors", + "model.layers.80.mlp.experts.67.up_proj.weight": "model-00088-of-00101.safetensors", + "model.layers.80.mlp.experts.67.down_proj.weight": "model-00088-of-00101.safetensors", + "model.layers.80.mlp.experts.68.gate_proj.weight": "model-00088-of-00101.safetensors", + "model.layers.80.mlp.experts.68.up_proj.weight": "model-00088-of-00101.safetensors", + "model.layers.80.mlp.experts.68.down_proj.weight": "model-00088-of-00101.safetensors", + "model.layers.80.mlp.experts.69.gate_proj.weight": "model-00088-of-00101.safetensors", + "model.layers.80.mlp.experts.69.up_proj.weight": "model-00088-of-00101.safetensors", + "model.layers.80.mlp.experts.69.down_proj.weight": "model-00088-of-00101.safetensors", + "model.layers.80.mlp.experts.70.gate_proj.weight": "model-00088-of-00101.safetensors", + "model.layers.80.mlp.experts.70.up_proj.weight": "model-00088-of-00101.safetensors", + "model.layers.80.mlp.experts.70.down_proj.weight": "model-00088-of-00101.safetensors", + "model.layers.80.mlp.experts.71.gate_proj.weight": "model-00088-of-00101.safetensors", + "model.layers.80.mlp.experts.71.up_proj.weight": "model-00088-of-00101.safetensors", + "model.layers.80.mlp.experts.71.down_proj.weight": "model-00088-of-00101.safetensors", + "model.layers.80.mlp.experts.72.gate_proj.weight": "model-00088-of-00101.safetensors", + "model.layers.80.mlp.experts.72.up_proj.weight": "model-00088-of-00101.safetensors", + "model.layers.80.mlp.experts.72.down_proj.weight": "model-00088-of-00101.safetensors", + "model.layers.80.mlp.experts.73.gate_proj.weight": "model-00088-of-00101.safetensors", + "model.layers.80.mlp.experts.73.up_proj.weight": "model-00088-of-00101.safetensors", + "model.layers.80.mlp.experts.73.down_proj.weight": "model-00088-of-00101.safetensors", + "model.layers.80.mlp.experts.74.gate_proj.weight": "model-00088-of-00101.safetensors", + "model.layers.80.mlp.experts.74.up_proj.weight": "model-00088-of-00101.safetensors", + "model.layers.80.mlp.experts.74.down_proj.weight": "model-00088-of-00101.safetensors", + "model.layers.80.mlp.experts.75.gate_proj.weight": "model-00088-of-00101.safetensors", + "model.layers.80.mlp.experts.75.up_proj.weight": "model-00088-of-00101.safetensors", + "model.layers.80.mlp.experts.75.down_proj.weight": "model-00088-of-00101.safetensors", + "model.layers.80.mlp.experts.76.gate_proj.weight": "model-00088-of-00101.safetensors", + "model.layers.80.mlp.experts.76.up_proj.weight": "model-00088-of-00101.safetensors", + "model.layers.80.mlp.experts.76.down_proj.weight": "model-00088-of-00101.safetensors", + "model.layers.80.mlp.experts.77.gate_proj.weight": "model-00088-of-00101.safetensors", + "model.layers.80.mlp.experts.77.up_proj.weight": "model-00088-of-00101.safetensors", + "model.layers.80.mlp.experts.77.down_proj.weight": "model-00088-of-00101.safetensors", + "model.layers.80.mlp.experts.78.gate_proj.weight": "model-00088-of-00101.safetensors", + "model.layers.80.mlp.experts.78.up_proj.weight": "model-00088-of-00101.safetensors", + "model.layers.80.mlp.experts.78.down_proj.weight": "model-00088-of-00101.safetensors", + "model.layers.80.mlp.experts.79.gate_proj.weight": "model-00088-of-00101.safetensors", + "model.layers.80.mlp.experts.79.up_proj.weight": "model-00088-of-00101.safetensors", + "model.layers.80.mlp.experts.79.down_proj.weight": "model-00088-of-00101.safetensors", + "model.layers.80.mlp.experts.80.gate_proj.weight": "model-00088-of-00101.safetensors", + "model.layers.80.mlp.experts.80.up_proj.weight": "model-00088-of-00101.safetensors", + "model.layers.80.mlp.experts.80.down_proj.weight": "model-00088-of-00101.safetensors", + "model.layers.80.mlp.experts.81.gate_proj.weight": "model-00088-of-00101.safetensors", + "model.layers.80.mlp.experts.81.up_proj.weight": "model-00088-of-00101.safetensors", + "model.layers.80.mlp.experts.81.down_proj.weight": "model-00088-of-00101.safetensors", + "model.layers.80.mlp.experts.82.gate_proj.weight": "model-00088-of-00101.safetensors", + "model.layers.80.mlp.experts.82.up_proj.weight": "model-00088-of-00101.safetensors", + "model.layers.80.mlp.experts.82.down_proj.weight": "model-00088-of-00101.safetensors", + "model.layers.80.mlp.experts.83.gate_proj.weight": "model-00088-of-00101.safetensors", + "model.layers.80.mlp.experts.83.up_proj.weight": "model-00088-of-00101.safetensors", + "model.layers.80.mlp.experts.83.down_proj.weight": "model-00088-of-00101.safetensors", + "model.layers.80.mlp.experts.84.gate_proj.weight": "model-00088-of-00101.safetensors", + "model.layers.80.mlp.experts.84.up_proj.weight": "model-00088-of-00101.safetensors", + "model.layers.80.mlp.experts.84.down_proj.weight": "model-00088-of-00101.safetensors", + "model.layers.80.mlp.experts.85.gate_proj.weight": "model-00088-of-00101.safetensors", + "model.layers.80.mlp.experts.85.up_proj.weight": "model-00088-of-00101.safetensors", + "model.layers.80.mlp.experts.85.down_proj.weight": "model-00088-of-00101.safetensors", + "model.layers.80.mlp.experts.86.gate_proj.weight": "model-00088-of-00101.safetensors", + "model.layers.80.mlp.experts.86.up_proj.weight": "model-00088-of-00101.safetensors", + "model.layers.80.mlp.experts.86.down_proj.weight": "model-00088-of-00101.safetensors", + "model.layers.80.mlp.experts.87.gate_proj.weight": "model-00088-of-00101.safetensors", + "model.layers.80.mlp.experts.87.up_proj.weight": "model-00088-of-00101.safetensors", + "model.layers.80.mlp.experts.87.down_proj.weight": "model-00088-of-00101.safetensors", + "model.layers.80.mlp.experts.88.gate_proj.weight": "model-00088-of-00101.safetensors", + "model.layers.80.mlp.experts.88.up_proj.weight": "model-00088-of-00101.safetensors", + "model.layers.80.mlp.experts.88.down_proj.weight": "model-00088-of-00101.safetensors", + "model.layers.80.mlp.experts.89.gate_proj.weight": "model-00088-of-00101.safetensors", + "model.layers.80.mlp.experts.89.up_proj.weight": "model-00088-of-00101.safetensors", + "model.layers.80.mlp.experts.89.down_proj.weight": "model-00088-of-00101.safetensors", + "model.layers.80.mlp.experts.90.gate_proj.weight": "model-00088-of-00101.safetensors", + "model.layers.80.mlp.experts.90.up_proj.weight": "model-00088-of-00101.safetensors", + "model.layers.80.mlp.experts.90.down_proj.weight": "model-00088-of-00101.safetensors", + "model.layers.80.mlp.experts.91.gate_proj.weight": "model-00088-of-00101.safetensors", + "model.layers.80.mlp.experts.91.up_proj.weight": "model-00088-of-00101.safetensors", + "model.layers.80.mlp.experts.91.down_proj.weight": "model-00088-of-00101.safetensors", + "model.layers.80.mlp.experts.92.gate_proj.weight": "model-00088-of-00101.safetensors", + "model.layers.80.mlp.experts.92.up_proj.weight": "model-00088-of-00101.safetensors", + "model.layers.80.mlp.experts.92.down_proj.weight": "model-00088-of-00101.safetensors", + "model.layers.80.mlp.experts.93.gate_proj.weight": "model-00088-of-00101.safetensors", + "model.layers.80.mlp.experts.93.up_proj.weight": "model-00088-of-00101.safetensors", + "model.layers.80.mlp.experts.93.down_proj.weight": "model-00088-of-00101.safetensors", + "model.layers.80.mlp.experts.94.gate_proj.weight": "model-00088-of-00101.safetensors", + "model.layers.80.mlp.experts.94.up_proj.weight": "model-00088-of-00101.safetensors", + "model.layers.80.mlp.experts.94.down_proj.weight": "model-00088-of-00101.safetensors", + "model.layers.80.mlp.experts.95.gate_proj.weight": "model-00088-of-00101.safetensors", + "model.layers.80.mlp.experts.95.up_proj.weight": "model-00088-of-00101.safetensors", + "model.layers.80.mlp.experts.95.down_proj.weight": "model-00088-of-00101.safetensors", + "model.layers.80.mlp.experts.96.gate_proj.weight": "model-00088-of-00101.safetensors", + "model.layers.80.mlp.experts.96.up_proj.weight": "model-00088-of-00101.safetensors", + "model.layers.80.mlp.experts.96.down_proj.weight": "model-00088-of-00101.safetensors", + "model.layers.80.mlp.experts.97.gate_proj.weight": "model-00088-of-00101.safetensors", + "model.layers.80.mlp.experts.97.up_proj.weight": "model-00088-of-00101.safetensors", + "model.layers.80.mlp.experts.97.down_proj.weight": "model-00088-of-00101.safetensors", + "model.layers.80.mlp.experts.98.gate_proj.weight": "model-00088-of-00101.safetensors", + "model.layers.80.mlp.experts.98.up_proj.weight": "model-00088-of-00101.safetensors", + "model.layers.80.mlp.experts.98.down_proj.weight": "model-00088-of-00101.safetensors", + "model.layers.80.mlp.experts.99.gate_proj.weight": "model-00088-of-00101.safetensors", + "model.layers.80.mlp.experts.99.up_proj.weight": "model-00088-of-00101.safetensors", + "model.layers.80.mlp.experts.99.down_proj.weight": "model-00088-of-00101.safetensors", + "model.layers.80.mlp.experts.100.gate_proj.weight": "model-00088-of-00101.safetensors", + "model.layers.80.mlp.experts.100.up_proj.weight": "model-00088-of-00101.safetensors", + "model.layers.80.mlp.experts.100.down_proj.weight": "model-00088-of-00101.safetensors", + "model.layers.80.mlp.experts.101.gate_proj.weight": "model-00088-of-00101.safetensors", + "model.layers.80.mlp.experts.101.up_proj.weight": "model-00088-of-00101.safetensors", + "model.layers.80.mlp.experts.101.down_proj.weight": "model-00088-of-00101.safetensors", + "model.layers.80.mlp.experts.102.gate_proj.weight": "model-00088-of-00101.safetensors", + "model.layers.80.mlp.experts.102.up_proj.weight": "model-00088-of-00101.safetensors", + "model.layers.80.mlp.experts.102.down_proj.weight": "model-00088-of-00101.safetensors", + "model.layers.80.mlp.experts.103.gate_proj.weight": "model-00088-of-00101.safetensors", + "model.layers.80.mlp.experts.103.up_proj.weight": "model-00088-of-00101.safetensors", + "model.layers.80.mlp.experts.103.down_proj.weight": "model-00088-of-00101.safetensors", + "model.layers.80.mlp.experts.104.gate_proj.weight": "model-00088-of-00101.safetensors", + "model.layers.80.mlp.experts.104.up_proj.weight": "model-00088-of-00101.safetensors", + "model.layers.80.mlp.experts.104.down_proj.weight": "model-00088-of-00101.safetensors", + "model.layers.80.mlp.experts.105.gate_proj.weight": "model-00088-of-00101.safetensors", + "model.layers.80.mlp.experts.105.up_proj.weight": "model-00088-of-00101.safetensors", + "model.layers.80.mlp.experts.105.down_proj.weight": "model-00088-of-00101.safetensors", + "model.layers.80.mlp.experts.106.gate_proj.weight": "model-00088-of-00101.safetensors", + "model.layers.80.mlp.experts.106.up_proj.weight": "model-00088-of-00101.safetensors", + "model.layers.80.mlp.experts.106.down_proj.weight": "model-00088-of-00101.safetensors", + "model.layers.80.mlp.experts.107.gate_proj.weight": "model-00088-of-00101.safetensors", + "model.layers.80.mlp.experts.107.up_proj.weight": "model-00088-of-00101.safetensors", + "model.layers.80.mlp.experts.107.down_proj.weight": "model-00088-of-00101.safetensors", + "model.layers.80.mlp.experts.108.gate_proj.weight": "model-00088-of-00101.safetensors", + "model.layers.80.mlp.experts.108.up_proj.weight": "model-00088-of-00101.safetensors", + "model.layers.80.mlp.experts.108.down_proj.weight": "model-00088-of-00101.safetensors", + "model.layers.80.mlp.experts.109.gate_proj.weight": "model-00088-of-00101.safetensors", + "model.layers.80.mlp.experts.109.up_proj.weight": "model-00088-of-00101.safetensors", + "model.layers.80.mlp.experts.109.down_proj.weight": "model-00088-of-00101.safetensors", + "model.layers.80.mlp.experts.110.gate_proj.weight": "model-00088-of-00101.safetensors", + "model.layers.80.mlp.experts.110.up_proj.weight": "model-00088-of-00101.safetensors", + "model.layers.80.mlp.experts.110.down_proj.weight": "model-00088-of-00101.safetensors", + "model.layers.80.mlp.experts.111.gate_proj.weight": "model-00088-of-00101.safetensors", + "model.layers.80.mlp.experts.111.up_proj.weight": "model-00088-of-00101.safetensors", + "model.layers.80.mlp.experts.111.down_proj.weight": "model-00088-of-00101.safetensors", + "model.layers.80.mlp.experts.112.gate_proj.weight": "model-00088-of-00101.safetensors", + "model.layers.80.mlp.experts.112.up_proj.weight": "model-00088-of-00101.safetensors", + "model.layers.80.mlp.experts.112.down_proj.weight": "model-00088-of-00101.safetensors", + "model.layers.80.mlp.experts.113.gate_proj.weight": "model-00088-of-00101.safetensors", + "model.layers.80.mlp.experts.113.up_proj.weight": "model-00088-of-00101.safetensors", + "model.layers.80.mlp.experts.113.down_proj.weight": "model-00088-of-00101.safetensors", + "model.layers.80.mlp.experts.114.gate_proj.weight": "model-00088-of-00101.safetensors", + "model.layers.80.mlp.experts.114.up_proj.weight": "model-00088-of-00101.safetensors", + "model.layers.80.mlp.experts.114.down_proj.weight": "model-00088-of-00101.safetensors", + "model.layers.80.mlp.experts.115.gate_proj.weight": "model-00088-of-00101.safetensors", + "model.layers.80.mlp.experts.115.up_proj.weight": "model-00088-of-00101.safetensors", + "model.layers.80.mlp.experts.115.down_proj.weight": "model-00088-of-00101.safetensors", + "model.layers.80.mlp.experts.116.gate_proj.weight": "model-00088-of-00101.safetensors", + "model.layers.80.mlp.experts.116.up_proj.weight": "model-00088-of-00101.safetensors", + "model.layers.80.mlp.experts.116.down_proj.weight": "model-00088-of-00101.safetensors", + "model.layers.80.mlp.experts.117.gate_proj.weight": "model-00088-of-00101.safetensors", + "model.layers.80.mlp.experts.117.up_proj.weight": "model-00088-of-00101.safetensors", + "model.layers.80.mlp.experts.117.down_proj.weight": "model-00088-of-00101.safetensors", + "model.layers.80.mlp.experts.118.gate_proj.weight": "model-00088-of-00101.safetensors", + "model.layers.80.mlp.experts.118.up_proj.weight": "model-00088-of-00101.safetensors", + "model.layers.80.mlp.experts.118.down_proj.weight": "model-00088-of-00101.safetensors", + "model.layers.80.mlp.experts.119.gate_proj.weight": "model-00088-of-00101.safetensors", + "model.layers.80.mlp.experts.119.up_proj.weight": "model-00088-of-00101.safetensors", + "model.layers.80.mlp.experts.119.down_proj.weight": "model-00088-of-00101.safetensors", + "model.layers.80.mlp.gate.weight": "model-00088-of-00101.safetensors", + "model.layers.80.mlp.gate.e_score_correction_bias": "model-00088-of-00101.safetensors", + "model.layers.80.mlp.shared_experts.gate_proj.weight": "model-00088-of-00101.safetensors", + "model.layers.80.mlp.shared_experts.up_proj.weight": "model-00088-of-00101.safetensors", + "model.layers.80.mlp.shared_experts.down_proj.weight": "model-00088-of-00101.safetensors", + "model.layers.80.input_layernorm.weight": "model-00088-of-00101.safetensors", + "model.layers.80.post_attention_layernorm.weight": "model-00088-of-00101.safetensors", + "model.layers.81.self_attn.q_proj.weight": "model-00088-of-00101.safetensors", + "model.layers.81.self_attn.q_proj.bias": "model-00088-of-00101.safetensors", + "model.layers.81.self_attn.k_proj.weight": "model-00088-of-00101.safetensors", + "model.layers.81.self_attn.k_proj.bias": "model-00088-of-00101.safetensors", + "model.layers.81.self_attn.v_proj.weight": "model-00088-of-00101.safetensors", + "model.layers.81.self_attn.v_proj.bias": "model-00088-of-00101.safetensors", + "model.layers.81.self_attn.o_proj.weight": "model-00088-of-00101.safetensors", + "model.layers.81.self_attn.q_norm.weight": "model-00088-of-00101.safetensors", + "model.layers.81.self_attn.k_norm.weight": "model-00088-of-00101.safetensors", + "model.layers.81.mlp.experts.0.gate_proj.weight": "model-00088-of-00101.safetensors", + "model.layers.81.mlp.experts.0.up_proj.weight": "model-00088-of-00101.safetensors", + "model.layers.81.mlp.experts.0.down_proj.weight": "model-00088-of-00101.safetensors", + "model.layers.81.mlp.experts.1.gate_proj.weight": "model-00088-of-00101.safetensors", + "model.layers.81.mlp.experts.1.up_proj.weight": "model-00088-of-00101.safetensors", + "model.layers.81.mlp.experts.1.down_proj.weight": "model-00088-of-00101.safetensors", + "model.layers.81.mlp.experts.2.gate_proj.weight": "model-00088-of-00101.safetensors", + "model.layers.81.mlp.experts.2.up_proj.weight": "model-00088-of-00101.safetensors", + "model.layers.81.mlp.experts.2.down_proj.weight": "model-00088-of-00101.safetensors", + "model.layers.81.mlp.experts.3.gate_proj.weight": "model-00088-of-00101.safetensors", + "model.layers.81.mlp.experts.3.up_proj.weight": "model-00088-of-00101.safetensors", + "model.layers.81.mlp.experts.3.down_proj.weight": "model-00088-of-00101.safetensors", + "model.layers.81.mlp.experts.4.gate_proj.weight": "model-00088-of-00101.safetensors", + "model.layers.81.mlp.experts.4.up_proj.weight": "model-00088-of-00101.safetensors", + "model.layers.81.mlp.experts.4.down_proj.weight": "model-00088-of-00101.safetensors", + "model.layers.81.mlp.experts.5.gate_proj.weight": "model-00088-of-00101.safetensors", + "model.layers.81.mlp.experts.5.up_proj.weight": "model-00088-of-00101.safetensors", + "model.layers.81.mlp.experts.5.down_proj.weight": "model-00088-of-00101.safetensors", + "model.layers.81.mlp.experts.6.gate_proj.weight": "model-00088-of-00101.safetensors", + "model.layers.81.mlp.experts.6.up_proj.weight": "model-00088-of-00101.safetensors", + "model.layers.81.mlp.experts.6.down_proj.weight": "model-00088-of-00101.safetensors", + "model.layers.81.mlp.experts.7.gate_proj.weight": "model-00088-of-00101.safetensors", + "model.layers.81.mlp.experts.7.up_proj.weight": "model-00088-of-00101.safetensors", + "model.layers.81.mlp.experts.7.down_proj.weight": "model-00088-of-00101.safetensors", + "model.layers.81.mlp.experts.8.gate_proj.weight": "model-00088-of-00101.safetensors", + "model.layers.81.mlp.experts.8.up_proj.weight": "model-00088-of-00101.safetensors", + "model.layers.81.mlp.experts.8.down_proj.weight": "model-00088-of-00101.safetensors", + "model.layers.81.mlp.experts.9.gate_proj.weight": "model-00088-of-00101.safetensors", + "model.layers.81.mlp.experts.9.up_proj.weight": "model-00088-of-00101.safetensors", + "model.layers.81.mlp.experts.9.down_proj.weight": "model-00088-of-00101.safetensors", + "model.layers.81.mlp.experts.10.gate_proj.weight": "model-00088-of-00101.safetensors", + "model.layers.81.mlp.experts.10.up_proj.weight": "model-00088-of-00101.safetensors", + "model.layers.81.mlp.experts.10.down_proj.weight": "model-00088-of-00101.safetensors", + "model.layers.81.mlp.experts.11.gate_proj.weight": "model-00088-of-00101.safetensors", + "model.layers.81.mlp.experts.11.up_proj.weight": "model-00088-of-00101.safetensors", + "model.layers.81.mlp.experts.11.down_proj.weight": "model-00088-of-00101.safetensors", + "model.layers.81.mlp.experts.12.gate_proj.weight": "model-00088-of-00101.safetensors", + "model.layers.81.mlp.experts.12.up_proj.weight": "model-00088-of-00101.safetensors", + "model.layers.81.mlp.experts.12.down_proj.weight": "model-00088-of-00101.safetensors", + "model.layers.81.mlp.experts.13.gate_proj.weight": "model-00088-of-00101.safetensors", + "model.layers.81.mlp.experts.13.up_proj.weight": "model-00088-of-00101.safetensors", + "model.layers.81.mlp.experts.13.down_proj.weight": "model-00088-of-00101.safetensors", + "model.layers.81.mlp.experts.14.gate_proj.weight": "model-00088-of-00101.safetensors", + "model.layers.81.mlp.experts.14.up_proj.weight": "model-00089-of-00101.safetensors", + "model.layers.81.mlp.experts.14.down_proj.weight": "model-00089-of-00101.safetensors", + "model.layers.81.mlp.experts.15.gate_proj.weight": "model-00089-of-00101.safetensors", + "model.layers.81.mlp.experts.15.up_proj.weight": "model-00089-of-00101.safetensors", + "model.layers.81.mlp.experts.15.down_proj.weight": "model-00089-of-00101.safetensors", + "model.layers.81.mlp.experts.16.gate_proj.weight": "model-00089-of-00101.safetensors", + "model.layers.81.mlp.experts.16.up_proj.weight": "model-00089-of-00101.safetensors", + "model.layers.81.mlp.experts.16.down_proj.weight": "model-00089-of-00101.safetensors", + "model.layers.81.mlp.experts.17.gate_proj.weight": "model-00089-of-00101.safetensors", + "model.layers.81.mlp.experts.17.up_proj.weight": "model-00089-of-00101.safetensors", + "model.layers.81.mlp.experts.17.down_proj.weight": "model-00089-of-00101.safetensors", + "model.layers.81.mlp.experts.18.gate_proj.weight": "model-00089-of-00101.safetensors", + "model.layers.81.mlp.experts.18.up_proj.weight": "model-00089-of-00101.safetensors", + "model.layers.81.mlp.experts.18.down_proj.weight": "model-00089-of-00101.safetensors", + "model.layers.81.mlp.experts.19.gate_proj.weight": "model-00089-of-00101.safetensors", + "model.layers.81.mlp.experts.19.up_proj.weight": "model-00089-of-00101.safetensors", + "model.layers.81.mlp.experts.19.down_proj.weight": "model-00089-of-00101.safetensors", + "model.layers.81.mlp.experts.20.gate_proj.weight": "model-00089-of-00101.safetensors", + "model.layers.81.mlp.experts.20.up_proj.weight": "model-00089-of-00101.safetensors", + "model.layers.81.mlp.experts.20.down_proj.weight": "model-00089-of-00101.safetensors", + "model.layers.81.mlp.experts.21.gate_proj.weight": "model-00089-of-00101.safetensors", + "model.layers.81.mlp.experts.21.up_proj.weight": "model-00089-of-00101.safetensors", + "model.layers.81.mlp.experts.21.down_proj.weight": "model-00089-of-00101.safetensors", + "model.layers.81.mlp.experts.22.gate_proj.weight": "model-00089-of-00101.safetensors", + "model.layers.81.mlp.experts.22.up_proj.weight": "model-00089-of-00101.safetensors", + "model.layers.81.mlp.experts.22.down_proj.weight": "model-00089-of-00101.safetensors", + "model.layers.81.mlp.experts.23.gate_proj.weight": "model-00089-of-00101.safetensors", + "model.layers.81.mlp.experts.23.up_proj.weight": "model-00089-of-00101.safetensors", + "model.layers.81.mlp.experts.23.down_proj.weight": "model-00089-of-00101.safetensors", + "model.layers.81.mlp.experts.24.gate_proj.weight": "model-00089-of-00101.safetensors", + "model.layers.81.mlp.experts.24.up_proj.weight": "model-00089-of-00101.safetensors", + "model.layers.81.mlp.experts.24.down_proj.weight": "model-00089-of-00101.safetensors", + "model.layers.81.mlp.experts.25.gate_proj.weight": "model-00089-of-00101.safetensors", + "model.layers.81.mlp.experts.25.up_proj.weight": "model-00089-of-00101.safetensors", + "model.layers.81.mlp.experts.25.down_proj.weight": "model-00089-of-00101.safetensors", + "model.layers.81.mlp.experts.26.gate_proj.weight": "model-00089-of-00101.safetensors", + "model.layers.81.mlp.experts.26.up_proj.weight": "model-00089-of-00101.safetensors", + "model.layers.81.mlp.experts.26.down_proj.weight": "model-00089-of-00101.safetensors", + "model.layers.81.mlp.experts.27.gate_proj.weight": "model-00089-of-00101.safetensors", + "model.layers.81.mlp.experts.27.up_proj.weight": "model-00089-of-00101.safetensors", + "model.layers.81.mlp.experts.27.down_proj.weight": "model-00089-of-00101.safetensors", + "model.layers.81.mlp.experts.28.gate_proj.weight": "model-00089-of-00101.safetensors", + "model.layers.81.mlp.experts.28.up_proj.weight": "model-00089-of-00101.safetensors", + "model.layers.81.mlp.experts.28.down_proj.weight": "model-00089-of-00101.safetensors", + "model.layers.81.mlp.experts.29.gate_proj.weight": "model-00089-of-00101.safetensors", + "model.layers.81.mlp.experts.29.up_proj.weight": "model-00089-of-00101.safetensors", + "model.layers.81.mlp.experts.29.down_proj.weight": "model-00089-of-00101.safetensors", + "model.layers.81.mlp.experts.30.gate_proj.weight": "model-00089-of-00101.safetensors", + "model.layers.81.mlp.experts.30.up_proj.weight": "model-00089-of-00101.safetensors", + "model.layers.81.mlp.experts.30.down_proj.weight": "model-00089-of-00101.safetensors", + "model.layers.81.mlp.experts.31.gate_proj.weight": "model-00089-of-00101.safetensors", + "model.layers.81.mlp.experts.31.up_proj.weight": "model-00089-of-00101.safetensors", + "model.layers.81.mlp.experts.31.down_proj.weight": "model-00089-of-00101.safetensors", + "model.layers.81.mlp.experts.32.gate_proj.weight": "model-00089-of-00101.safetensors", + "model.layers.81.mlp.experts.32.up_proj.weight": "model-00089-of-00101.safetensors", + "model.layers.81.mlp.experts.32.down_proj.weight": "model-00089-of-00101.safetensors", + "model.layers.81.mlp.experts.33.gate_proj.weight": "model-00089-of-00101.safetensors", + "model.layers.81.mlp.experts.33.up_proj.weight": "model-00089-of-00101.safetensors", + "model.layers.81.mlp.experts.33.down_proj.weight": "model-00089-of-00101.safetensors", + "model.layers.81.mlp.experts.34.gate_proj.weight": "model-00089-of-00101.safetensors", + "model.layers.81.mlp.experts.34.up_proj.weight": "model-00089-of-00101.safetensors", + "model.layers.81.mlp.experts.34.down_proj.weight": "model-00089-of-00101.safetensors", + "model.layers.81.mlp.experts.35.gate_proj.weight": "model-00089-of-00101.safetensors", + "model.layers.81.mlp.experts.35.up_proj.weight": "model-00089-of-00101.safetensors", + "model.layers.81.mlp.experts.35.down_proj.weight": "model-00089-of-00101.safetensors", + "model.layers.81.mlp.experts.36.gate_proj.weight": "model-00089-of-00101.safetensors", + "model.layers.81.mlp.experts.36.up_proj.weight": "model-00089-of-00101.safetensors", + "model.layers.81.mlp.experts.36.down_proj.weight": "model-00089-of-00101.safetensors", + "model.layers.81.mlp.experts.37.gate_proj.weight": "model-00089-of-00101.safetensors", + "model.layers.81.mlp.experts.37.up_proj.weight": "model-00089-of-00101.safetensors", + "model.layers.81.mlp.experts.37.down_proj.weight": "model-00089-of-00101.safetensors", + "model.layers.81.mlp.experts.38.gate_proj.weight": "model-00089-of-00101.safetensors", + "model.layers.81.mlp.experts.38.up_proj.weight": "model-00089-of-00101.safetensors", + "model.layers.81.mlp.experts.38.down_proj.weight": "model-00089-of-00101.safetensors", + "model.layers.81.mlp.experts.39.gate_proj.weight": "model-00089-of-00101.safetensors", + "model.layers.81.mlp.experts.39.up_proj.weight": "model-00089-of-00101.safetensors", + "model.layers.81.mlp.experts.39.down_proj.weight": "model-00089-of-00101.safetensors", + "model.layers.81.mlp.experts.40.gate_proj.weight": "model-00089-of-00101.safetensors", + "model.layers.81.mlp.experts.40.up_proj.weight": "model-00089-of-00101.safetensors", + "model.layers.81.mlp.experts.40.down_proj.weight": "model-00089-of-00101.safetensors", + "model.layers.81.mlp.experts.41.gate_proj.weight": "model-00089-of-00101.safetensors", + "model.layers.81.mlp.experts.41.up_proj.weight": "model-00089-of-00101.safetensors", + "model.layers.81.mlp.experts.41.down_proj.weight": "model-00089-of-00101.safetensors", + "model.layers.81.mlp.experts.42.gate_proj.weight": "model-00089-of-00101.safetensors", + "model.layers.81.mlp.experts.42.up_proj.weight": "model-00089-of-00101.safetensors", + "model.layers.81.mlp.experts.42.down_proj.weight": "model-00089-of-00101.safetensors", + "model.layers.81.mlp.experts.43.gate_proj.weight": "model-00089-of-00101.safetensors", + "model.layers.81.mlp.experts.43.up_proj.weight": "model-00089-of-00101.safetensors", + "model.layers.81.mlp.experts.43.down_proj.weight": "model-00089-of-00101.safetensors", + "model.layers.81.mlp.experts.44.gate_proj.weight": "model-00089-of-00101.safetensors", + "model.layers.81.mlp.experts.44.up_proj.weight": "model-00089-of-00101.safetensors", + "model.layers.81.mlp.experts.44.down_proj.weight": "model-00089-of-00101.safetensors", + "model.layers.81.mlp.experts.45.gate_proj.weight": "model-00089-of-00101.safetensors", + "model.layers.81.mlp.experts.45.up_proj.weight": "model-00089-of-00101.safetensors", + "model.layers.81.mlp.experts.45.down_proj.weight": "model-00089-of-00101.safetensors", + "model.layers.81.mlp.experts.46.gate_proj.weight": "model-00089-of-00101.safetensors", + "model.layers.81.mlp.experts.46.up_proj.weight": "model-00089-of-00101.safetensors", + "model.layers.81.mlp.experts.46.down_proj.weight": "model-00089-of-00101.safetensors", + "model.layers.81.mlp.experts.47.gate_proj.weight": "model-00089-of-00101.safetensors", + "model.layers.81.mlp.experts.47.up_proj.weight": "model-00089-of-00101.safetensors", + "model.layers.81.mlp.experts.47.down_proj.weight": "model-00089-of-00101.safetensors", + "model.layers.81.mlp.experts.48.gate_proj.weight": "model-00089-of-00101.safetensors", + "model.layers.81.mlp.experts.48.up_proj.weight": "model-00089-of-00101.safetensors", + "model.layers.81.mlp.experts.48.down_proj.weight": "model-00089-of-00101.safetensors", + "model.layers.81.mlp.experts.49.gate_proj.weight": "model-00089-of-00101.safetensors", + "model.layers.81.mlp.experts.49.up_proj.weight": "model-00089-of-00101.safetensors", + "model.layers.81.mlp.experts.49.down_proj.weight": "model-00089-of-00101.safetensors", + "model.layers.81.mlp.experts.50.gate_proj.weight": "model-00089-of-00101.safetensors", + "model.layers.81.mlp.experts.50.up_proj.weight": "model-00089-of-00101.safetensors", + "model.layers.81.mlp.experts.50.down_proj.weight": "model-00089-of-00101.safetensors", + "model.layers.81.mlp.experts.51.gate_proj.weight": "model-00089-of-00101.safetensors", + "model.layers.81.mlp.experts.51.up_proj.weight": "model-00089-of-00101.safetensors", + "model.layers.81.mlp.experts.51.down_proj.weight": "model-00089-of-00101.safetensors", + "model.layers.81.mlp.experts.52.gate_proj.weight": "model-00089-of-00101.safetensors", + "model.layers.81.mlp.experts.52.up_proj.weight": "model-00089-of-00101.safetensors", + "model.layers.81.mlp.experts.52.down_proj.weight": "model-00089-of-00101.safetensors", + "model.layers.81.mlp.experts.53.gate_proj.weight": "model-00089-of-00101.safetensors", + "model.layers.81.mlp.experts.53.up_proj.weight": "model-00089-of-00101.safetensors", + "model.layers.81.mlp.experts.53.down_proj.weight": "model-00089-of-00101.safetensors", + "model.layers.81.mlp.experts.54.gate_proj.weight": "model-00089-of-00101.safetensors", + "model.layers.81.mlp.experts.54.up_proj.weight": "model-00089-of-00101.safetensors", + "model.layers.81.mlp.experts.54.down_proj.weight": "model-00089-of-00101.safetensors", + "model.layers.81.mlp.experts.55.gate_proj.weight": "model-00089-of-00101.safetensors", + "model.layers.81.mlp.experts.55.up_proj.weight": "model-00089-of-00101.safetensors", + "model.layers.81.mlp.experts.55.down_proj.weight": "model-00089-of-00101.safetensors", + "model.layers.81.mlp.experts.56.gate_proj.weight": "model-00089-of-00101.safetensors", + "model.layers.81.mlp.experts.56.up_proj.weight": "model-00089-of-00101.safetensors", + "model.layers.81.mlp.experts.56.down_proj.weight": "model-00089-of-00101.safetensors", + "model.layers.81.mlp.experts.57.gate_proj.weight": "model-00089-of-00101.safetensors", + "model.layers.81.mlp.experts.57.up_proj.weight": "model-00089-of-00101.safetensors", + "model.layers.81.mlp.experts.57.down_proj.weight": "model-00089-of-00101.safetensors", + "model.layers.81.mlp.experts.58.gate_proj.weight": "model-00089-of-00101.safetensors", + "model.layers.81.mlp.experts.58.up_proj.weight": "model-00089-of-00101.safetensors", + "model.layers.81.mlp.experts.58.down_proj.weight": "model-00089-of-00101.safetensors", + "model.layers.81.mlp.experts.59.gate_proj.weight": "model-00089-of-00101.safetensors", + "model.layers.81.mlp.experts.59.up_proj.weight": "model-00089-of-00101.safetensors", + "model.layers.81.mlp.experts.59.down_proj.weight": "model-00089-of-00101.safetensors", + "model.layers.81.mlp.experts.60.gate_proj.weight": "model-00089-of-00101.safetensors", + "model.layers.81.mlp.experts.60.up_proj.weight": "model-00089-of-00101.safetensors", + "model.layers.81.mlp.experts.60.down_proj.weight": "model-00089-of-00101.safetensors", + "model.layers.81.mlp.experts.61.gate_proj.weight": "model-00089-of-00101.safetensors", + "model.layers.81.mlp.experts.61.up_proj.weight": "model-00089-of-00101.safetensors", + "model.layers.81.mlp.experts.61.down_proj.weight": "model-00089-of-00101.safetensors", + "model.layers.81.mlp.experts.62.gate_proj.weight": "model-00089-of-00101.safetensors", + "model.layers.81.mlp.experts.62.up_proj.weight": "model-00089-of-00101.safetensors", + "model.layers.81.mlp.experts.62.down_proj.weight": "model-00089-of-00101.safetensors", + "model.layers.81.mlp.experts.63.gate_proj.weight": "model-00089-of-00101.safetensors", + "model.layers.81.mlp.experts.63.up_proj.weight": "model-00089-of-00101.safetensors", + "model.layers.81.mlp.experts.63.down_proj.weight": "model-00089-of-00101.safetensors", + "model.layers.81.mlp.experts.64.gate_proj.weight": "model-00089-of-00101.safetensors", + "model.layers.81.mlp.experts.64.up_proj.weight": "model-00089-of-00101.safetensors", + "model.layers.81.mlp.experts.64.down_proj.weight": "model-00089-of-00101.safetensors", + "model.layers.81.mlp.experts.65.gate_proj.weight": "model-00089-of-00101.safetensors", + "model.layers.81.mlp.experts.65.up_proj.weight": "model-00089-of-00101.safetensors", + "model.layers.81.mlp.experts.65.down_proj.weight": "model-00089-of-00101.safetensors", + "model.layers.81.mlp.experts.66.gate_proj.weight": "model-00089-of-00101.safetensors", + "model.layers.81.mlp.experts.66.up_proj.weight": "model-00089-of-00101.safetensors", + "model.layers.81.mlp.experts.66.down_proj.weight": "model-00089-of-00101.safetensors", + "model.layers.81.mlp.experts.67.gate_proj.weight": "model-00089-of-00101.safetensors", + "model.layers.81.mlp.experts.67.up_proj.weight": "model-00089-of-00101.safetensors", + "model.layers.81.mlp.experts.67.down_proj.weight": "model-00089-of-00101.safetensors", + "model.layers.81.mlp.experts.68.gate_proj.weight": "model-00089-of-00101.safetensors", + "model.layers.81.mlp.experts.68.up_proj.weight": "model-00089-of-00101.safetensors", + "model.layers.81.mlp.experts.68.down_proj.weight": "model-00089-of-00101.safetensors", + "model.layers.81.mlp.experts.69.gate_proj.weight": "model-00089-of-00101.safetensors", + "model.layers.81.mlp.experts.69.up_proj.weight": "model-00089-of-00101.safetensors", + "model.layers.81.mlp.experts.69.down_proj.weight": "model-00089-of-00101.safetensors", + "model.layers.81.mlp.experts.70.gate_proj.weight": "model-00089-of-00101.safetensors", + "model.layers.81.mlp.experts.70.up_proj.weight": "model-00089-of-00101.safetensors", + "model.layers.81.mlp.experts.70.down_proj.weight": "model-00089-of-00101.safetensors", + "model.layers.81.mlp.experts.71.gate_proj.weight": "model-00089-of-00101.safetensors", + "model.layers.81.mlp.experts.71.up_proj.weight": "model-00089-of-00101.safetensors", + "model.layers.81.mlp.experts.71.down_proj.weight": "model-00089-of-00101.safetensors", + "model.layers.81.mlp.experts.72.gate_proj.weight": "model-00089-of-00101.safetensors", + "model.layers.81.mlp.experts.72.up_proj.weight": "model-00089-of-00101.safetensors", + "model.layers.81.mlp.experts.72.down_proj.weight": "model-00089-of-00101.safetensors", + "model.layers.81.mlp.experts.73.gate_proj.weight": "model-00089-of-00101.safetensors", + "model.layers.81.mlp.experts.73.up_proj.weight": "model-00089-of-00101.safetensors", + "model.layers.81.mlp.experts.73.down_proj.weight": "model-00089-of-00101.safetensors", + "model.layers.81.mlp.experts.74.gate_proj.weight": "model-00089-of-00101.safetensors", + "model.layers.81.mlp.experts.74.up_proj.weight": "model-00089-of-00101.safetensors", + "model.layers.81.mlp.experts.74.down_proj.weight": "model-00089-of-00101.safetensors", + "model.layers.81.mlp.experts.75.gate_proj.weight": "model-00089-of-00101.safetensors", + "model.layers.81.mlp.experts.75.up_proj.weight": "model-00089-of-00101.safetensors", + "model.layers.81.mlp.experts.75.down_proj.weight": "model-00089-of-00101.safetensors", + "model.layers.81.mlp.experts.76.gate_proj.weight": "model-00089-of-00101.safetensors", + "model.layers.81.mlp.experts.76.up_proj.weight": "model-00089-of-00101.safetensors", + "model.layers.81.mlp.experts.76.down_proj.weight": "model-00089-of-00101.safetensors", + "model.layers.81.mlp.experts.77.gate_proj.weight": "model-00089-of-00101.safetensors", + "model.layers.81.mlp.experts.77.up_proj.weight": "model-00089-of-00101.safetensors", + "model.layers.81.mlp.experts.77.down_proj.weight": "model-00089-of-00101.safetensors", + "model.layers.81.mlp.experts.78.gate_proj.weight": "model-00089-of-00101.safetensors", + "model.layers.81.mlp.experts.78.up_proj.weight": "model-00089-of-00101.safetensors", + "model.layers.81.mlp.experts.78.down_proj.weight": "model-00089-of-00101.safetensors", + "model.layers.81.mlp.experts.79.gate_proj.weight": "model-00089-of-00101.safetensors", + "model.layers.81.mlp.experts.79.up_proj.weight": "model-00089-of-00101.safetensors", + "model.layers.81.mlp.experts.79.down_proj.weight": "model-00089-of-00101.safetensors", + "model.layers.81.mlp.experts.80.gate_proj.weight": "model-00089-of-00101.safetensors", + "model.layers.81.mlp.experts.80.up_proj.weight": "model-00089-of-00101.safetensors", + "model.layers.81.mlp.experts.80.down_proj.weight": "model-00089-of-00101.safetensors", + "model.layers.81.mlp.experts.81.gate_proj.weight": "model-00089-of-00101.safetensors", + "model.layers.81.mlp.experts.81.up_proj.weight": "model-00089-of-00101.safetensors", + "model.layers.81.mlp.experts.81.down_proj.weight": "model-00089-of-00101.safetensors", + "model.layers.81.mlp.experts.82.gate_proj.weight": "model-00089-of-00101.safetensors", + "model.layers.81.mlp.experts.82.up_proj.weight": "model-00089-of-00101.safetensors", + "model.layers.81.mlp.experts.82.down_proj.weight": "model-00089-of-00101.safetensors", + "model.layers.81.mlp.experts.83.gate_proj.weight": "model-00089-of-00101.safetensors", + "model.layers.81.mlp.experts.83.up_proj.weight": "model-00089-of-00101.safetensors", + "model.layers.81.mlp.experts.83.down_proj.weight": "model-00089-of-00101.safetensors", + "model.layers.81.mlp.experts.84.gate_proj.weight": "model-00089-of-00101.safetensors", + "model.layers.81.mlp.experts.84.up_proj.weight": "model-00089-of-00101.safetensors", + "model.layers.81.mlp.experts.84.down_proj.weight": "model-00089-of-00101.safetensors", + "model.layers.81.mlp.experts.85.gate_proj.weight": "model-00089-of-00101.safetensors", + "model.layers.81.mlp.experts.85.up_proj.weight": "model-00089-of-00101.safetensors", + "model.layers.81.mlp.experts.85.down_proj.weight": "model-00089-of-00101.safetensors", + "model.layers.81.mlp.experts.86.gate_proj.weight": "model-00089-of-00101.safetensors", + "model.layers.81.mlp.experts.86.up_proj.weight": "model-00089-of-00101.safetensors", + "model.layers.81.mlp.experts.86.down_proj.weight": "model-00089-of-00101.safetensors", + "model.layers.81.mlp.experts.87.gate_proj.weight": "model-00089-of-00101.safetensors", + "model.layers.81.mlp.experts.87.up_proj.weight": "model-00089-of-00101.safetensors", + "model.layers.81.mlp.experts.87.down_proj.weight": "model-00089-of-00101.safetensors", + "model.layers.81.mlp.experts.88.gate_proj.weight": "model-00089-of-00101.safetensors", + "model.layers.81.mlp.experts.88.up_proj.weight": "model-00089-of-00101.safetensors", + "model.layers.81.mlp.experts.88.down_proj.weight": "model-00089-of-00101.safetensors", + "model.layers.81.mlp.experts.89.gate_proj.weight": "model-00089-of-00101.safetensors", + "model.layers.81.mlp.experts.89.up_proj.weight": "model-00089-of-00101.safetensors", + "model.layers.81.mlp.experts.89.down_proj.weight": "model-00089-of-00101.safetensors", + "model.layers.81.mlp.experts.90.gate_proj.weight": "model-00089-of-00101.safetensors", + "model.layers.81.mlp.experts.90.up_proj.weight": "model-00089-of-00101.safetensors", + "model.layers.81.mlp.experts.90.down_proj.weight": "model-00089-of-00101.safetensors", + "model.layers.81.mlp.experts.91.gate_proj.weight": "model-00089-of-00101.safetensors", + "model.layers.81.mlp.experts.91.up_proj.weight": "model-00089-of-00101.safetensors", + "model.layers.81.mlp.experts.91.down_proj.weight": "model-00089-of-00101.safetensors", + "model.layers.81.mlp.experts.92.gate_proj.weight": "model-00089-of-00101.safetensors", + "model.layers.81.mlp.experts.92.up_proj.weight": "model-00089-of-00101.safetensors", + "model.layers.81.mlp.experts.92.down_proj.weight": "model-00089-of-00101.safetensors", + "model.layers.81.mlp.experts.93.gate_proj.weight": "model-00089-of-00101.safetensors", + "model.layers.81.mlp.experts.93.up_proj.weight": "model-00089-of-00101.safetensors", + "model.layers.81.mlp.experts.93.down_proj.weight": "model-00089-of-00101.safetensors", + "model.layers.81.mlp.experts.94.gate_proj.weight": "model-00089-of-00101.safetensors", + "model.layers.81.mlp.experts.94.up_proj.weight": "model-00089-of-00101.safetensors", + "model.layers.81.mlp.experts.94.down_proj.weight": "model-00089-of-00101.safetensors", + "model.layers.81.mlp.experts.95.gate_proj.weight": "model-00089-of-00101.safetensors", + "model.layers.81.mlp.experts.95.up_proj.weight": "model-00089-of-00101.safetensors", + "model.layers.81.mlp.experts.95.down_proj.weight": "model-00089-of-00101.safetensors", + "model.layers.81.mlp.experts.96.gate_proj.weight": "model-00089-of-00101.safetensors", + "model.layers.81.mlp.experts.96.up_proj.weight": "model-00089-of-00101.safetensors", + "model.layers.81.mlp.experts.96.down_proj.weight": "model-00089-of-00101.safetensors", + "model.layers.81.mlp.experts.97.gate_proj.weight": "model-00089-of-00101.safetensors", + "model.layers.81.mlp.experts.97.up_proj.weight": "model-00089-of-00101.safetensors", + "model.layers.81.mlp.experts.97.down_proj.weight": "model-00089-of-00101.safetensors", + "model.layers.81.mlp.experts.98.gate_proj.weight": "model-00089-of-00101.safetensors", + "model.layers.81.mlp.experts.98.up_proj.weight": "model-00089-of-00101.safetensors", + "model.layers.81.mlp.experts.98.down_proj.weight": "model-00089-of-00101.safetensors", + "model.layers.81.mlp.experts.99.gate_proj.weight": "model-00089-of-00101.safetensors", + "model.layers.81.mlp.experts.99.up_proj.weight": "model-00089-of-00101.safetensors", + "model.layers.81.mlp.experts.99.down_proj.weight": "model-00089-of-00101.safetensors", + "model.layers.81.mlp.experts.100.gate_proj.weight": "model-00089-of-00101.safetensors", + "model.layers.81.mlp.experts.100.up_proj.weight": "model-00089-of-00101.safetensors", + "model.layers.81.mlp.experts.100.down_proj.weight": "model-00089-of-00101.safetensors", + "model.layers.81.mlp.experts.101.gate_proj.weight": "model-00089-of-00101.safetensors", + "model.layers.81.mlp.experts.101.up_proj.weight": "model-00089-of-00101.safetensors", + "model.layers.81.mlp.experts.101.down_proj.weight": "model-00089-of-00101.safetensors", + "model.layers.81.mlp.experts.102.gate_proj.weight": "model-00089-of-00101.safetensors", + "model.layers.81.mlp.experts.102.up_proj.weight": "model-00089-of-00101.safetensors", + "model.layers.81.mlp.experts.102.down_proj.weight": "model-00089-of-00101.safetensors", + "model.layers.81.mlp.experts.103.gate_proj.weight": "model-00089-of-00101.safetensors", + "model.layers.81.mlp.experts.103.up_proj.weight": "model-00089-of-00101.safetensors", + "model.layers.81.mlp.experts.103.down_proj.weight": "model-00089-of-00101.safetensors", + "model.layers.81.mlp.experts.104.gate_proj.weight": "model-00089-of-00101.safetensors", + "model.layers.81.mlp.experts.104.up_proj.weight": "model-00089-of-00101.safetensors", + "model.layers.81.mlp.experts.104.down_proj.weight": "model-00089-of-00101.safetensors", + "model.layers.81.mlp.experts.105.gate_proj.weight": "model-00089-of-00101.safetensors", + "model.layers.81.mlp.experts.105.up_proj.weight": "model-00089-of-00101.safetensors", + "model.layers.81.mlp.experts.105.down_proj.weight": "model-00089-of-00101.safetensors", + "model.layers.81.mlp.experts.106.gate_proj.weight": "model-00089-of-00101.safetensors", + "model.layers.81.mlp.experts.106.up_proj.weight": "model-00089-of-00101.safetensors", + "model.layers.81.mlp.experts.106.down_proj.weight": "model-00089-of-00101.safetensors", + "model.layers.81.mlp.experts.107.gate_proj.weight": "model-00089-of-00101.safetensors", + "model.layers.81.mlp.experts.107.up_proj.weight": "model-00089-of-00101.safetensors", + "model.layers.81.mlp.experts.107.down_proj.weight": "model-00089-of-00101.safetensors", + "model.layers.81.mlp.experts.108.gate_proj.weight": "model-00089-of-00101.safetensors", + "model.layers.81.mlp.experts.108.up_proj.weight": "model-00089-of-00101.safetensors", + "model.layers.81.mlp.experts.108.down_proj.weight": "model-00089-of-00101.safetensors", + "model.layers.81.mlp.experts.109.gate_proj.weight": "model-00089-of-00101.safetensors", + "model.layers.81.mlp.experts.109.up_proj.weight": "model-00089-of-00101.safetensors", + "model.layers.81.mlp.experts.109.down_proj.weight": "model-00089-of-00101.safetensors", + "model.layers.81.mlp.experts.110.gate_proj.weight": "model-00089-of-00101.safetensors", + "model.layers.81.mlp.experts.110.up_proj.weight": "model-00089-of-00101.safetensors", + "model.layers.81.mlp.experts.110.down_proj.weight": "model-00089-of-00101.safetensors", + "model.layers.81.mlp.experts.111.gate_proj.weight": "model-00089-of-00101.safetensors", + "model.layers.81.mlp.experts.111.up_proj.weight": "model-00089-of-00101.safetensors", + "model.layers.81.mlp.experts.111.down_proj.weight": "model-00089-of-00101.safetensors", + "model.layers.81.mlp.experts.112.gate_proj.weight": "model-00089-of-00101.safetensors", + "model.layers.81.mlp.experts.112.up_proj.weight": "model-00089-of-00101.safetensors", + "model.layers.81.mlp.experts.112.down_proj.weight": "model-00089-of-00101.safetensors", + "model.layers.81.mlp.experts.113.gate_proj.weight": "model-00089-of-00101.safetensors", + "model.layers.81.mlp.experts.113.up_proj.weight": "model-00089-of-00101.safetensors", + "model.layers.81.mlp.experts.113.down_proj.weight": "model-00089-of-00101.safetensors", + "model.layers.81.mlp.experts.114.gate_proj.weight": "model-00089-of-00101.safetensors", + "model.layers.81.mlp.experts.114.up_proj.weight": "model-00089-of-00101.safetensors", + "model.layers.81.mlp.experts.114.down_proj.weight": "model-00089-of-00101.safetensors", + "model.layers.81.mlp.experts.115.gate_proj.weight": "model-00089-of-00101.safetensors", + "model.layers.81.mlp.experts.115.up_proj.weight": "model-00089-of-00101.safetensors", + "model.layers.81.mlp.experts.115.down_proj.weight": "model-00089-of-00101.safetensors", + "model.layers.81.mlp.experts.116.gate_proj.weight": "model-00089-of-00101.safetensors", + "model.layers.81.mlp.experts.116.up_proj.weight": "model-00089-of-00101.safetensors", + "model.layers.81.mlp.experts.116.down_proj.weight": "model-00089-of-00101.safetensors", + "model.layers.81.mlp.experts.117.gate_proj.weight": "model-00089-of-00101.safetensors", + "model.layers.81.mlp.experts.117.up_proj.weight": "model-00089-of-00101.safetensors", + "model.layers.81.mlp.experts.117.down_proj.weight": "model-00089-of-00101.safetensors", + "model.layers.81.mlp.experts.118.gate_proj.weight": "model-00089-of-00101.safetensors", + "model.layers.81.mlp.experts.118.up_proj.weight": "model-00089-of-00101.safetensors", + "model.layers.81.mlp.experts.118.down_proj.weight": "model-00089-of-00101.safetensors", + "model.layers.81.mlp.experts.119.gate_proj.weight": "model-00089-of-00101.safetensors", + "model.layers.81.mlp.experts.119.up_proj.weight": "model-00089-of-00101.safetensors", + "model.layers.81.mlp.experts.119.down_proj.weight": "model-00089-of-00101.safetensors", + "model.layers.81.mlp.gate.weight": "model-00089-of-00101.safetensors", + "model.layers.81.mlp.gate.e_score_correction_bias": "model-00089-of-00101.safetensors", + "model.layers.81.mlp.shared_experts.gate_proj.weight": "model-00089-of-00101.safetensors", + "model.layers.81.mlp.shared_experts.up_proj.weight": "model-00089-of-00101.safetensors", + "model.layers.81.mlp.shared_experts.down_proj.weight": "model-00089-of-00101.safetensors", + "model.layers.81.input_layernorm.weight": "model-00089-of-00101.safetensors", + "model.layers.81.post_attention_layernorm.weight": "model-00089-of-00101.safetensors", + "model.layers.82.self_attn.q_proj.weight": "model-00089-of-00101.safetensors", + "model.layers.82.self_attn.q_proj.bias": "model-00089-of-00101.safetensors", + "model.layers.82.self_attn.k_proj.weight": "model-00089-of-00101.safetensors", + "model.layers.82.self_attn.k_proj.bias": "model-00089-of-00101.safetensors", + "model.layers.82.self_attn.v_proj.weight": "model-00089-of-00101.safetensors", + "model.layers.82.self_attn.v_proj.bias": "model-00089-of-00101.safetensors", + "model.layers.82.self_attn.o_proj.weight": "model-00089-of-00101.safetensors", + "model.layers.82.self_attn.q_norm.weight": "model-00089-of-00101.safetensors", + "model.layers.82.self_attn.k_norm.weight": "model-00089-of-00101.safetensors", + "model.layers.82.mlp.experts.0.gate_proj.weight": "model-00089-of-00101.safetensors", + "model.layers.82.mlp.experts.0.up_proj.weight": "model-00089-of-00101.safetensors", + "model.layers.82.mlp.experts.0.down_proj.weight": "model-00089-of-00101.safetensors", + "model.layers.82.mlp.experts.1.gate_proj.weight": "model-00090-of-00101.safetensors", + "model.layers.82.mlp.experts.1.up_proj.weight": "model-00090-of-00101.safetensors", + "model.layers.82.mlp.experts.1.down_proj.weight": "model-00090-of-00101.safetensors", + "model.layers.82.mlp.experts.2.gate_proj.weight": "model-00090-of-00101.safetensors", + "model.layers.82.mlp.experts.2.up_proj.weight": "model-00090-of-00101.safetensors", + "model.layers.82.mlp.experts.2.down_proj.weight": "model-00090-of-00101.safetensors", + "model.layers.82.mlp.experts.3.gate_proj.weight": "model-00090-of-00101.safetensors", + "model.layers.82.mlp.experts.3.up_proj.weight": "model-00090-of-00101.safetensors", + "model.layers.82.mlp.experts.3.down_proj.weight": "model-00090-of-00101.safetensors", + "model.layers.82.mlp.experts.4.gate_proj.weight": "model-00090-of-00101.safetensors", + "model.layers.82.mlp.experts.4.up_proj.weight": "model-00090-of-00101.safetensors", + "model.layers.82.mlp.experts.4.down_proj.weight": "model-00090-of-00101.safetensors", + "model.layers.82.mlp.experts.5.gate_proj.weight": "model-00090-of-00101.safetensors", + "model.layers.82.mlp.experts.5.up_proj.weight": "model-00090-of-00101.safetensors", + "model.layers.82.mlp.experts.5.down_proj.weight": "model-00090-of-00101.safetensors", + "model.layers.82.mlp.experts.6.gate_proj.weight": "model-00090-of-00101.safetensors", + "model.layers.82.mlp.experts.6.up_proj.weight": "model-00090-of-00101.safetensors", + "model.layers.82.mlp.experts.6.down_proj.weight": "model-00090-of-00101.safetensors", + "model.layers.82.mlp.experts.7.gate_proj.weight": "model-00090-of-00101.safetensors", + "model.layers.82.mlp.experts.7.up_proj.weight": "model-00090-of-00101.safetensors", + "model.layers.82.mlp.experts.7.down_proj.weight": "model-00090-of-00101.safetensors", + "model.layers.82.mlp.experts.8.gate_proj.weight": "model-00090-of-00101.safetensors", + "model.layers.82.mlp.experts.8.up_proj.weight": "model-00090-of-00101.safetensors", + "model.layers.82.mlp.experts.8.down_proj.weight": "model-00090-of-00101.safetensors", + "model.layers.82.mlp.experts.9.gate_proj.weight": "model-00090-of-00101.safetensors", + "model.layers.82.mlp.experts.9.up_proj.weight": "model-00090-of-00101.safetensors", + "model.layers.82.mlp.experts.9.down_proj.weight": "model-00090-of-00101.safetensors", + "model.layers.82.mlp.experts.10.gate_proj.weight": "model-00090-of-00101.safetensors", + "model.layers.82.mlp.experts.10.up_proj.weight": "model-00090-of-00101.safetensors", + "model.layers.82.mlp.experts.10.down_proj.weight": "model-00090-of-00101.safetensors", + "model.layers.82.mlp.experts.11.gate_proj.weight": "model-00090-of-00101.safetensors", + "model.layers.82.mlp.experts.11.up_proj.weight": "model-00090-of-00101.safetensors", + "model.layers.82.mlp.experts.11.down_proj.weight": "model-00090-of-00101.safetensors", + "model.layers.82.mlp.experts.12.gate_proj.weight": "model-00090-of-00101.safetensors", + "model.layers.82.mlp.experts.12.up_proj.weight": "model-00090-of-00101.safetensors", + "model.layers.82.mlp.experts.12.down_proj.weight": "model-00090-of-00101.safetensors", + "model.layers.82.mlp.experts.13.gate_proj.weight": "model-00090-of-00101.safetensors", + "model.layers.82.mlp.experts.13.up_proj.weight": "model-00090-of-00101.safetensors", + "model.layers.82.mlp.experts.13.down_proj.weight": "model-00090-of-00101.safetensors", + "model.layers.82.mlp.experts.14.gate_proj.weight": "model-00090-of-00101.safetensors", + "model.layers.82.mlp.experts.14.up_proj.weight": "model-00090-of-00101.safetensors", + "model.layers.82.mlp.experts.14.down_proj.weight": "model-00090-of-00101.safetensors", + "model.layers.82.mlp.experts.15.gate_proj.weight": "model-00090-of-00101.safetensors", + "model.layers.82.mlp.experts.15.up_proj.weight": "model-00090-of-00101.safetensors", + "model.layers.82.mlp.experts.15.down_proj.weight": "model-00090-of-00101.safetensors", + "model.layers.82.mlp.experts.16.gate_proj.weight": "model-00090-of-00101.safetensors", + "model.layers.82.mlp.experts.16.up_proj.weight": "model-00090-of-00101.safetensors", + "model.layers.82.mlp.experts.16.down_proj.weight": "model-00090-of-00101.safetensors", + "model.layers.82.mlp.experts.17.gate_proj.weight": "model-00090-of-00101.safetensors", + "model.layers.82.mlp.experts.17.up_proj.weight": "model-00090-of-00101.safetensors", + "model.layers.82.mlp.experts.17.down_proj.weight": "model-00090-of-00101.safetensors", + "model.layers.82.mlp.experts.18.gate_proj.weight": "model-00090-of-00101.safetensors", + "model.layers.82.mlp.experts.18.up_proj.weight": "model-00090-of-00101.safetensors", + "model.layers.82.mlp.experts.18.down_proj.weight": "model-00090-of-00101.safetensors", + "model.layers.82.mlp.experts.19.gate_proj.weight": "model-00090-of-00101.safetensors", + "model.layers.82.mlp.experts.19.up_proj.weight": "model-00090-of-00101.safetensors", + "model.layers.82.mlp.experts.19.down_proj.weight": "model-00090-of-00101.safetensors", + "model.layers.82.mlp.experts.20.gate_proj.weight": "model-00090-of-00101.safetensors", + "model.layers.82.mlp.experts.20.up_proj.weight": "model-00090-of-00101.safetensors", + "model.layers.82.mlp.experts.20.down_proj.weight": "model-00090-of-00101.safetensors", + "model.layers.82.mlp.experts.21.gate_proj.weight": "model-00090-of-00101.safetensors", + "model.layers.82.mlp.experts.21.up_proj.weight": "model-00090-of-00101.safetensors", + "model.layers.82.mlp.experts.21.down_proj.weight": "model-00090-of-00101.safetensors", + "model.layers.82.mlp.experts.22.gate_proj.weight": "model-00090-of-00101.safetensors", + "model.layers.82.mlp.experts.22.up_proj.weight": "model-00090-of-00101.safetensors", + "model.layers.82.mlp.experts.22.down_proj.weight": "model-00090-of-00101.safetensors", + "model.layers.82.mlp.experts.23.gate_proj.weight": "model-00090-of-00101.safetensors", + "model.layers.82.mlp.experts.23.up_proj.weight": "model-00090-of-00101.safetensors", + "model.layers.82.mlp.experts.23.down_proj.weight": "model-00090-of-00101.safetensors", + "model.layers.82.mlp.experts.24.gate_proj.weight": "model-00090-of-00101.safetensors", + "model.layers.82.mlp.experts.24.up_proj.weight": "model-00090-of-00101.safetensors", + "model.layers.82.mlp.experts.24.down_proj.weight": "model-00090-of-00101.safetensors", + "model.layers.82.mlp.experts.25.gate_proj.weight": "model-00090-of-00101.safetensors", + "model.layers.82.mlp.experts.25.up_proj.weight": "model-00090-of-00101.safetensors", + "model.layers.82.mlp.experts.25.down_proj.weight": "model-00090-of-00101.safetensors", + "model.layers.82.mlp.experts.26.gate_proj.weight": "model-00090-of-00101.safetensors", + "model.layers.82.mlp.experts.26.up_proj.weight": "model-00090-of-00101.safetensors", + "model.layers.82.mlp.experts.26.down_proj.weight": "model-00090-of-00101.safetensors", + "model.layers.82.mlp.experts.27.gate_proj.weight": "model-00090-of-00101.safetensors", + "model.layers.82.mlp.experts.27.up_proj.weight": "model-00090-of-00101.safetensors", + "model.layers.82.mlp.experts.27.down_proj.weight": "model-00090-of-00101.safetensors", + "model.layers.82.mlp.experts.28.gate_proj.weight": "model-00090-of-00101.safetensors", + "model.layers.82.mlp.experts.28.up_proj.weight": "model-00090-of-00101.safetensors", + "model.layers.82.mlp.experts.28.down_proj.weight": "model-00090-of-00101.safetensors", + "model.layers.82.mlp.experts.29.gate_proj.weight": "model-00090-of-00101.safetensors", + "model.layers.82.mlp.experts.29.up_proj.weight": "model-00090-of-00101.safetensors", + "model.layers.82.mlp.experts.29.down_proj.weight": "model-00090-of-00101.safetensors", + "model.layers.82.mlp.experts.30.gate_proj.weight": "model-00090-of-00101.safetensors", + "model.layers.82.mlp.experts.30.up_proj.weight": "model-00090-of-00101.safetensors", + "model.layers.82.mlp.experts.30.down_proj.weight": "model-00090-of-00101.safetensors", + "model.layers.82.mlp.experts.31.gate_proj.weight": "model-00090-of-00101.safetensors", + "model.layers.82.mlp.experts.31.up_proj.weight": "model-00090-of-00101.safetensors", + "model.layers.82.mlp.experts.31.down_proj.weight": "model-00090-of-00101.safetensors", + "model.layers.82.mlp.experts.32.gate_proj.weight": "model-00090-of-00101.safetensors", + "model.layers.82.mlp.experts.32.up_proj.weight": "model-00090-of-00101.safetensors", + "model.layers.82.mlp.experts.32.down_proj.weight": "model-00090-of-00101.safetensors", + "model.layers.82.mlp.experts.33.gate_proj.weight": "model-00090-of-00101.safetensors", + "model.layers.82.mlp.experts.33.up_proj.weight": "model-00090-of-00101.safetensors", + "model.layers.82.mlp.experts.33.down_proj.weight": "model-00090-of-00101.safetensors", + "model.layers.82.mlp.experts.34.gate_proj.weight": "model-00090-of-00101.safetensors", + "model.layers.82.mlp.experts.34.up_proj.weight": "model-00090-of-00101.safetensors", + "model.layers.82.mlp.experts.34.down_proj.weight": "model-00090-of-00101.safetensors", + "model.layers.82.mlp.experts.35.gate_proj.weight": "model-00090-of-00101.safetensors", + "model.layers.82.mlp.experts.35.up_proj.weight": "model-00090-of-00101.safetensors", + "model.layers.82.mlp.experts.35.down_proj.weight": "model-00090-of-00101.safetensors", + "model.layers.82.mlp.experts.36.gate_proj.weight": "model-00090-of-00101.safetensors", + "model.layers.82.mlp.experts.36.up_proj.weight": "model-00090-of-00101.safetensors", + "model.layers.82.mlp.experts.36.down_proj.weight": "model-00090-of-00101.safetensors", + "model.layers.82.mlp.experts.37.gate_proj.weight": "model-00090-of-00101.safetensors", + "model.layers.82.mlp.experts.37.up_proj.weight": "model-00090-of-00101.safetensors", + "model.layers.82.mlp.experts.37.down_proj.weight": "model-00090-of-00101.safetensors", + "model.layers.82.mlp.experts.38.gate_proj.weight": "model-00090-of-00101.safetensors", + "model.layers.82.mlp.experts.38.up_proj.weight": "model-00090-of-00101.safetensors", + "model.layers.82.mlp.experts.38.down_proj.weight": "model-00090-of-00101.safetensors", + "model.layers.82.mlp.experts.39.gate_proj.weight": "model-00090-of-00101.safetensors", + "model.layers.82.mlp.experts.39.up_proj.weight": "model-00090-of-00101.safetensors", + "model.layers.82.mlp.experts.39.down_proj.weight": "model-00090-of-00101.safetensors", + "model.layers.82.mlp.experts.40.gate_proj.weight": "model-00090-of-00101.safetensors", + "model.layers.82.mlp.experts.40.up_proj.weight": "model-00090-of-00101.safetensors", + "model.layers.82.mlp.experts.40.down_proj.weight": "model-00090-of-00101.safetensors", + "model.layers.82.mlp.experts.41.gate_proj.weight": "model-00090-of-00101.safetensors", + "model.layers.82.mlp.experts.41.up_proj.weight": "model-00090-of-00101.safetensors", + "model.layers.82.mlp.experts.41.down_proj.weight": "model-00090-of-00101.safetensors", + "model.layers.82.mlp.experts.42.gate_proj.weight": "model-00090-of-00101.safetensors", + "model.layers.82.mlp.experts.42.up_proj.weight": "model-00090-of-00101.safetensors", + "model.layers.82.mlp.experts.42.down_proj.weight": "model-00090-of-00101.safetensors", + "model.layers.82.mlp.experts.43.gate_proj.weight": "model-00090-of-00101.safetensors", + "model.layers.82.mlp.experts.43.up_proj.weight": "model-00090-of-00101.safetensors", + "model.layers.82.mlp.experts.43.down_proj.weight": "model-00090-of-00101.safetensors", + "model.layers.82.mlp.experts.44.gate_proj.weight": "model-00090-of-00101.safetensors", + "model.layers.82.mlp.experts.44.up_proj.weight": "model-00090-of-00101.safetensors", + "model.layers.82.mlp.experts.44.down_proj.weight": "model-00090-of-00101.safetensors", + "model.layers.82.mlp.experts.45.gate_proj.weight": "model-00090-of-00101.safetensors", + "model.layers.82.mlp.experts.45.up_proj.weight": "model-00090-of-00101.safetensors", + "model.layers.82.mlp.experts.45.down_proj.weight": "model-00090-of-00101.safetensors", + "model.layers.82.mlp.experts.46.gate_proj.weight": "model-00090-of-00101.safetensors", + "model.layers.82.mlp.experts.46.up_proj.weight": "model-00090-of-00101.safetensors", + "model.layers.82.mlp.experts.46.down_proj.weight": "model-00090-of-00101.safetensors", + "model.layers.82.mlp.experts.47.gate_proj.weight": "model-00090-of-00101.safetensors", + "model.layers.82.mlp.experts.47.up_proj.weight": "model-00090-of-00101.safetensors", + "model.layers.82.mlp.experts.47.down_proj.weight": "model-00090-of-00101.safetensors", + "model.layers.82.mlp.experts.48.gate_proj.weight": "model-00090-of-00101.safetensors", + "model.layers.82.mlp.experts.48.up_proj.weight": "model-00090-of-00101.safetensors", + "model.layers.82.mlp.experts.48.down_proj.weight": "model-00090-of-00101.safetensors", + "model.layers.82.mlp.experts.49.gate_proj.weight": "model-00090-of-00101.safetensors", + "model.layers.82.mlp.experts.49.up_proj.weight": "model-00090-of-00101.safetensors", + "model.layers.82.mlp.experts.49.down_proj.weight": "model-00090-of-00101.safetensors", + "model.layers.82.mlp.experts.50.gate_proj.weight": "model-00090-of-00101.safetensors", + "model.layers.82.mlp.experts.50.up_proj.weight": "model-00090-of-00101.safetensors", + "model.layers.82.mlp.experts.50.down_proj.weight": "model-00090-of-00101.safetensors", + "model.layers.82.mlp.experts.51.gate_proj.weight": "model-00090-of-00101.safetensors", + "model.layers.82.mlp.experts.51.up_proj.weight": "model-00090-of-00101.safetensors", + "model.layers.82.mlp.experts.51.down_proj.weight": "model-00090-of-00101.safetensors", + "model.layers.82.mlp.experts.52.gate_proj.weight": "model-00090-of-00101.safetensors", + "model.layers.82.mlp.experts.52.up_proj.weight": "model-00090-of-00101.safetensors", + "model.layers.82.mlp.experts.52.down_proj.weight": "model-00090-of-00101.safetensors", + "model.layers.82.mlp.experts.53.gate_proj.weight": "model-00090-of-00101.safetensors", + "model.layers.82.mlp.experts.53.up_proj.weight": "model-00090-of-00101.safetensors", + "model.layers.82.mlp.experts.53.down_proj.weight": "model-00090-of-00101.safetensors", + "model.layers.82.mlp.experts.54.gate_proj.weight": "model-00090-of-00101.safetensors", + "model.layers.82.mlp.experts.54.up_proj.weight": "model-00090-of-00101.safetensors", + "model.layers.82.mlp.experts.54.down_proj.weight": "model-00090-of-00101.safetensors", + "model.layers.82.mlp.experts.55.gate_proj.weight": "model-00090-of-00101.safetensors", + "model.layers.82.mlp.experts.55.up_proj.weight": "model-00090-of-00101.safetensors", + "model.layers.82.mlp.experts.55.down_proj.weight": "model-00090-of-00101.safetensors", + "model.layers.82.mlp.experts.56.gate_proj.weight": "model-00090-of-00101.safetensors", + "model.layers.82.mlp.experts.56.up_proj.weight": "model-00090-of-00101.safetensors", + "model.layers.82.mlp.experts.56.down_proj.weight": "model-00090-of-00101.safetensors", + "model.layers.82.mlp.experts.57.gate_proj.weight": "model-00090-of-00101.safetensors", + "model.layers.82.mlp.experts.57.up_proj.weight": "model-00090-of-00101.safetensors", + "model.layers.82.mlp.experts.57.down_proj.weight": "model-00090-of-00101.safetensors", + "model.layers.82.mlp.experts.58.gate_proj.weight": "model-00090-of-00101.safetensors", + "model.layers.82.mlp.experts.58.up_proj.weight": "model-00090-of-00101.safetensors", + "model.layers.82.mlp.experts.58.down_proj.weight": "model-00090-of-00101.safetensors", + "model.layers.82.mlp.experts.59.gate_proj.weight": "model-00090-of-00101.safetensors", + "model.layers.82.mlp.experts.59.up_proj.weight": "model-00090-of-00101.safetensors", + "model.layers.82.mlp.experts.59.down_proj.weight": "model-00090-of-00101.safetensors", + "model.layers.82.mlp.experts.60.gate_proj.weight": "model-00090-of-00101.safetensors", + "model.layers.82.mlp.experts.60.up_proj.weight": "model-00090-of-00101.safetensors", + "model.layers.82.mlp.experts.60.down_proj.weight": "model-00090-of-00101.safetensors", + "model.layers.82.mlp.experts.61.gate_proj.weight": "model-00090-of-00101.safetensors", + "model.layers.82.mlp.experts.61.up_proj.weight": "model-00090-of-00101.safetensors", + "model.layers.82.mlp.experts.61.down_proj.weight": "model-00090-of-00101.safetensors", + "model.layers.82.mlp.experts.62.gate_proj.weight": "model-00090-of-00101.safetensors", + "model.layers.82.mlp.experts.62.up_proj.weight": "model-00090-of-00101.safetensors", + "model.layers.82.mlp.experts.62.down_proj.weight": "model-00090-of-00101.safetensors", + "model.layers.82.mlp.experts.63.gate_proj.weight": "model-00090-of-00101.safetensors", + "model.layers.82.mlp.experts.63.up_proj.weight": "model-00090-of-00101.safetensors", + "model.layers.82.mlp.experts.63.down_proj.weight": "model-00090-of-00101.safetensors", + "model.layers.82.mlp.experts.64.gate_proj.weight": "model-00090-of-00101.safetensors", + "model.layers.82.mlp.experts.64.up_proj.weight": "model-00090-of-00101.safetensors", + "model.layers.82.mlp.experts.64.down_proj.weight": "model-00090-of-00101.safetensors", + "model.layers.82.mlp.experts.65.gate_proj.weight": "model-00090-of-00101.safetensors", + "model.layers.82.mlp.experts.65.up_proj.weight": "model-00090-of-00101.safetensors", + "model.layers.82.mlp.experts.65.down_proj.weight": "model-00090-of-00101.safetensors", + "model.layers.82.mlp.experts.66.gate_proj.weight": "model-00090-of-00101.safetensors", + "model.layers.82.mlp.experts.66.up_proj.weight": "model-00090-of-00101.safetensors", + "model.layers.82.mlp.experts.66.down_proj.weight": "model-00090-of-00101.safetensors", + "model.layers.82.mlp.experts.67.gate_proj.weight": "model-00090-of-00101.safetensors", + "model.layers.82.mlp.experts.67.up_proj.weight": "model-00090-of-00101.safetensors", + "model.layers.82.mlp.experts.67.down_proj.weight": "model-00090-of-00101.safetensors", + "model.layers.82.mlp.experts.68.gate_proj.weight": "model-00090-of-00101.safetensors", + "model.layers.82.mlp.experts.68.up_proj.weight": "model-00090-of-00101.safetensors", + "model.layers.82.mlp.experts.68.down_proj.weight": "model-00090-of-00101.safetensors", + "model.layers.82.mlp.experts.69.gate_proj.weight": "model-00090-of-00101.safetensors", + "model.layers.82.mlp.experts.69.up_proj.weight": "model-00090-of-00101.safetensors", + "model.layers.82.mlp.experts.69.down_proj.weight": "model-00090-of-00101.safetensors", + "model.layers.82.mlp.experts.70.gate_proj.weight": "model-00090-of-00101.safetensors", + "model.layers.82.mlp.experts.70.up_proj.weight": "model-00090-of-00101.safetensors", + "model.layers.82.mlp.experts.70.down_proj.weight": "model-00090-of-00101.safetensors", + "model.layers.82.mlp.experts.71.gate_proj.weight": "model-00090-of-00101.safetensors", + "model.layers.82.mlp.experts.71.up_proj.weight": "model-00090-of-00101.safetensors", + "model.layers.82.mlp.experts.71.down_proj.weight": "model-00090-of-00101.safetensors", + "model.layers.82.mlp.experts.72.gate_proj.weight": "model-00090-of-00101.safetensors", + "model.layers.82.mlp.experts.72.up_proj.weight": "model-00090-of-00101.safetensors", + "model.layers.82.mlp.experts.72.down_proj.weight": "model-00090-of-00101.safetensors", + "model.layers.82.mlp.experts.73.gate_proj.weight": "model-00090-of-00101.safetensors", + "model.layers.82.mlp.experts.73.up_proj.weight": "model-00090-of-00101.safetensors", + "model.layers.82.mlp.experts.73.down_proj.weight": "model-00090-of-00101.safetensors", + "model.layers.82.mlp.experts.74.gate_proj.weight": "model-00090-of-00101.safetensors", + "model.layers.82.mlp.experts.74.up_proj.weight": "model-00090-of-00101.safetensors", + "model.layers.82.mlp.experts.74.down_proj.weight": "model-00090-of-00101.safetensors", + "model.layers.82.mlp.experts.75.gate_proj.weight": "model-00090-of-00101.safetensors", + "model.layers.82.mlp.experts.75.up_proj.weight": "model-00090-of-00101.safetensors", + "model.layers.82.mlp.experts.75.down_proj.weight": "model-00090-of-00101.safetensors", + "model.layers.82.mlp.experts.76.gate_proj.weight": "model-00090-of-00101.safetensors", + "model.layers.82.mlp.experts.76.up_proj.weight": "model-00090-of-00101.safetensors", + "model.layers.82.mlp.experts.76.down_proj.weight": "model-00090-of-00101.safetensors", + "model.layers.82.mlp.experts.77.gate_proj.weight": "model-00090-of-00101.safetensors", + "model.layers.82.mlp.experts.77.up_proj.weight": "model-00090-of-00101.safetensors", + "model.layers.82.mlp.experts.77.down_proj.weight": "model-00090-of-00101.safetensors", + "model.layers.82.mlp.experts.78.gate_proj.weight": "model-00090-of-00101.safetensors", + "model.layers.82.mlp.experts.78.up_proj.weight": "model-00090-of-00101.safetensors", + "model.layers.82.mlp.experts.78.down_proj.weight": "model-00090-of-00101.safetensors", + "model.layers.82.mlp.experts.79.gate_proj.weight": "model-00090-of-00101.safetensors", + "model.layers.82.mlp.experts.79.up_proj.weight": "model-00090-of-00101.safetensors", + "model.layers.82.mlp.experts.79.down_proj.weight": "model-00090-of-00101.safetensors", + "model.layers.82.mlp.experts.80.gate_proj.weight": "model-00090-of-00101.safetensors", + "model.layers.82.mlp.experts.80.up_proj.weight": "model-00090-of-00101.safetensors", + "model.layers.82.mlp.experts.80.down_proj.weight": "model-00090-of-00101.safetensors", + "model.layers.82.mlp.experts.81.gate_proj.weight": "model-00090-of-00101.safetensors", + "model.layers.82.mlp.experts.81.up_proj.weight": "model-00090-of-00101.safetensors", + "model.layers.82.mlp.experts.81.down_proj.weight": "model-00090-of-00101.safetensors", + "model.layers.82.mlp.experts.82.gate_proj.weight": "model-00090-of-00101.safetensors", + "model.layers.82.mlp.experts.82.up_proj.weight": "model-00090-of-00101.safetensors", + "model.layers.82.mlp.experts.82.down_proj.weight": "model-00090-of-00101.safetensors", + "model.layers.82.mlp.experts.83.gate_proj.weight": "model-00090-of-00101.safetensors", + "model.layers.82.mlp.experts.83.up_proj.weight": "model-00090-of-00101.safetensors", + "model.layers.82.mlp.experts.83.down_proj.weight": "model-00090-of-00101.safetensors", + "model.layers.82.mlp.experts.84.gate_proj.weight": "model-00090-of-00101.safetensors", + "model.layers.82.mlp.experts.84.up_proj.weight": "model-00090-of-00101.safetensors", + "model.layers.82.mlp.experts.84.down_proj.weight": "model-00090-of-00101.safetensors", + "model.layers.82.mlp.experts.85.gate_proj.weight": "model-00090-of-00101.safetensors", + "model.layers.82.mlp.experts.85.up_proj.weight": "model-00090-of-00101.safetensors", + "model.layers.82.mlp.experts.85.down_proj.weight": "model-00090-of-00101.safetensors", + "model.layers.82.mlp.experts.86.gate_proj.weight": "model-00090-of-00101.safetensors", + "model.layers.82.mlp.experts.86.up_proj.weight": "model-00090-of-00101.safetensors", + "model.layers.82.mlp.experts.86.down_proj.weight": "model-00090-of-00101.safetensors", + "model.layers.82.mlp.experts.87.gate_proj.weight": "model-00090-of-00101.safetensors", + "model.layers.82.mlp.experts.87.up_proj.weight": "model-00090-of-00101.safetensors", + "model.layers.82.mlp.experts.87.down_proj.weight": "model-00090-of-00101.safetensors", + "model.layers.82.mlp.experts.88.gate_proj.weight": "model-00090-of-00101.safetensors", + "model.layers.82.mlp.experts.88.up_proj.weight": "model-00090-of-00101.safetensors", + "model.layers.82.mlp.experts.88.down_proj.weight": "model-00090-of-00101.safetensors", + "model.layers.82.mlp.experts.89.gate_proj.weight": "model-00090-of-00101.safetensors", + "model.layers.82.mlp.experts.89.up_proj.weight": "model-00090-of-00101.safetensors", + "model.layers.82.mlp.experts.89.down_proj.weight": "model-00090-of-00101.safetensors", + "model.layers.82.mlp.experts.90.gate_proj.weight": "model-00090-of-00101.safetensors", + "model.layers.82.mlp.experts.90.up_proj.weight": "model-00090-of-00101.safetensors", + "model.layers.82.mlp.experts.90.down_proj.weight": "model-00090-of-00101.safetensors", + "model.layers.82.mlp.experts.91.gate_proj.weight": "model-00090-of-00101.safetensors", + "model.layers.82.mlp.experts.91.up_proj.weight": "model-00090-of-00101.safetensors", + "model.layers.82.mlp.experts.91.down_proj.weight": "model-00090-of-00101.safetensors", + "model.layers.82.mlp.experts.92.gate_proj.weight": "model-00090-of-00101.safetensors", + "model.layers.82.mlp.experts.92.up_proj.weight": "model-00090-of-00101.safetensors", + "model.layers.82.mlp.experts.92.down_proj.weight": "model-00090-of-00101.safetensors", + "model.layers.82.mlp.experts.93.gate_proj.weight": "model-00090-of-00101.safetensors", + "model.layers.82.mlp.experts.93.up_proj.weight": "model-00090-of-00101.safetensors", + "model.layers.82.mlp.experts.93.down_proj.weight": "model-00090-of-00101.safetensors", + "model.layers.82.mlp.experts.94.gate_proj.weight": "model-00090-of-00101.safetensors", + "model.layers.82.mlp.experts.94.up_proj.weight": "model-00090-of-00101.safetensors", + "model.layers.82.mlp.experts.94.down_proj.weight": "model-00090-of-00101.safetensors", + "model.layers.82.mlp.experts.95.gate_proj.weight": "model-00090-of-00101.safetensors", + "model.layers.82.mlp.experts.95.up_proj.weight": "model-00090-of-00101.safetensors", + "model.layers.82.mlp.experts.95.down_proj.weight": "model-00090-of-00101.safetensors", + "model.layers.82.mlp.experts.96.gate_proj.weight": "model-00090-of-00101.safetensors", + "model.layers.82.mlp.experts.96.up_proj.weight": "model-00090-of-00101.safetensors", + "model.layers.82.mlp.experts.96.down_proj.weight": "model-00090-of-00101.safetensors", + "model.layers.82.mlp.experts.97.gate_proj.weight": "model-00090-of-00101.safetensors", + "model.layers.82.mlp.experts.97.up_proj.weight": "model-00090-of-00101.safetensors", + "model.layers.82.mlp.experts.97.down_proj.weight": "model-00090-of-00101.safetensors", + "model.layers.82.mlp.experts.98.gate_proj.weight": "model-00090-of-00101.safetensors", + "model.layers.82.mlp.experts.98.up_proj.weight": "model-00090-of-00101.safetensors", + "model.layers.82.mlp.experts.98.down_proj.weight": "model-00090-of-00101.safetensors", + "model.layers.82.mlp.experts.99.gate_proj.weight": "model-00090-of-00101.safetensors", + "model.layers.82.mlp.experts.99.up_proj.weight": "model-00090-of-00101.safetensors", + "model.layers.82.mlp.experts.99.down_proj.weight": "model-00090-of-00101.safetensors", + "model.layers.82.mlp.experts.100.gate_proj.weight": "model-00090-of-00101.safetensors", + "model.layers.82.mlp.experts.100.up_proj.weight": "model-00090-of-00101.safetensors", + "model.layers.82.mlp.experts.100.down_proj.weight": "model-00090-of-00101.safetensors", + "model.layers.82.mlp.experts.101.gate_proj.weight": "model-00090-of-00101.safetensors", + "model.layers.82.mlp.experts.101.up_proj.weight": "model-00090-of-00101.safetensors", + "model.layers.82.mlp.experts.101.down_proj.weight": "model-00090-of-00101.safetensors", + "model.layers.82.mlp.experts.102.gate_proj.weight": "model-00090-of-00101.safetensors", + "model.layers.82.mlp.experts.102.up_proj.weight": "model-00090-of-00101.safetensors", + "model.layers.82.mlp.experts.102.down_proj.weight": "model-00090-of-00101.safetensors", + "model.layers.82.mlp.experts.103.gate_proj.weight": "model-00090-of-00101.safetensors", + "model.layers.82.mlp.experts.103.up_proj.weight": "model-00090-of-00101.safetensors", + "model.layers.82.mlp.experts.103.down_proj.weight": "model-00090-of-00101.safetensors", + "model.layers.82.mlp.experts.104.gate_proj.weight": "model-00090-of-00101.safetensors", + "model.layers.82.mlp.experts.104.up_proj.weight": "model-00090-of-00101.safetensors", + "model.layers.82.mlp.experts.104.down_proj.weight": "model-00090-of-00101.safetensors", + "model.layers.82.mlp.experts.105.gate_proj.weight": "model-00090-of-00101.safetensors", + "model.layers.82.mlp.experts.105.up_proj.weight": "model-00090-of-00101.safetensors", + "model.layers.82.mlp.experts.105.down_proj.weight": "model-00090-of-00101.safetensors", + "model.layers.82.mlp.experts.106.gate_proj.weight": "model-00090-of-00101.safetensors", + "model.layers.82.mlp.experts.106.up_proj.weight": "model-00090-of-00101.safetensors", + "model.layers.82.mlp.experts.106.down_proj.weight": "model-00090-of-00101.safetensors", + "model.layers.82.mlp.experts.107.gate_proj.weight": "model-00090-of-00101.safetensors", + "model.layers.82.mlp.experts.107.up_proj.weight": "model-00090-of-00101.safetensors", + "model.layers.82.mlp.experts.107.down_proj.weight": "model-00090-of-00101.safetensors", + "model.layers.82.mlp.experts.108.gate_proj.weight": "model-00090-of-00101.safetensors", + "model.layers.82.mlp.experts.108.up_proj.weight": "model-00090-of-00101.safetensors", + "model.layers.82.mlp.experts.108.down_proj.weight": "model-00090-of-00101.safetensors", + "model.layers.82.mlp.experts.109.gate_proj.weight": "model-00090-of-00101.safetensors", + "model.layers.82.mlp.experts.109.up_proj.weight": "model-00090-of-00101.safetensors", + "model.layers.82.mlp.experts.109.down_proj.weight": "model-00090-of-00101.safetensors", + "model.layers.82.mlp.experts.110.gate_proj.weight": "model-00090-of-00101.safetensors", + "model.layers.82.mlp.experts.110.up_proj.weight": "model-00090-of-00101.safetensors", + "model.layers.82.mlp.experts.110.down_proj.weight": "model-00090-of-00101.safetensors", + "model.layers.82.mlp.experts.111.gate_proj.weight": "model-00090-of-00101.safetensors", + "model.layers.82.mlp.experts.111.up_proj.weight": "model-00090-of-00101.safetensors", + "model.layers.82.mlp.experts.111.down_proj.weight": "model-00090-of-00101.safetensors", + "model.layers.82.mlp.experts.112.gate_proj.weight": "model-00090-of-00101.safetensors", + "model.layers.82.mlp.experts.112.up_proj.weight": "model-00090-of-00101.safetensors", + "model.layers.82.mlp.experts.112.down_proj.weight": "model-00090-of-00101.safetensors", + "model.layers.82.mlp.experts.113.gate_proj.weight": "model-00090-of-00101.safetensors", + "model.layers.82.mlp.experts.113.up_proj.weight": "model-00090-of-00101.safetensors", + "model.layers.82.mlp.experts.113.down_proj.weight": "model-00090-of-00101.safetensors", + "model.layers.82.mlp.experts.114.gate_proj.weight": "model-00090-of-00101.safetensors", + "model.layers.82.mlp.experts.114.up_proj.weight": "model-00090-of-00101.safetensors", + "model.layers.82.mlp.experts.114.down_proj.weight": "model-00091-of-00101.safetensors", + "model.layers.82.mlp.experts.115.gate_proj.weight": "model-00091-of-00101.safetensors", + "model.layers.82.mlp.experts.115.up_proj.weight": "model-00091-of-00101.safetensors", + "model.layers.82.mlp.experts.115.down_proj.weight": "model-00091-of-00101.safetensors", + "model.layers.82.mlp.experts.116.gate_proj.weight": "model-00091-of-00101.safetensors", + "model.layers.82.mlp.experts.116.up_proj.weight": "model-00091-of-00101.safetensors", + "model.layers.82.mlp.experts.116.down_proj.weight": "model-00091-of-00101.safetensors", + "model.layers.82.mlp.experts.117.gate_proj.weight": "model-00091-of-00101.safetensors", + "model.layers.82.mlp.experts.117.up_proj.weight": "model-00091-of-00101.safetensors", + "model.layers.82.mlp.experts.117.down_proj.weight": "model-00091-of-00101.safetensors", + "model.layers.82.mlp.experts.118.gate_proj.weight": "model-00091-of-00101.safetensors", + "model.layers.82.mlp.experts.118.up_proj.weight": "model-00091-of-00101.safetensors", + "model.layers.82.mlp.experts.118.down_proj.weight": "model-00091-of-00101.safetensors", + "model.layers.82.mlp.experts.119.gate_proj.weight": "model-00091-of-00101.safetensors", + "model.layers.82.mlp.experts.119.up_proj.weight": "model-00091-of-00101.safetensors", + "model.layers.82.mlp.experts.119.down_proj.weight": "model-00091-of-00101.safetensors", + "model.layers.82.mlp.gate.weight": "model-00091-of-00101.safetensors", + "model.layers.82.mlp.gate.e_score_correction_bias": "model-00091-of-00101.safetensors", + "model.layers.82.mlp.shared_experts.gate_proj.weight": "model-00091-of-00101.safetensors", + "model.layers.82.mlp.shared_experts.up_proj.weight": "model-00091-of-00101.safetensors", + "model.layers.82.mlp.shared_experts.down_proj.weight": "model-00091-of-00101.safetensors", + "model.layers.82.input_layernorm.weight": "model-00091-of-00101.safetensors", + "model.layers.82.post_attention_layernorm.weight": "model-00091-of-00101.safetensors", + "model.layers.83.self_attn.q_proj.weight": "model-00091-of-00101.safetensors", + "model.layers.83.self_attn.q_proj.bias": "model-00091-of-00101.safetensors", + "model.layers.83.self_attn.k_proj.weight": "model-00091-of-00101.safetensors", + "model.layers.83.self_attn.k_proj.bias": "model-00091-of-00101.safetensors", + "model.layers.83.self_attn.v_proj.weight": "model-00091-of-00101.safetensors", + "model.layers.83.self_attn.v_proj.bias": "model-00091-of-00101.safetensors", + "model.layers.83.self_attn.o_proj.weight": "model-00091-of-00101.safetensors", + "model.layers.83.self_attn.q_norm.weight": "model-00091-of-00101.safetensors", + "model.layers.83.self_attn.k_norm.weight": "model-00091-of-00101.safetensors", + "model.layers.83.mlp.experts.0.gate_proj.weight": "model-00091-of-00101.safetensors", + "model.layers.83.mlp.experts.0.up_proj.weight": "model-00091-of-00101.safetensors", + "model.layers.83.mlp.experts.0.down_proj.weight": "model-00091-of-00101.safetensors", + "model.layers.83.mlp.experts.1.gate_proj.weight": "model-00091-of-00101.safetensors", + "model.layers.83.mlp.experts.1.up_proj.weight": "model-00091-of-00101.safetensors", + "model.layers.83.mlp.experts.1.down_proj.weight": "model-00091-of-00101.safetensors", + "model.layers.83.mlp.experts.2.gate_proj.weight": "model-00091-of-00101.safetensors", + "model.layers.83.mlp.experts.2.up_proj.weight": "model-00091-of-00101.safetensors", + "model.layers.83.mlp.experts.2.down_proj.weight": "model-00091-of-00101.safetensors", + "model.layers.83.mlp.experts.3.gate_proj.weight": "model-00091-of-00101.safetensors", + "model.layers.83.mlp.experts.3.up_proj.weight": "model-00091-of-00101.safetensors", + "model.layers.83.mlp.experts.3.down_proj.weight": "model-00091-of-00101.safetensors", + "model.layers.83.mlp.experts.4.gate_proj.weight": "model-00091-of-00101.safetensors", + "model.layers.83.mlp.experts.4.up_proj.weight": "model-00091-of-00101.safetensors", + "model.layers.83.mlp.experts.4.down_proj.weight": "model-00091-of-00101.safetensors", + "model.layers.83.mlp.experts.5.gate_proj.weight": "model-00091-of-00101.safetensors", + "model.layers.83.mlp.experts.5.up_proj.weight": "model-00091-of-00101.safetensors", + "model.layers.83.mlp.experts.5.down_proj.weight": "model-00091-of-00101.safetensors", + "model.layers.83.mlp.experts.6.gate_proj.weight": "model-00091-of-00101.safetensors", + "model.layers.83.mlp.experts.6.up_proj.weight": "model-00091-of-00101.safetensors", + "model.layers.83.mlp.experts.6.down_proj.weight": "model-00091-of-00101.safetensors", + "model.layers.83.mlp.experts.7.gate_proj.weight": "model-00091-of-00101.safetensors", + "model.layers.83.mlp.experts.7.up_proj.weight": "model-00091-of-00101.safetensors", + "model.layers.83.mlp.experts.7.down_proj.weight": "model-00091-of-00101.safetensors", + "model.layers.83.mlp.experts.8.gate_proj.weight": "model-00091-of-00101.safetensors", + "model.layers.83.mlp.experts.8.up_proj.weight": "model-00091-of-00101.safetensors", + "model.layers.83.mlp.experts.8.down_proj.weight": "model-00091-of-00101.safetensors", + "model.layers.83.mlp.experts.9.gate_proj.weight": "model-00091-of-00101.safetensors", + "model.layers.83.mlp.experts.9.up_proj.weight": "model-00091-of-00101.safetensors", + "model.layers.83.mlp.experts.9.down_proj.weight": "model-00091-of-00101.safetensors", + "model.layers.83.mlp.experts.10.gate_proj.weight": "model-00091-of-00101.safetensors", + "model.layers.83.mlp.experts.10.up_proj.weight": "model-00091-of-00101.safetensors", + "model.layers.83.mlp.experts.10.down_proj.weight": "model-00091-of-00101.safetensors", + "model.layers.83.mlp.experts.11.gate_proj.weight": "model-00091-of-00101.safetensors", + "model.layers.83.mlp.experts.11.up_proj.weight": "model-00091-of-00101.safetensors", + "model.layers.83.mlp.experts.11.down_proj.weight": "model-00091-of-00101.safetensors", + "model.layers.83.mlp.experts.12.gate_proj.weight": "model-00091-of-00101.safetensors", + "model.layers.83.mlp.experts.12.up_proj.weight": "model-00091-of-00101.safetensors", + "model.layers.83.mlp.experts.12.down_proj.weight": "model-00091-of-00101.safetensors", + "model.layers.83.mlp.experts.13.gate_proj.weight": "model-00091-of-00101.safetensors", + "model.layers.83.mlp.experts.13.up_proj.weight": "model-00091-of-00101.safetensors", + "model.layers.83.mlp.experts.13.down_proj.weight": "model-00091-of-00101.safetensors", + "model.layers.83.mlp.experts.14.gate_proj.weight": "model-00091-of-00101.safetensors", + "model.layers.83.mlp.experts.14.up_proj.weight": "model-00091-of-00101.safetensors", + "model.layers.83.mlp.experts.14.down_proj.weight": "model-00091-of-00101.safetensors", + "model.layers.83.mlp.experts.15.gate_proj.weight": "model-00091-of-00101.safetensors", + "model.layers.83.mlp.experts.15.up_proj.weight": "model-00091-of-00101.safetensors", + "model.layers.83.mlp.experts.15.down_proj.weight": "model-00091-of-00101.safetensors", + "model.layers.83.mlp.experts.16.gate_proj.weight": "model-00091-of-00101.safetensors", + "model.layers.83.mlp.experts.16.up_proj.weight": "model-00091-of-00101.safetensors", + "model.layers.83.mlp.experts.16.down_proj.weight": "model-00091-of-00101.safetensors", + "model.layers.83.mlp.experts.17.gate_proj.weight": "model-00091-of-00101.safetensors", + "model.layers.83.mlp.experts.17.up_proj.weight": "model-00091-of-00101.safetensors", + "model.layers.83.mlp.experts.17.down_proj.weight": "model-00091-of-00101.safetensors", + "model.layers.83.mlp.experts.18.gate_proj.weight": "model-00091-of-00101.safetensors", + "model.layers.83.mlp.experts.18.up_proj.weight": "model-00091-of-00101.safetensors", + "model.layers.83.mlp.experts.18.down_proj.weight": "model-00091-of-00101.safetensors", + "model.layers.83.mlp.experts.19.gate_proj.weight": "model-00091-of-00101.safetensors", + "model.layers.83.mlp.experts.19.up_proj.weight": "model-00091-of-00101.safetensors", + "model.layers.83.mlp.experts.19.down_proj.weight": "model-00091-of-00101.safetensors", + "model.layers.83.mlp.experts.20.gate_proj.weight": "model-00091-of-00101.safetensors", + "model.layers.83.mlp.experts.20.up_proj.weight": "model-00091-of-00101.safetensors", + "model.layers.83.mlp.experts.20.down_proj.weight": "model-00091-of-00101.safetensors", + "model.layers.83.mlp.experts.21.gate_proj.weight": "model-00091-of-00101.safetensors", + "model.layers.83.mlp.experts.21.up_proj.weight": "model-00091-of-00101.safetensors", + "model.layers.83.mlp.experts.21.down_proj.weight": "model-00091-of-00101.safetensors", + "model.layers.83.mlp.experts.22.gate_proj.weight": "model-00091-of-00101.safetensors", + "model.layers.83.mlp.experts.22.up_proj.weight": "model-00091-of-00101.safetensors", + "model.layers.83.mlp.experts.22.down_proj.weight": "model-00091-of-00101.safetensors", + "model.layers.83.mlp.experts.23.gate_proj.weight": "model-00091-of-00101.safetensors", + "model.layers.83.mlp.experts.23.up_proj.weight": "model-00091-of-00101.safetensors", + "model.layers.83.mlp.experts.23.down_proj.weight": "model-00091-of-00101.safetensors", + "model.layers.83.mlp.experts.24.gate_proj.weight": "model-00091-of-00101.safetensors", + "model.layers.83.mlp.experts.24.up_proj.weight": "model-00091-of-00101.safetensors", + "model.layers.83.mlp.experts.24.down_proj.weight": "model-00091-of-00101.safetensors", + "model.layers.83.mlp.experts.25.gate_proj.weight": "model-00091-of-00101.safetensors", + "model.layers.83.mlp.experts.25.up_proj.weight": "model-00091-of-00101.safetensors", + "model.layers.83.mlp.experts.25.down_proj.weight": "model-00091-of-00101.safetensors", + "model.layers.83.mlp.experts.26.gate_proj.weight": "model-00091-of-00101.safetensors", + "model.layers.83.mlp.experts.26.up_proj.weight": "model-00091-of-00101.safetensors", + "model.layers.83.mlp.experts.26.down_proj.weight": "model-00091-of-00101.safetensors", + "model.layers.83.mlp.experts.27.gate_proj.weight": "model-00091-of-00101.safetensors", + "model.layers.83.mlp.experts.27.up_proj.weight": "model-00091-of-00101.safetensors", + "model.layers.83.mlp.experts.27.down_proj.weight": "model-00091-of-00101.safetensors", + "model.layers.83.mlp.experts.28.gate_proj.weight": "model-00091-of-00101.safetensors", + "model.layers.83.mlp.experts.28.up_proj.weight": "model-00091-of-00101.safetensors", + "model.layers.83.mlp.experts.28.down_proj.weight": "model-00091-of-00101.safetensors", + "model.layers.83.mlp.experts.29.gate_proj.weight": "model-00091-of-00101.safetensors", + "model.layers.83.mlp.experts.29.up_proj.weight": "model-00091-of-00101.safetensors", + "model.layers.83.mlp.experts.29.down_proj.weight": "model-00091-of-00101.safetensors", + "model.layers.83.mlp.experts.30.gate_proj.weight": "model-00091-of-00101.safetensors", + "model.layers.83.mlp.experts.30.up_proj.weight": "model-00091-of-00101.safetensors", + "model.layers.83.mlp.experts.30.down_proj.weight": "model-00091-of-00101.safetensors", + "model.layers.83.mlp.experts.31.gate_proj.weight": "model-00091-of-00101.safetensors", + "model.layers.83.mlp.experts.31.up_proj.weight": "model-00091-of-00101.safetensors", + "model.layers.83.mlp.experts.31.down_proj.weight": "model-00091-of-00101.safetensors", + "model.layers.83.mlp.experts.32.gate_proj.weight": "model-00091-of-00101.safetensors", + "model.layers.83.mlp.experts.32.up_proj.weight": "model-00091-of-00101.safetensors", + "model.layers.83.mlp.experts.32.down_proj.weight": "model-00091-of-00101.safetensors", + "model.layers.83.mlp.experts.33.gate_proj.weight": "model-00091-of-00101.safetensors", + "model.layers.83.mlp.experts.33.up_proj.weight": "model-00091-of-00101.safetensors", + "model.layers.83.mlp.experts.33.down_proj.weight": "model-00091-of-00101.safetensors", + "model.layers.83.mlp.experts.34.gate_proj.weight": "model-00091-of-00101.safetensors", + "model.layers.83.mlp.experts.34.up_proj.weight": "model-00091-of-00101.safetensors", + "model.layers.83.mlp.experts.34.down_proj.weight": "model-00091-of-00101.safetensors", + "model.layers.83.mlp.experts.35.gate_proj.weight": "model-00091-of-00101.safetensors", + "model.layers.83.mlp.experts.35.up_proj.weight": "model-00091-of-00101.safetensors", + "model.layers.83.mlp.experts.35.down_proj.weight": "model-00091-of-00101.safetensors", + "model.layers.83.mlp.experts.36.gate_proj.weight": "model-00091-of-00101.safetensors", + "model.layers.83.mlp.experts.36.up_proj.weight": "model-00091-of-00101.safetensors", + "model.layers.83.mlp.experts.36.down_proj.weight": "model-00091-of-00101.safetensors", + "model.layers.83.mlp.experts.37.gate_proj.weight": "model-00091-of-00101.safetensors", + "model.layers.83.mlp.experts.37.up_proj.weight": "model-00091-of-00101.safetensors", + "model.layers.83.mlp.experts.37.down_proj.weight": "model-00091-of-00101.safetensors", + "model.layers.83.mlp.experts.38.gate_proj.weight": "model-00091-of-00101.safetensors", + "model.layers.83.mlp.experts.38.up_proj.weight": "model-00091-of-00101.safetensors", + "model.layers.83.mlp.experts.38.down_proj.weight": "model-00091-of-00101.safetensors", + "model.layers.83.mlp.experts.39.gate_proj.weight": "model-00091-of-00101.safetensors", + "model.layers.83.mlp.experts.39.up_proj.weight": "model-00091-of-00101.safetensors", + "model.layers.83.mlp.experts.39.down_proj.weight": "model-00091-of-00101.safetensors", + "model.layers.83.mlp.experts.40.gate_proj.weight": "model-00091-of-00101.safetensors", + "model.layers.83.mlp.experts.40.up_proj.weight": "model-00091-of-00101.safetensors", + "model.layers.83.mlp.experts.40.down_proj.weight": "model-00091-of-00101.safetensors", + "model.layers.83.mlp.experts.41.gate_proj.weight": "model-00091-of-00101.safetensors", + "model.layers.83.mlp.experts.41.up_proj.weight": "model-00091-of-00101.safetensors", + "model.layers.83.mlp.experts.41.down_proj.weight": "model-00091-of-00101.safetensors", + "model.layers.83.mlp.experts.42.gate_proj.weight": "model-00091-of-00101.safetensors", + "model.layers.83.mlp.experts.42.up_proj.weight": "model-00091-of-00101.safetensors", + "model.layers.83.mlp.experts.42.down_proj.weight": "model-00091-of-00101.safetensors", + "model.layers.83.mlp.experts.43.gate_proj.weight": "model-00091-of-00101.safetensors", + "model.layers.83.mlp.experts.43.up_proj.weight": "model-00091-of-00101.safetensors", + "model.layers.83.mlp.experts.43.down_proj.weight": "model-00091-of-00101.safetensors", + "model.layers.83.mlp.experts.44.gate_proj.weight": "model-00091-of-00101.safetensors", + "model.layers.83.mlp.experts.44.up_proj.weight": "model-00091-of-00101.safetensors", + "model.layers.83.mlp.experts.44.down_proj.weight": "model-00091-of-00101.safetensors", + "model.layers.83.mlp.experts.45.gate_proj.weight": "model-00091-of-00101.safetensors", + "model.layers.83.mlp.experts.45.up_proj.weight": "model-00091-of-00101.safetensors", + "model.layers.83.mlp.experts.45.down_proj.weight": "model-00091-of-00101.safetensors", + "model.layers.83.mlp.experts.46.gate_proj.weight": "model-00091-of-00101.safetensors", + "model.layers.83.mlp.experts.46.up_proj.weight": "model-00091-of-00101.safetensors", + "model.layers.83.mlp.experts.46.down_proj.weight": "model-00091-of-00101.safetensors", + "model.layers.83.mlp.experts.47.gate_proj.weight": "model-00091-of-00101.safetensors", + "model.layers.83.mlp.experts.47.up_proj.weight": "model-00091-of-00101.safetensors", + "model.layers.83.mlp.experts.47.down_proj.weight": "model-00091-of-00101.safetensors", + "model.layers.83.mlp.experts.48.gate_proj.weight": "model-00091-of-00101.safetensors", + "model.layers.83.mlp.experts.48.up_proj.weight": "model-00091-of-00101.safetensors", + "model.layers.83.mlp.experts.48.down_proj.weight": "model-00091-of-00101.safetensors", + "model.layers.83.mlp.experts.49.gate_proj.weight": "model-00091-of-00101.safetensors", + "model.layers.83.mlp.experts.49.up_proj.weight": "model-00091-of-00101.safetensors", + "model.layers.83.mlp.experts.49.down_proj.weight": "model-00091-of-00101.safetensors", + "model.layers.83.mlp.experts.50.gate_proj.weight": "model-00091-of-00101.safetensors", + "model.layers.83.mlp.experts.50.up_proj.weight": "model-00091-of-00101.safetensors", + "model.layers.83.mlp.experts.50.down_proj.weight": "model-00091-of-00101.safetensors", + "model.layers.83.mlp.experts.51.gate_proj.weight": "model-00091-of-00101.safetensors", + "model.layers.83.mlp.experts.51.up_proj.weight": "model-00091-of-00101.safetensors", + "model.layers.83.mlp.experts.51.down_proj.weight": "model-00091-of-00101.safetensors", + "model.layers.83.mlp.experts.52.gate_proj.weight": "model-00091-of-00101.safetensors", + "model.layers.83.mlp.experts.52.up_proj.weight": "model-00091-of-00101.safetensors", + "model.layers.83.mlp.experts.52.down_proj.weight": "model-00091-of-00101.safetensors", + "model.layers.83.mlp.experts.53.gate_proj.weight": "model-00091-of-00101.safetensors", + "model.layers.83.mlp.experts.53.up_proj.weight": "model-00091-of-00101.safetensors", + "model.layers.83.mlp.experts.53.down_proj.weight": "model-00091-of-00101.safetensors", + "model.layers.83.mlp.experts.54.gate_proj.weight": "model-00091-of-00101.safetensors", + "model.layers.83.mlp.experts.54.up_proj.weight": "model-00091-of-00101.safetensors", + "model.layers.83.mlp.experts.54.down_proj.weight": "model-00091-of-00101.safetensors", + "model.layers.83.mlp.experts.55.gate_proj.weight": "model-00091-of-00101.safetensors", + "model.layers.83.mlp.experts.55.up_proj.weight": "model-00091-of-00101.safetensors", + "model.layers.83.mlp.experts.55.down_proj.weight": "model-00091-of-00101.safetensors", + "model.layers.83.mlp.experts.56.gate_proj.weight": "model-00091-of-00101.safetensors", + "model.layers.83.mlp.experts.56.up_proj.weight": "model-00091-of-00101.safetensors", + "model.layers.83.mlp.experts.56.down_proj.weight": "model-00091-of-00101.safetensors", + "model.layers.83.mlp.experts.57.gate_proj.weight": "model-00091-of-00101.safetensors", + "model.layers.83.mlp.experts.57.up_proj.weight": "model-00091-of-00101.safetensors", + "model.layers.83.mlp.experts.57.down_proj.weight": "model-00091-of-00101.safetensors", + "model.layers.83.mlp.experts.58.gate_proj.weight": "model-00091-of-00101.safetensors", + "model.layers.83.mlp.experts.58.up_proj.weight": "model-00091-of-00101.safetensors", + "model.layers.83.mlp.experts.58.down_proj.weight": "model-00091-of-00101.safetensors", + "model.layers.83.mlp.experts.59.gate_proj.weight": "model-00091-of-00101.safetensors", + "model.layers.83.mlp.experts.59.up_proj.weight": "model-00091-of-00101.safetensors", + "model.layers.83.mlp.experts.59.down_proj.weight": "model-00091-of-00101.safetensors", + "model.layers.83.mlp.experts.60.gate_proj.weight": "model-00091-of-00101.safetensors", + "model.layers.83.mlp.experts.60.up_proj.weight": "model-00091-of-00101.safetensors", + "model.layers.83.mlp.experts.60.down_proj.weight": "model-00091-of-00101.safetensors", + "model.layers.83.mlp.experts.61.gate_proj.weight": "model-00091-of-00101.safetensors", + "model.layers.83.mlp.experts.61.up_proj.weight": "model-00091-of-00101.safetensors", + "model.layers.83.mlp.experts.61.down_proj.weight": "model-00091-of-00101.safetensors", + "model.layers.83.mlp.experts.62.gate_proj.weight": "model-00091-of-00101.safetensors", + "model.layers.83.mlp.experts.62.up_proj.weight": "model-00091-of-00101.safetensors", + "model.layers.83.mlp.experts.62.down_proj.weight": "model-00091-of-00101.safetensors", + "model.layers.83.mlp.experts.63.gate_proj.weight": "model-00091-of-00101.safetensors", + "model.layers.83.mlp.experts.63.up_proj.weight": "model-00091-of-00101.safetensors", + "model.layers.83.mlp.experts.63.down_proj.weight": "model-00091-of-00101.safetensors", + "model.layers.83.mlp.experts.64.gate_proj.weight": "model-00091-of-00101.safetensors", + "model.layers.83.mlp.experts.64.up_proj.weight": "model-00091-of-00101.safetensors", + "model.layers.83.mlp.experts.64.down_proj.weight": "model-00091-of-00101.safetensors", + "model.layers.83.mlp.experts.65.gate_proj.weight": "model-00091-of-00101.safetensors", + "model.layers.83.mlp.experts.65.up_proj.weight": "model-00091-of-00101.safetensors", + "model.layers.83.mlp.experts.65.down_proj.weight": "model-00091-of-00101.safetensors", + "model.layers.83.mlp.experts.66.gate_proj.weight": "model-00091-of-00101.safetensors", + "model.layers.83.mlp.experts.66.up_proj.weight": "model-00091-of-00101.safetensors", + "model.layers.83.mlp.experts.66.down_proj.weight": "model-00091-of-00101.safetensors", + "model.layers.83.mlp.experts.67.gate_proj.weight": "model-00091-of-00101.safetensors", + "model.layers.83.mlp.experts.67.up_proj.weight": "model-00091-of-00101.safetensors", + "model.layers.83.mlp.experts.67.down_proj.weight": "model-00091-of-00101.safetensors", + "model.layers.83.mlp.experts.68.gate_proj.weight": "model-00091-of-00101.safetensors", + "model.layers.83.mlp.experts.68.up_proj.weight": "model-00091-of-00101.safetensors", + "model.layers.83.mlp.experts.68.down_proj.weight": "model-00091-of-00101.safetensors", + "model.layers.83.mlp.experts.69.gate_proj.weight": "model-00091-of-00101.safetensors", + "model.layers.83.mlp.experts.69.up_proj.weight": "model-00091-of-00101.safetensors", + "model.layers.83.mlp.experts.69.down_proj.weight": "model-00091-of-00101.safetensors", + "model.layers.83.mlp.experts.70.gate_proj.weight": "model-00091-of-00101.safetensors", + "model.layers.83.mlp.experts.70.up_proj.weight": "model-00091-of-00101.safetensors", + "model.layers.83.mlp.experts.70.down_proj.weight": "model-00091-of-00101.safetensors", + "model.layers.83.mlp.experts.71.gate_proj.weight": "model-00091-of-00101.safetensors", + "model.layers.83.mlp.experts.71.up_proj.weight": "model-00091-of-00101.safetensors", + "model.layers.83.mlp.experts.71.down_proj.weight": "model-00091-of-00101.safetensors", + "model.layers.83.mlp.experts.72.gate_proj.weight": "model-00091-of-00101.safetensors", + "model.layers.83.mlp.experts.72.up_proj.weight": "model-00091-of-00101.safetensors", + "model.layers.83.mlp.experts.72.down_proj.weight": "model-00091-of-00101.safetensors", + "model.layers.83.mlp.experts.73.gate_proj.weight": "model-00091-of-00101.safetensors", + "model.layers.83.mlp.experts.73.up_proj.weight": "model-00091-of-00101.safetensors", + "model.layers.83.mlp.experts.73.down_proj.weight": "model-00091-of-00101.safetensors", + "model.layers.83.mlp.experts.74.gate_proj.weight": "model-00091-of-00101.safetensors", + "model.layers.83.mlp.experts.74.up_proj.weight": "model-00091-of-00101.safetensors", + "model.layers.83.mlp.experts.74.down_proj.weight": "model-00091-of-00101.safetensors", + "model.layers.83.mlp.experts.75.gate_proj.weight": "model-00091-of-00101.safetensors", + "model.layers.83.mlp.experts.75.up_proj.weight": "model-00091-of-00101.safetensors", + "model.layers.83.mlp.experts.75.down_proj.weight": "model-00091-of-00101.safetensors", + "model.layers.83.mlp.experts.76.gate_proj.weight": "model-00091-of-00101.safetensors", + "model.layers.83.mlp.experts.76.up_proj.weight": "model-00091-of-00101.safetensors", + "model.layers.83.mlp.experts.76.down_proj.weight": "model-00091-of-00101.safetensors", + "model.layers.83.mlp.experts.77.gate_proj.weight": "model-00091-of-00101.safetensors", + "model.layers.83.mlp.experts.77.up_proj.weight": "model-00091-of-00101.safetensors", + "model.layers.83.mlp.experts.77.down_proj.weight": "model-00091-of-00101.safetensors", + "model.layers.83.mlp.experts.78.gate_proj.weight": "model-00091-of-00101.safetensors", + "model.layers.83.mlp.experts.78.up_proj.weight": "model-00091-of-00101.safetensors", + "model.layers.83.mlp.experts.78.down_proj.weight": "model-00091-of-00101.safetensors", + "model.layers.83.mlp.experts.79.gate_proj.weight": "model-00091-of-00101.safetensors", + "model.layers.83.mlp.experts.79.up_proj.weight": "model-00091-of-00101.safetensors", + "model.layers.83.mlp.experts.79.down_proj.weight": "model-00091-of-00101.safetensors", + "model.layers.83.mlp.experts.80.gate_proj.weight": "model-00091-of-00101.safetensors", + "model.layers.83.mlp.experts.80.up_proj.weight": "model-00091-of-00101.safetensors", + "model.layers.83.mlp.experts.80.down_proj.weight": "model-00091-of-00101.safetensors", + "model.layers.83.mlp.experts.81.gate_proj.weight": "model-00091-of-00101.safetensors", + "model.layers.83.mlp.experts.81.up_proj.weight": "model-00091-of-00101.safetensors", + "model.layers.83.mlp.experts.81.down_proj.weight": "model-00091-of-00101.safetensors", + "model.layers.83.mlp.experts.82.gate_proj.weight": "model-00091-of-00101.safetensors", + "model.layers.83.mlp.experts.82.up_proj.weight": "model-00091-of-00101.safetensors", + "model.layers.83.mlp.experts.82.down_proj.weight": "model-00091-of-00101.safetensors", + "model.layers.83.mlp.experts.83.gate_proj.weight": "model-00091-of-00101.safetensors", + "model.layers.83.mlp.experts.83.up_proj.weight": "model-00091-of-00101.safetensors", + "model.layers.83.mlp.experts.83.down_proj.weight": "model-00091-of-00101.safetensors", + "model.layers.83.mlp.experts.84.gate_proj.weight": "model-00091-of-00101.safetensors", + "model.layers.83.mlp.experts.84.up_proj.weight": "model-00091-of-00101.safetensors", + "model.layers.83.mlp.experts.84.down_proj.weight": "model-00091-of-00101.safetensors", + "model.layers.83.mlp.experts.85.gate_proj.weight": "model-00091-of-00101.safetensors", + "model.layers.83.mlp.experts.85.up_proj.weight": "model-00091-of-00101.safetensors", + "model.layers.83.mlp.experts.85.down_proj.weight": "model-00091-of-00101.safetensors", + "model.layers.83.mlp.experts.86.gate_proj.weight": "model-00091-of-00101.safetensors", + "model.layers.83.mlp.experts.86.up_proj.weight": "model-00091-of-00101.safetensors", + "model.layers.83.mlp.experts.86.down_proj.weight": "model-00091-of-00101.safetensors", + "model.layers.83.mlp.experts.87.gate_proj.weight": "model-00091-of-00101.safetensors", + "model.layers.83.mlp.experts.87.up_proj.weight": "model-00091-of-00101.safetensors", + "model.layers.83.mlp.experts.87.down_proj.weight": "model-00091-of-00101.safetensors", + "model.layers.83.mlp.experts.88.gate_proj.weight": "model-00091-of-00101.safetensors", + "model.layers.83.mlp.experts.88.up_proj.weight": "model-00091-of-00101.safetensors", + "model.layers.83.mlp.experts.88.down_proj.weight": "model-00091-of-00101.safetensors", + "model.layers.83.mlp.experts.89.gate_proj.weight": "model-00091-of-00101.safetensors", + "model.layers.83.mlp.experts.89.up_proj.weight": "model-00091-of-00101.safetensors", + "model.layers.83.mlp.experts.89.down_proj.weight": "model-00091-of-00101.safetensors", + "model.layers.83.mlp.experts.90.gate_proj.weight": "model-00091-of-00101.safetensors", + "model.layers.83.mlp.experts.90.up_proj.weight": "model-00091-of-00101.safetensors", + "model.layers.83.mlp.experts.90.down_proj.weight": "model-00091-of-00101.safetensors", + "model.layers.83.mlp.experts.91.gate_proj.weight": "model-00091-of-00101.safetensors", + "model.layers.83.mlp.experts.91.up_proj.weight": "model-00091-of-00101.safetensors", + "model.layers.83.mlp.experts.91.down_proj.weight": "model-00091-of-00101.safetensors", + "model.layers.83.mlp.experts.92.gate_proj.weight": "model-00091-of-00101.safetensors", + "model.layers.83.mlp.experts.92.up_proj.weight": "model-00091-of-00101.safetensors", + "model.layers.83.mlp.experts.92.down_proj.weight": "model-00091-of-00101.safetensors", + "model.layers.83.mlp.experts.93.gate_proj.weight": "model-00091-of-00101.safetensors", + "model.layers.83.mlp.experts.93.up_proj.weight": "model-00091-of-00101.safetensors", + "model.layers.83.mlp.experts.93.down_proj.weight": "model-00091-of-00101.safetensors", + "model.layers.83.mlp.experts.94.gate_proj.weight": "model-00091-of-00101.safetensors", + "model.layers.83.mlp.experts.94.up_proj.weight": "model-00091-of-00101.safetensors", + "model.layers.83.mlp.experts.94.down_proj.weight": "model-00091-of-00101.safetensors", + "model.layers.83.mlp.experts.95.gate_proj.weight": "model-00091-of-00101.safetensors", + "model.layers.83.mlp.experts.95.up_proj.weight": "model-00091-of-00101.safetensors", + "model.layers.83.mlp.experts.95.down_proj.weight": "model-00091-of-00101.safetensors", + "model.layers.83.mlp.experts.96.gate_proj.weight": "model-00091-of-00101.safetensors", + "model.layers.83.mlp.experts.96.up_proj.weight": "model-00091-of-00101.safetensors", + "model.layers.83.mlp.experts.96.down_proj.weight": "model-00091-of-00101.safetensors", + "model.layers.83.mlp.experts.97.gate_proj.weight": "model-00091-of-00101.safetensors", + "model.layers.83.mlp.experts.97.up_proj.weight": "model-00091-of-00101.safetensors", + "model.layers.83.mlp.experts.97.down_proj.weight": "model-00091-of-00101.safetensors", + "model.layers.83.mlp.experts.98.gate_proj.weight": "model-00091-of-00101.safetensors", + "model.layers.83.mlp.experts.98.up_proj.weight": "model-00091-of-00101.safetensors", + "model.layers.83.mlp.experts.98.down_proj.weight": "model-00091-of-00101.safetensors", + "model.layers.83.mlp.experts.99.gate_proj.weight": "model-00091-of-00101.safetensors", + "model.layers.83.mlp.experts.99.up_proj.weight": "model-00091-of-00101.safetensors", + "model.layers.83.mlp.experts.99.down_proj.weight": "model-00091-of-00101.safetensors", + "model.layers.83.mlp.experts.100.gate_proj.weight": "model-00091-of-00101.safetensors", + "model.layers.83.mlp.experts.100.up_proj.weight": "model-00091-of-00101.safetensors", + "model.layers.83.mlp.experts.100.down_proj.weight": "model-00091-of-00101.safetensors", + "model.layers.83.mlp.experts.101.gate_proj.weight": "model-00091-of-00101.safetensors", + "model.layers.83.mlp.experts.101.up_proj.weight": "model-00092-of-00101.safetensors", + "model.layers.83.mlp.experts.101.down_proj.weight": "model-00092-of-00101.safetensors", + "model.layers.83.mlp.experts.102.gate_proj.weight": "model-00092-of-00101.safetensors", + "model.layers.83.mlp.experts.102.up_proj.weight": "model-00092-of-00101.safetensors", + "model.layers.83.mlp.experts.102.down_proj.weight": "model-00092-of-00101.safetensors", + "model.layers.83.mlp.experts.103.gate_proj.weight": "model-00092-of-00101.safetensors", + "model.layers.83.mlp.experts.103.up_proj.weight": "model-00092-of-00101.safetensors", + "model.layers.83.mlp.experts.103.down_proj.weight": "model-00092-of-00101.safetensors", + "model.layers.83.mlp.experts.104.gate_proj.weight": "model-00092-of-00101.safetensors", + "model.layers.83.mlp.experts.104.up_proj.weight": "model-00092-of-00101.safetensors", + "model.layers.83.mlp.experts.104.down_proj.weight": "model-00092-of-00101.safetensors", + "model.layers.83.mlp.experts.105.gate_proj.weight": "model-00092-of-00101.safetensors", + "model.layers.83.mlp.experts.105.up_proj.weight": "model-00092-of-00101.safetensors", + "model.layers.83.mlp.experts.105.down_proj.weight": "model-00092-of-00101.safetensors", + "model.layers.83.mlp.experts.106.gate_proj.weight": "model-00092-of-00101.safetensors", + "model.layers.83.mlp.experts.106.up_proj.weight": "model-00092-of-00101.safetensors", + "model.layers.83.mlp.experts.106.down_proj.weight": "model-00092-of-00101.safetensors", + "model.layers.83.mlp.experts.107.gate_proj.weight": "model-00092-of-00101.safetensors", + "model.layers.83.mlp.experts.107.up_proj.weight": "model-00092-of-00101.safetensors", + "model.layers.83.mlp.experts.107.down_proj.weight": "model-00092-of-00101.safetensors", + "model.layers.83.mlp.experts.108.gate_proj.weight": "model-00092-of-00101.safetensors", + "model.layers.83.mlp.experts.108.up_proj.weight": "model-00092-of-00101.safetensors", + "model.layers.83.mlp.experts.108.down_proj.weight": "model-00092-of-00101.safetensors", + "model.layers.83.mlp.experts.109.gate_proj.weight": "model-00092-of-00101.safetensors", + "model.layers.83.mlp.experts.109.up_proj.weight": "model-00092-of-00101.safetensors", + "model.layers.83.mlp.experts.109.down_proj.weight": "model-00092-of-00101.safetensors", + "model.layers.83.mlp.experts.110.gate_proj.weight": "model-00092-of-00101.safetensors", + "model.layers.83.mlp.experts.110.up_proj.weight": "model-00092-of-00101.safetensors", + "model.layers.83.mlp.experts.110.down_proj.weight": "model-00092-of-00101.safetensors", + "model.layers.83.mlp.experts.111.gate_proj.weight": "model-00092-of-00101.safetensors", + "model.layers.83.mlp.experts.111.up_proj.weight": "model-00092-of-00101.safetensors", + "model.layers.83.mlp.experts.111.down_proj.weight": "model-00092-of-00101.safetensors", + "model.layers.83.mlp.experts.112.gate_proj.weight": "model-00092-of-00101.safetensors", + "model.layers.83.mlp.experts.112.up_proj.weight": "model-00092-of-00101.safetensors", + "model.layers.83.mlp.experts.112.down_proj.weight": "model-00092-of-00101.safetensors", + "model.layers.83.mlp.experts.113.gate_proj.weight": "model-00092-of-00101.safetensors", + "model.layers.83.mlp.experts.113.up_proj.weight": "model-00092-of-00101.safetensors", + "model.layers.83.mlp.experts.113.down_proj.weight": "model-00092-of-00101.safetensors", + "model.layers.83.mlp.experts.114.gate_proj.weight": "model-00092-of-00101.safetensors", + "model.layers.83.mlp.experts.114.up_proj.weight": "model-00092-of-00101.safetensors", + "model.layers.83.mlp.experts.114.down_proj.weight": "model-00092-of-00101.safetensors", + "model.layers.83.mlp.experts.115.gate_proj.weight": "model-00092-of-00101.safetensors", + "model.layers.83.mlp.experts.115.up_proj.weight": "model-00092-of-00101.safetensors", + "model.layers.83.mlp.experts.115.down_proj.weight": "model-00092-of-00101.safetensors", + "model.layers.83.mlp.experts.116.gate_proj.weight": "model-00092-of-00101.safetensors", + "model.layers.83.mlp.experts.116.up_proj.weight": "model-00092-of-00101.safetensors", + "model.layers.83.mlp.experts.116.down_proj.weight": "model-00092-of-00101.safetensors", + "model.layers.83.mlp.experts.117.gate_proj.weight": "model-00092-of-00101.safetensors", + "model.layers.83.mlp.experts.117.up_proj.weight": "model-00092-of-00101.safetensors", + "model.layers.83.mlp.experts.117.down_proj.weight": "model-00092-of-00101.safetensors", + "model.layers.83.mlp.experts.118.gate_proj.weight": "model-00092-of-00101.safetensors", + "model.layers.83.mlp.experts.118.up_proj.weight": "model-00092-of-00101.safetensors", + "model.layers.83.mlp.experts.118.down_proj.weight": "model-00092-of-00101.safetensors", + "model.layers.83.mlp.experts.119.gate_proj.weight": "model-00092-of-00101.safetensors", + "model.layers.83.mlp.experts.119.up_proj.weight": "model-00092-of-00101.safetensors", + "model.layers.83.mlp.experts.119.down_proj.weight": "model-00092-of-00101.safetensors", + "model.layers.83.mlp.gate.weight": "model-00092-of-00101.safetensors", + "model.layers.83.mlp.gate.e_score_correction_bias": "model-00092-of-00101.safetensors", + "model.layers.83.mlp.shared_experts.gate_proj.weight": "model-00092-of-00101.safetensors", + "model.layers.83.mlp.shared_experts.up_proj.weight": "model-00092-of-00101.safetensors", + "model.layers.83.mlp.shared_experts.down_proj.weight": "model-00092-of-00101.safetensors", + "model.layers.83.input_layernorm.weight": "model-00092-of-00101.safetensors", + "model.layers.83.post_attention_layernorm.weight": "model-00092-of-00101.safetensors", + "model.layers.84.self_attn.q_proj.weight": "model-00092-of-00101.safetensors", + "model.layers.84.self_attn.q_proj.bias": "model-00092-of-00101.safetensors", + "model.layers.84.self_attn.k_proj.weight": "model-00092-of-00101.safetensors", + "model.layers.84.self_attn.k_proj.bias": "model-00092-of-00101.safetensors", + "model.layers.84.self_attn.v_proj.weight": "model-00092-of-00101.safetensors", + "model.layers.84.self_attn.v_proj.bias": "model-00092-of-00101.safetensors", + "model.layers.84.self_attn.o_proj.weight": "model-00092-of-00101.safetensors", + "model.layers.84.self_attn.q_norm.weight": "model-00092-of-00101.safetensors", + "model.layers.84.self_attn.k_norm.weight": "model-00092-of-00101.safetensors", + "model.layers.84.mlp.experts.0.gate_proj.weight": "model-00092-of-00101.safetensors", + "model.layers.84.mlp.experts.0.up_proj.weight": "model-00092-of-00101.safetensors", + "model.layers.84.mlp.experts.0.down_proj.weight": "model-00092-of-00101.safetensors", + "model.layers.84.mlp.experts.1.gate_proj.weight": "model-00092-of-00101.safetensors", + "model.layers.84.mlp.experts.1.up_proj.weight": "model-00092-of-00101.safetensors", + "model.layers.84.mlp.experts.1.down_proj.weight": "model-00092-of-00101.safetensors", + "model.layers.84.mlp.experts.2.gate_proj.weight": "model-00092-of-00101.safetensors", + "model.layers.84.mlp.experts.2.up_proj.weight": "model-00092-of-00101.safetensors", + "model.layers.84.mlp.experts.2.down_proj.weight": "model-00092-of-00101.safetensors", + "model.layers.84.mlp.experts.3.gate_proj.weight": "model-00092-of-00101.safetensors", + "model.layers.84.mlp.experts.3.up_proj.weight": "model-00092-of-00101.safetensors", + "model.layers.84.mlp.experts.3.down_proj.weight": "model-00092-of-00101.safetensors", + "model.layers.84.mlp.experts.4.gate_proj.weight": "model-00092-of-00101.safetensors", + "model.layers.84.mlp.experts.4.up_proj.weight": "model-00092-of-00101.safetensors", + "model.layers.84.mlp.experts.4.down_proj.weight": "model-00092-of-00101.safetensors", + "model.layers.84.mlp.experts.5.gate_proj.weight": "model-00092-of-00101.safetensors", + "model.layers.84.mlp.experts.5.up_proj.weight": "model-00092-of-00101.safetensors", + "model.layers.84.mlp.experts.5.down_proj.weight": "model-00092-of-00101.safetensors", + "model.layers.84.mlp.experts.6.gate_proj.weight": "model-00092-of-00101.safetensors", + "model.layers.84.mlp.experts.6.up_proj.weight": "model-00092-of-00101.safetensors", + "model.layers.84.mlp.experts.6.down_proj.weight": "model-00092-of-00101.safetensors", + "model.layers.84.mlp.experts.7.gate_proj.weight": "model-00092-of-00101.safetensors", + "model.layers.84.mlp.experts.7.up_proj.weight": "model-00092-of-00101.safetensors", + "model.layers.84.mlp.experts.7.down_proj.weight": "model-00092-of-00101.safetensors", + "model.layers.84.mlp.experts.8.gate_proj.weight": "model-00092-of-00101.safetensors", + "model.layers.84.mlp.experts.8.up_proj.weight": "model-00092-of-00101.safetensors", + "model.layers.84.mlp.experts.8.down_proj.weight": "model-00092-of-00101.safetensors", + "model.layers.84.mlp.experts.9.gate_proj.weight": "model-00092-of-00101.safetensors", + "model.layers.84.mlp.experts.9.up_proj.weight": "model-00092-of-00101.safetensors", + "model.layers.84.mlp.experts.9.down_proj.weight": "model-00092-of-00101.safetensors", + "model.layers.84.mlp.experts.10.gate_proj.weight": "model-00092-of-00101.safetensors", + "model.layers.84.mlp.experts.10.up_proj.weight": "model-00092-of-00101.safetensors", + "model.layers.84.mlp.experts.10.down_proj.weight": "model-00092-of-00101.safetensors", + "model.layers.84.mlp.experts.11.gate_proj.weight": "model-00092-of-00101.safetensors", + "model.layers.84.mlp.experts.11.up_proj.weight": "model-00092-of-00101.safetensors", + "model.layers.84.mlp.experts.11.down_proj.weight": "model-00092-of-00101.safetensors", + "model.layers.84.mlp.experts.12.gate_proj.weight": "model-00092-of-00101.safetensors", + "model.layers.84.mlp.experts.12.up_proj.weight": "model-00092-of-00101.safetensors", + "model.layers.84.mlp.experts.12.down_proj.weight": "model-00092-of-00101.safetensors", + "model.layers.84.mlp.experts.13.gate_proj.weight": "model-00092-of-00101.safetensors", + "model.layers.84.mlp.experts.13.up_proj.weight": "model-00092-of-00101.safetensors", + "model.layers.84.mlp.experts.13.down_proj.weight": "model-00092-of-00101.safetensors", + "model.layers.84.mlp.experts.14.gate_proj.weight": "model-00092-of-00101.safetensors", + "model.layers.84.mlp.experts.14.up_proj.weight": "model-00092-of-00101.safetensors", + "model.layers.84.mlp.experts.14.down_proj.weight": "model-00092-of-00101.safetensors", + "model.layers.84.mlp.experts.15.gate_proj.weight": "model-00092-of-00101.safetensors", + "model.layers.84.mlp.experts.15.up_proj.weight": "model-00092-of-00101.safetensors", + "model.layers.84.mlp.experts.15.down_proj.weight": "model-00092-of-00101.safetensors", + "model.layers.84.mlp.experts.16.gate_proj.weight": "model-00092-of-00101.safetensors", + "model.layers.84.mlp.experts.16.up_proj.weight": "model-00092-of-00101.safetensors", + "model.layers.84.mlp.experts.16.down_proj.weight": "model-00092-of-00101.safetensors", + "model.layers.84.mlp.experts.17.gate_proj.weight": "model-00092-of-00101.safetensors", + "model.layers.84.mlp.experts.17.up_proj.weight": "model-00092-of-00101.safetensors", + "model.layers.84.mlp.experts.17.down_proj.weight": "model-00092-of-00101.safetensors", + "model.layers.84.mlp.experts.18.gate_proj.weight": "model-00092-of-00101.safetensors", + "model.layers.84.mlp.experts.18.up_proj.weight": "model-00092-of-00101.safetensors", + "model.layers.84.mlp.experts.18.down_proj.weight": "model-00092-of-00101.safetensors", + "model.layers.84.mlp.experts.19.gate_proj.weight": "model-00092-of-00101.safetensors", + "model.layers.84.mlp.experts.19.up_proj.weight": "model-00092-of-00101.safetensors", + "model.layers.84.mlp.experts.19.down_proj.weight": "model-00092-of-00101.safetensors", + "model.layers.84.mlp.experts.20.gate_proj.weight": "model-00092-of-00101.safetensors", + "model.layers.84.mlp.experts.20.up_proj.weight": "model-00092-of-00101.safetensors", + "model.layers.84.mlp.experts.20.down_proj.weight": "model-00092-of-00101.safetensors", + "model.layers.84.mlp.experts.21.gate_proj.weight": "model-00092-of-00101.safetensors", + "model.layers.84.mlp.experts.21.up_proj.weight": "model-00092-of-00101.safetensors", + "model.layers.84.mlp.experts.21.down_proj.weight": "model-00092-of-00101.safetensors", + "model.layers.84.mlp.experts.22.gate_proj.weight": "model-00092-of-00101.safetensors", + "model.layers.84.mlp.experts.22.up_proj.weight": "model-00092-of-00101.safetensors", + "model.layers.84.mlp.experts.22.down_proj.weight": "model-00092-of-00101.safetensors", + "model.layers.84.mlp.experts.23.gate_proj.weight": "model-00092-of-00101.safetensors", + "model.layers.84.mlp.experts.23.up_proj.weight": "model-00092-of-00101.safetensors", + "model.layers.84.mlp.experts.23.down_proj.weight": "model-00092-of-00101.safetensors", + "model.layers.84.mlp.experts.24.gate_proj.weight": "model-00092-of-00101.safetensors", + "model.layers.84.mlp.experts.24.up_proj.weight": "model-00092-of-00101.safetensors", + "model.layers.84.mlp.experts.24.down_proj.weight": "model-00092-of-00101.safetensors", + "model.layers.84.mlp.experts.25.gate_proj.weight": "model-00092-of-00101.safetensors", + "model.layers.84.mlp.experts.25.up_proj.weight": "model-00092-of-00101.safetensors", + "model.layers.84.mlp.experts.25.down_proj.weight": "model-00092-of-00101.safetensors", + "model.layers.84.mlp.experts.26.gate_proj.weight": "model-00092-of-00101.safetensors", + "model.layers.84.mlp.experts.26.up_proj.weight": "model-00092-of-00101.safetensors", + "model.layers.84.mlp.experts.26.down_proj.weight": "model-00092-of-00101.safetensors", + "model.layers.84.mlp.experts.27.gate_proj.weight": "model-00092-of-00101.safetensors", + "model.layers.84.mlp.experts.27.up_proj.weight": "model-00092-of-00101.safetensors", + "model.layers.84.mlp.experts.27.down_proj.weight": "model-00092-of-00101.safetensors", + "model.layers.84.mlp.experts.28.gate_proj.weight": "model-00092-of-00101.safetensors", + "model.layers.84.mlp.experts.28.up_proj.weight": "model-00092-of-00101.safetensors", + "model.layers.84.mlp.experts.28.down_proj.weight": "model-00092-of-00101.safetensors", + "model.layers.84.mlp.experts.29.gate_proj.weight": "model-00092-of-00101.safetensors", + "model.layers.84.mlp.experts.29.up_proj.weight": "model-00092-of-00101.safetensors", + "model.layers.84.mlp.experts.29.down_proj.weight": "model-00092-of-00101.safetensors", + "model.layers.84.mlp.experts.30.gate_proj.weight": "model-00092-of-00101.safetensors", + "model.layers.84.mlp.experts.30.up_proj.weight": "model-00092-of-00101.safetensors", + "model.layers.84.mlp.experts.30.down_proj.weight": "model-00092-of-00101.safetensors", + "model.layers.84.mlp.experts.31.gate_proj.weight": "model-00092-of-00101.safetensors", + "model.layers.84.mlp.experts.31.up_proj.weight": "model-00092-of-00101.safetensors", + "model.layers.84.mlp.experts.31.down_proj.weight": "model-00092-of-00101.safetensors", + "model.layers.84.mlp.experts.32.gate_proj.weight": "model-00092-of-00101.safetensors", + "model.layers.84.mlp.experts.32.up_proj.weight": "model-00092-of-00101.safetensors", + "model.layers.84.mlp.experts.32.down_proj.weight": "model-00092-of-00101.safetensors", + "model.layers.84.mlp.experts.33.gate_proj.weight": "model-00092-of-00101.safetensors", + "model.layers.84.mlp.experts.33.up_proj.weight": "model-00092-of-00101.safetensors", + "model.layers.84.mlp.experts.33.down_proj.weight": "model-00092-of-00101.safetensors", + "model.layers.84.mlp.experts.34.gate_proj.weight": "model-00092-of-00101.safetensors", + "model.layers.84.mlp.experts.34.up_proj.weight": "model-00092-of-00101.safetensors", + "model.layers.84.mlp.experts.34.down_proj.weight": "model-00092-of-00101.safetensors", + "model.layers.84.mlp.experts.35.gate_proj.weight": "model-00092-of-00101.safetensors", + "model.layers.84.mlp.experts.35.up_proj.weight": "model-00092-of-00101.safetensors", + "model.layers.84.mlp.experts.35.down_proj.weight": "model-00092-of-00101.safetensors", + "model.layers.84.mlp.experts.36.gate_proj.weight": "model-00092-of-00101.safetensors", + "model.layers.84.mlp.experts.36.up_proj.weight": "model-00092-of-00101.safetensors", + "model.layers.84.mlp.experts.36.down_proj.weight": "model-00092-of-00101.safetensors", + "model.layers.84.mlp.experts.37.gate_proj.weight": "model-00092-of-00101.safetensors", + "model.layers.84.mlp.experts.37.up_proj.weight": "model-00092-of-00101.safetensors", + "model.layers.84.mlp.experts.37.down_proj.weight": "model-00092-of-00101.safetensors", + "model.layers.84.mlp.experts.38.gate_proj.weight": "model-00092-of-00101.safetensors", + "model.layers.84.mlp.experts.38.up_proj.weight": "model-00092-of-00101.safetensors", + "model.layers.84.mlp.experts.38.down_proj.weight": "model-00092-of-00101.safetensors", + "model.layers.84.mlp.experts.39.gate_proj.weight": "model-00092-of-00101.safetensors", + "model.layers.84.mlp.experts.39.up_proj.weight": "model-00092-of-00101.safetensors", + "model.layers.84.mlp.experts.39.down_proj.weight": "model-00092-of-00101.safetensors", + "model.layers.84.mlp.experts.40.gate_proj.weight": "model-00092-of-00101.safetensors", + "model.layers.84.mlp.experts.40.up_proj.weight": "model-00092-of-00101.safetensors", + "model.layers.84.mlp.experts.40.down_proj.weight": "model-00092-of-00101.safetensors", + "model.layers.84.mlp.experts.41.gate_proj.weight": "model-00092-of-00101.safetensors", + "model.layers.84.mlp.experts.41.up_proj.weight": "model-00092-of-00101.safetensors", + "model.layers.84.mlp.experts.41.down_proj.weight": "model-00092-of-00101.safetensors", + "model.layers.84.mlp.experts.42.gate_proj.weight": "model-00092-of-00101.safetensors", + "model.layers.84.mlp.experts.42.up_proj.weight": "model-00092-of-00101.safetensors", + "model.layers.84.mlp.experts.42.down_proj.weight": "model-00092-of-00101.safetensors", + "model.layers.84.mlp.experts.43.gate_proj.weight": "model-00092-of-00101.safetensors", + "model.layers.84.mlp.experts.43.up_proj.weight": "model-00092-of-00101.safetensors", + "model.layers.84.mlp.experts.43.down_proj.weight": "model-00092-of-00101.safetensors", + "model.layers.84.mlp.experts.44.gate_proj.weight": "model-00092-of-00101.safetensors", + "model.layers.84.mlp.experts.44.up_proj.weight": "model-00092-of-00101.safetensors", + "model.layers.84.mlp.experts.44.down_proj.weight": "model-00092-of-00101.safetensors", + "model.layers.84.mlp.experts.45.gate_proj.weight": "model-00092-of-00101.safetensors", + "model.layers.84.mlp.experts.45.up_proj.weight": "model-00092-of-00101.safetensors", + "model.layers.84.mlp.experts.45.down_proj.weight": "model-00092-of-00101.safetensors", + "model.layers.84.mlp.experts.46.gate_proj.weight": "model-00092-of-00101.safetensors", + "model.layers.84.mlp.experts.46.up_proj.weight": "model-00092-of-00101.safetensors", + "model.layers.84.mlp.experts.46.down_proj.weight": "model-00092-of-00101.safetensors", + "model.layers.84.mlp.experts.47.gate_proj.weight": "model-00092-of-00101.safetensors", + "model.layers.84.mlp.experts.47.up_proj.weight": "model-00092-of-00101.safetensors", + "model.layers.84.mlp.experts.47.down_proj.weight": "model-00092-of-00101.safetensors", + "model.layers.84.mlp.experts.48.gate_proj.weight": "model-00092-of-00101.safetensors", + "model.layers.84.mlp.experts.48.up_proj.weight": "model-00092-of-00101.safetensors", + "model.layers.84.mlp.experts.48.down_proj.weight": "model-00092-of-00101.safetensors", + "model.layers.84.mlp.experts.49.gate_proj.weight": "model-00092-of-00101.safetensors", + "model.layers.84.mlp.experts.49.up_proj.weight": "model-00092-of-00101.safetensors", + "model.layers.84.mlp.experts.49.down_proj.weight": "model-00092-of-00101.safetensors", + "model.layers.84.mlp.experts.50.gate_proj.weight": "model-00092-of-00101.safetensors", + "model.layers.84.mlp.experts.50.up_proj.weight": "model-00092-of-00101.safetensors", + "model.layers.84.mlp.experts.50.down_proj.weight": "model-00092-of-00101.safetensors", + "model.layers.84.mlp.experts.51.gate_proj.weight": "model-00092-of-00101.safetensors", + "model.layers.84.mlp.experts.51.up_proj.weight": "model-00092-of-00101.safetensors", + "model.layers.84.mlp.experts.51.down_proj.weight": "model-00092-of-00101.safetensors", + "model.layers.84.mlp.experts.52.gate_proj.weight": "model-00092-of-00101.safetensors", + "model.layers.84.mlp.experts.52.up_proj.weight": "model-00092-of-00101.safetensors", + "model.layers.84.mlp.experts.52.down_proj.weight": "model-00092-of-00101.safetensors", + "model.layers.84.mlp.experts.53.gate_proj.weight": "model-00092-of-00101.safetensors", + "model.layers.84.mlp.experts.53.up_proj.weight": "model-00092-of-00101.safetensors", + "model.layers.84.mlp.experts.53.down_proj.weight": "model-00092-of-00101.safetensors", + "model.layers.84.mlp.experts.54.gate_proj.weight": "model-00092-of-00101.safetensors", + "model.layers.84.mlp.experts.54.up_proj.weight": "model-00092-of-00101.safetensors", + "model.layers.84.mlp.experts.54.down_proj.weight": "model-00092-of-00101.safetensors", + "model.layers.84.mlp.experts.55.gate_proj.weight": "model-00092-of-00101.safetensors", + "model.layers.84.mlp.experts.55.up_proj.weight": "model-00092-of-00101.safetensors", + "model.layers.84.mlp.experts.55.down_proj.weight": "model-00092-of-00101.safetensors", + "model.layers.84.mlp.experts.56.gate_proj.weight": "model-00092-of-00101.safetensors", + "model.layers.84.mlp.experts.56.up_proj.weight": "model-00092-of-00101.safetensors", + "model.layers.84.mlp.experts.56.down_proj.weight": "model-00092-of-00101.safetensors", + "model.layers.84.mlp.experts.57.gate_proj.weight": "model-00092-of-00101.safetensors", + "model.layers.84.mlp.experts.57.up_proj.weight": "model-00092-of-00101.safetensors", + "model.layers.84.mlp.experts.57.down_proj.weight": "model-00092-of-00101.safetensors", + "model.layers.84.mlp.experts.58.gate_proj.weight": "model-00092-of-00101.safetensors", + "model.layers.84.mlp.experts.58.up_proj.weight": "model-00092-of-00101.safetensors", + "model.layers.84.mlp.experts.58.down_proj.weight": "model-00092-of-00101.safetensors", + "model.layers.84.mlp.experts.59.gate_proj.weight": "model-00092-of-00101.safetensors", + "model.layers.84.mlp.experts.59.up_proj.weight": "model-00092-of-00101.safetensors", + "model.layers.84.mlp.experts.59.down_proj.weight": "model-00092-of-00101.safetensors", + "model.layers.84.mlp.experts.60.gate_proj.weight": "model-00092-of-00101.safetensors", + "model.layers.84.mlp.experts.60.up_proj.weight": "model-00092-of-00101.safetensors", + "model.layers.84.mlp.experts.60.down_proj.weight": "model-00092-of-00101.safetensors", + "model.layers.84.mlp.experts.61.gate_proj.weight": "model-00092-of-00101.safetensors", + "model.layers.84.mlp.experts.61.up_proj.weight": "model-00092-of-00101.safetensors", + "model.layers.84.mlp.experts.61.down_proj.weight": "model-00092-of-00101.safetensors", + "model.layers.84.mlp.experts.62.gate_proj.weight": "model-00092-of-00101.safetensors", + "model.layers.84.mlp.experts.62.up_proj.weight": "model-00092-of-00101.safetensors", + "model.layers.84.mlp.experts.62.down_proj.weight": "model-00092-of-00101.safetensors", + "model.layers.84.mlp.experts.63.gate_proj.weight": "model-00092-of-00101.safetensors", + "model.layers.84.mlp.experts.63.up_proj.weight": "model-00092-of-00101.safetensors", + "model.layers.84.mlp.experts.63.down_proj.weight": "model-00092-of-00101.safetensors", + "model.layers.84.mlp.experts.64.gate_proj.weight": "model-00092-of-00101.safetensors", + "model.layers.84.mlp.experts.64.up_proj.weight": "model-00092-of-00101.safetensors", + "model.layers.84.mlp.experts.64.down_proj.weight": "model-00092-of-00101.safetensors", + "model.layers.84.mlp.experts.65.gate_proj.weight": "model-00092-of-00101.safetensors", + "model.layers.84.mlp.experts.65.up_proj.weight": "model-00092-of-00101.safetensors", + "model.layers.84.mlp.experts.65.down_proj.weight": "model-00092-of-00101.safetensors", + "model.layers.84.mlp.experts.66.gate_proj.weight": "model-00092-of-00101.safetensors", + "model.layers.84.mlp.experts.66.up_proj.weight": "model-00092-of-00101.safetensors", + "model.layers.84.mlp.experts.66.down_proj.weight": "model-00092-of-00101.safetensors", + "model.layers.84.mlp.experts.67.gate_proj.weight": "model-00092-of-00101.safetensors", + "model.layers.84.mlp.experts.67.up_proj.weight": "model-00092-of-00101.safetensors", + "model.layers.84.mlp.experts.67.down_proj.weight": "model-00092-of-00101.safetensors", + "model.layers.84.mlp.experts.68.gate_proj.weight": "model-00092-of-00101.safetensors", + "model.layers.84.mlp.experts.68.up_proj.weight": "model-00092-of-00101.safetensors", + "model.layers.84.mlp.experts.68.down_proj.weight": "model-00092-of-00101.safetensors", + "model.layers.84.mlp.experts.69.gate_proj.weight": "model-00092-of-00101.safetensors", + "model.layers.84.mlp.experts.69.up_proj.weight": "model-00092-of-00101.safetensors", + "model.layers.84.mlp.experts.69.down_proj.weight": "model-00092-of-00101.safetensors", + "model.layers.84.mlp.experts.70.gate_proj.weight": "model-00092-of-00101.safetensors", + "model.layers.84.mlp.experts.70.up_proj.weight": "model-00092-of-00101.safetensors", + "model.layers.84.mlp.experts.70.down_proj.weight": "model-00092-of-00101.safetensors", + "model.layers.84.mlp.experts.71.gate_proj.weight": "model-00092-of-00101.safetensors", + "model.layers.84.mlp.experts.71.up_proj.weight": "model-00092-of-00101.safetensors", + "model.layers.84.mlp.experts.71.down_proj.weight": "model-00092-of-00101.safetensors", + "model.layers.84.mlp.experts.72.gate_proj.weight": "model-00092-of-00101.safetensors", + "model.layers.84.mlp.experts.72.up_proj.weight": "model-00092-of-00101.safetensors", + "model.layers.84.mlp.experts.72.down_proj.weight": "model-00092-of-00101.safetensors", + "model.layers.84.mlp.experts.73.gate_proj.weight": "model-00092-of-00101.safetensors", + "model.layers.84.mlp.experts.73.up_proj.weight": "model-00092-of-00101.safetensors", + "model.layers.84.mlp.experts.73.down_proj.weight": "model-00092-of-00101.safetensors", + "model.layers.84.mlp.experts.74.gate_proj.weight": "model-00092-of-00101.safetensors", + "model.layers.84.mlp.experts.74.up_proj.weight": "model-00092-of-00101.safetensors", + "model.layers.84.mlp.experts.74.down_proj.weight": "model-00092-of-00101.safetensors", + "model.layers.84.mlp.experts.75.gate_proj.weight": "model-00092-of-00101.safetensors", + "model.layers.84.mlp.experts.75.up_proj.weight": "model-00092-of-00101.safetensors", + "model.layers.84.mlp.experts.75.down_proj.weight": "model-00092-of-00101.safetensors", + "model.layers.84.mlp.experts.76.gate_proj.weight": "model-00092-of-00101.safetensors", + "model.layers.84.mlp.experts.76.up_proj.weight": "model-00092-of-00101.safetensors", + "model.layers.84.mlp.experts.76.down_proj.weight": "model-00092-of-00101.safetensors", + "model.layers.84.mlp.experts.77.gate_proj.weight": "model-00092-of-00101.safetensors", + "model.layers.84.mlp.experts.77.up_proj.weight": "model-00092-of-00101.safetensors", + "model.layers.84.mlp.experts.77.down_proj.weight": "model-00092-of-00101.safetensors", + "model.layers.84.mlp.experts.78.gate_proj.weight": "model-00092-of-00101.safetensors", + "model.layers.84.mlp.experts.78.up_proj.weight": "model-00092-of-00101.safetensors", + "model.layers.84.mlp.experts.78.down_proj.weight": "model-00092-of-00101.safetensors", + "model.layers.84.mlp.experts.79.gate_proj.weight": "model-00092-of-00101.safetensors", + "model.layers.84.mlp.experts.79.up_proj.weight": "model-00092-of-00101.safetensors", + "model.layers.84.mlp.experts.79.down_proj.weight": "model-00092-of-00101.safetensors", + "model.layers.84.mlp.experts.80.gate_proj.weight": "model-00092-of-00101.safetensors", + "model.layers.84.mlp.experts.80.up_proj.weight": "model-00092-of-00101.safetensors", + "model.layers.84.mlp.experts.80.down_proj.weight": "model-00092-of-00101.safetensors", + "model.layers.84.mlp.experts.81.gate_proj.weight": "model-00092-of-00101.safetensors", + "model.layers.84.mlp.experts.81.up_proj.weight": "model-00092-of-00101.safetensors", + "model.layers.84.mlp.experts.81.down_proj.weight": "model-00092-of-00101.safetensors", + "model.layers.84.mlp.experts.82.gate_proj.weight": "model-00092-of-00101.safetensors", + "model.layers.84.mlp.experts.82.up_proj.weight": "model-00092-of-00101.safetensors", + "model.layers.84.mlp.experts.82.down_proj.weight": "model-00092-of-00101.safetensors", + "model.layers.84.mlp.experts.83.gate_proj.weight": "model-00092-of-00101.safetensors", + "model.layers.84.mlp.experts.83.up_proj.weight": "model-00092-of-00101.safetensors", + "model.layers.84.mlp.experts.83.down_proj.weight": "model-00092-of-00101.safetensors", + "model.layers.84.mlp.experts.84.gate_proj.weight": "model-00092-of-00101.safetensors", + "model.layers.84.mlp.experts.84.up_proj.weight": "model-00092-of-00101.safetensors", + "model.layers.84.mlp.experts.84.down_proj.weight": "model-00092-of-00101.safetensors", + "model.layers.84.mlp.experts.85.gate_proj.weight": "model-00092-of-00101.safetensors", + "model.layers.84.mlp.experts.85.up_proj.weight": "model-00092-of-00101.safetensors", + "model.layers.84.mlp.experts.85.down_proj.weight": "model-00092-of-00101.safetensors", + "model.layers.84.mlp.experts.86.gate_proj.weight": "model-00092-of-00101.safetensors", + "model.layers.84.mlp.experts.86.up_proj.weight": "model-00092-of-00101.safetensors", + "model.layers.84.mlp.experts.86.down_proj.weight": "model-00092-of-00101.safetensors", + "model.layers.84.mlp.experts.87.gate_proj.weight": "model-00092-of-00101.safetensors", + "model.layers.84.mlp.experts.87.up_proj.weight": "model-00092-of-00101.safetensors", + "model.layers.84.mlp.experts.87.down_proj.weight": "model-00092-of-00101.safetensors", + "model.layers.84.mlp.experts.88.gate_proj.weight": "model-00093-of-00101.safetensors", + "model.layers.84.mlp.experts.88.up_proj.weight": "model-00093-of-00101.safetensors", + "model.layers.84.mlp.experts.88.down_proj.weight": "model-00093-of-00101.safetensors", + "model.layers.84.mlp.experts.89.gate_proj.weight": "model-00093-of-00101.safetensors", + "model.layers.84.mlp.experts.89.up_proj.weight": "model-00093-of-00101.safetensors", + "model.layers.84.mlp.experts.89.down_proj.weight": "model-00093-of-00101.safetensors", + "model.layers.84.mlp.experts.90.gate_proj.weight": "model-00093-of-00101.safetensors", + "model.layers.84.mlp.experts.90.up_proj.weight": "model-00093-of-00101.safetensors", + "model.layers.84.mlp.experts.90.down_proj.weight": "model-00093-of-00101.safetensors", + "model.layers.84.mlp.experts.91.gate_proj.weight": "model-00093-of-00101.safetensors", + "model.layers.84.mlp.experts.91.up_proj.weight": "model-00093-of-00101.safetensors", + "model.layers.84.mlp.experts.91.down_proj.weight": "model-00093-of-00101.safetensors", + "model.layers.84.mlp.experts.92.gate_proj.weight": "model-00093-of-00101.safetensors", + "model.layers.84.mlp.experts.92.up_proj.weight": "model-00093-of-00101.safetensors", + "model.layers.84.mlp.experts.92.down_proj.weight": "model-00093-of-00101.safetensors", + "model.layers.84.mlp.experts.93.gate_proj.weight": "model-00093-of-00101.safetensors", + "model.layers.84.mlp.experts.93.up_proj.weight": "model-00093-of-00101.safetensors", + "model.layers.84.mlp.experts.93.down_proj.weight": "model-00093-of-00101.safetensors", + "model.layers.84.mlp.experts.94.gate_proj.weight": "model-00093-of-00101.safetensors", + "model.layers.84.mlp.experts.94.up_proj.weight": "model-00093-of-00101.safetensors", + "model.layers.84.mlp.experts.94.down_proj.weight": "model-00093-of-00101.safetensors", + "model.layers.84.mlp.experts.95.gate_proj.weight": "model-00093-of-00101.safetensors", + "model.layers.84.mlp.experts.95.up_proj.weight": "model-00093-of-00101.safetensors", + "model.layers.84.mlp.experts.95.down_proj.weight": "model-00093-of-00101.safetensors", + "model.layers.84.mlp.experts.96.gate_proj.weight": "model-00093-of-00101.safetensors", + "model.layers.84.mlp.experts.96.up_proj.weight": "model-00093-of-00101.safetensors", + "model.layers.84.mlp.experts.96.down_proj.weight": "model-00093-of-00101.safetensors", + "model.layers.84.mlp.experts.97.gate_proj.weight": "model-00093-of-00101.safetensors", + "model.layers.84.mlp.experts.97.up_proj.weight": "model-00093-of-00101.safetensors", + "model.layers.84.mlp.experts.97.down_proj.weight": "model-00093-of-00101.safetensors", + "model.layers.84.mlp.experts.98.gate_proj.weight": "model-00093-of-00101.safetensors", + "model.layers.84.mlp.experts.98.up_proj.weight": "model-00093-of-00101.safetensors", + "model.layers.84.mlp.experts.98.down_proj.weight": "model-00093-of-00101.safetensors", + "model.layers.84.mlp.experts.99.gate_proj.weight": "model-00093-of-00101.safetensors", + "model.layers.84.mlp.experts.99.up_proj.weight": "model-00093-of-00101.safetensors", + "model.layers.84.mlp.experts.99.down_proj.weight": "model-00093-of-00101.safetensors", + "model.layers.84.mlp.experts.100.gate_proj.weight": "model-00093-of-00101.safetensors", + "model.layers.84.mlp.experts.100.up_proj.weight": "model-00093-of-00101.safetensors", + "model.layers.84.mlp.experts.100.down_proj.weight": "model-00093-of-00101.safetensors", + "model.layers.84.mlp.experts.101.gate_proj.weight": "model-00093-of-00101.safetensors", + "model.layers.84.mlp.experts.101.up_proj.weight": "model-00093-of-00101.safetensors", + "model.layers.84.mlp.experts.101.down_proj.weight": "model-00093-of-00101.safetensors", + "model.layers.84.mlp.experts.102.gate_proj.weight": "model-00093-of-00101.safetensors", + "model.layers.84.mlp.experts.102.up_proj.weight": "model-00093-of-00101.safetensors", + "model.layers.84.mlp.experts.102.down_proj.weight": "model-00093-of-00101.safetensors", + "model.layers.84.mlp.experts.103.gate_proj.weight": "model-00093-of-00101.safetensors", + "model.layers.84.mlp.experts.103.up_proj.weight": "model-00093-of-00101.safetensors", + "model.layers.84.mlp.experts.103.down_proj.weight": "model-00093-of-00101.safetensors", + "model.layers.84.mlp.experts.104.gate_proj.weight": "model-00093-of-00101.safetensors", + "model.layers.84.mlp.experts.104.up_proj.weight": "model-00093-of-00101.safetensors", + "model.layers.84.mlp.experts.104.down_proj.weight": "model-00093-of-00101.safetensors", + "model.layers.84.mlp.experts.105.gate_proj.weight": "model-00093-of-00101.safetensors", + "model.layers.84.mlp.experts.105.up_proj.weight": "model-00093-of-00101.safetensors", + "model.layers.84.mlp.experts.105.down_proj.weight": "model-00093-of-00101.safetensors", + "model.layers.84.mlp.experts.106.gate_proj.weight": "model-00093-of-00101.safetensors", + "model.layers.84.mlp.experts.106.up_proj.weight": "model-00093-of-00101.safetensors", + "model.layers.84.mlp.experts.106.down_proj.weight": "model-00093-of-00101.safetensors", + "model.layers.84.mlp.experts.107.gate_proj.weight": "model-00093-of-00101.safetensors", + "model.layers.84.mlp.experts.107.up_proj.weight": "model-00093-of-00101.safetensors", + "model.layers.84.mlp.experts.107.down_proj.weight": "model-00093-of-00101.safetensors", + "model.layers.84.mlp.experts.108.gate_proj.weight": "model-00093-of-00101.safetensors", + "model.layers.84.mlp.experts.108.up_proj.weight": "model-00093-of-00101.safetensors", + "model.layers.84.mlp.experts.108.down_proj.weight": "model-00093-of-00101.safetensors", + "model.layers.84.mlp.experts.109.gate_proj.weight": "model-00093-of-00101.safetensors", + "model.layers.84.mlp.experts.109.up_proj.weight": "model-00093-of-00101.safetensors", + "model.layers.84.mlp.experts.109.down_proj.weight": "model-00093-of-00101.safetensors", + "model.layers.84.mlp.experts.110.gate_proj.weight": "model-00093-of-00101.safetensors", + "model.layers.84.mlp.experts.110.up_proj.weight": "model-00093-of-00101.safetensors", + "model.layers.84.mlp.experts.110.down_proj.weight": "model-00093-of-00101.safetensors", + "model.layers.84.mlp.experts.111.gate_proj.weight": "model-00093-of-00101.safetensors", + "model.layers.84.mlp.experts.111.up_proj.weight": "model-00093-of-00101.safetensors", + "model.layers.84.mlp.experts.111.down_proj.weight": "model-00093-of-00101.safetensors", + "model.layers.84.mlp.experts.112.gate_proj.weight": "model-00093-of-00101.safetensors", + "model.layers.84.mlp.experts.112.up_proj.weight": "model-00093-of-00101.safetensors", + "model.layers.84.mlp.experts.112.down_proj.weight": "model-00093-of-00101.safetensors", + "model.layers.84.mlp.experts.113.gate_proj.weight": "model-00093-of-00101.safetensors", + "model.layers.84.mlp.experts.113.up_proj.weight": "model-00093-of-00101.safetensors", + "model.layers.84.mlp.experts.113.down_proj.weight": "model-00093-of-00101.safetensors", + "model.layers.84.mlp.experts.114.gate_proj.weight": "model-00093-of-00101.safetensors", + "model.layers.84.mlp.experts.114.up_proj.weight": "model-00093-of-00101.safetensors", + "model.layers.84.mlp.experts.114.down_proj.weight": "model-00093-of-00101.safetensors", + "model.layers.84.mlp.experts.115.gate_proj.weight": "model-00093-of-00101.safetensors", + "model.layers.84.mlp.experts.115.up_proj.weight": "model-00093-of-00101.safetensors", + "model.layers.84.mlp.experts.115.down_proj.weight": "model-00093-of-00101.safetensors", + "model.layers.84.mlp.experts.116.gate_proj.weight": "model-00093-of-00101.safetensors", + "model.layers.84.mlp.experts.116.up_proj.weight": "model-00093-of-00101.safetensors", + "model.layers.84.mlp.experts.116.down_proj.weight": "model-00093-of-00101.safetensors", + "model.layers.84.mlp.experts.117.gate_proj.weight": "model-00093-of-00101.safetensors", + "model.layers.84.mlp.experts.117.up_proj.weight": "model-00093-of-00101.safetensors", + "model.layers.84.mlp.experts.117.down_proj.weight": "model-00093-of-00101.safetensors", + "model.layers.84.mlp.experts.118.gate_proj.weight": "model-00093-of-00101.safetensors", + "model.layers.84.mlp.experts.118.up_proj.weight": "model-00093-of-00101.safetensors", + "model.layers.84.mlp.experts.118.down_proj.weight": "model-00093-of-00101.safetensors", + "model.layers.84.mlp.experts.119.gate_proj.weight": "model-00093-of-00101.safetensors", + "model.layers.84.mlp.experts.119.up_proj.weight": "model-00093-of-00101.safetensors", + "model.layers.84.mlp.experts.119.down_proj.weight": "model-00093-of-00101.safetensors", + "model.layers.84.mlp.gate.weight": "model-00093-of-00101.safetensors", + "model.layers.84.mlp.gate.e_score_correction_bias": "model-00093-of-00101.safetensors", + "model.layers.84.mlp.shared_experts.gate_proj.weight": "model-00093-of-00101.safetensors", + "model.layers.84.mlp.shared_experts.up_proj.weight": "model-00093-of-00101.safetensors", + "model.layers.84.mlp.shared_experts.down_proj.weight": "model-00093-of-00101.safetensors", + "model.layers.84.input_layernorm.weight": "model-00093-of-00101.safetensors", + "model.layers.84.post_attention_layernorm.weight": "model-00093-of-00101.safetensors", + "model.layers.85.self_attn.q_proj.weight": "model-00093-of-00101.safetensors", + "model.layers.85.self_attn.q_proj.bias": "model-00093-of-00101.safetensors", + "model.layers.85.self_attn.k_proj.weight": "model-00093-of-00101.safetensors", + "model.layers.85.self_attn.k_proj.bias": "model-00093-of-00101.safetensors", + "model.layers.85.self_attn.v_proj.weight": "model-00093-of-00101.safetensors", + "model.layers.85.self_attn.v_proj.bias": "model-00093-of-00101.safetensors", + "model.layers.85.self_attn.o_proj.weight": "model-00093-of-00101.safetensors", + "model.layers.85.self_attn.q_norm.weight": "model-00093-of-00101.safetensors", + "model.layers.85.self_attn.k_norm.weight": "model-00093-of-00101.safetensors", + "model.layers.85.mlp.experts.0.gate_proj.weight": "model-00093-of-00101.safetensors", + "model.layers.85.mlp.experts.0.up_proj.weight": "model-00093-of-00101.safetensors", + "model.layers.85.mlp.experts.0.down_proj.weight": "model-00093-of-00101.safetensors", + "model.layers.85.mlp.experts.1.gate_proj.weight": "model-00093-of-00101.safetensors", + "model.layers.85.mlp.experts.1.up_proj.weight": "model-00093-of-00101.safetensors", + "model.layers.85.mlp.experts.1.down_proj.weight": "model-00093-of-00101.safetensors", + "model.layers.85.mlp.experts.2.gate_proj.weight": "model-00093-of-00101.safetensors", + "model.layers.85.mlp.experts.2.up_proj.weight": "model-00093-of-00101.safetensors", + "model.layers.85.mlp.experts.2.down_proj.weight": "model-00093-of-00101.safetensors", + "model.layers.85.mlp.experts.3.gate_proj.weight": "model-00093-of-00101.safetensors", + "model.layers.85.mlp.experts.3.up_proj.weight": "model-00093-of-00101.safetensors", + "model.layers.85.mlp.experts.3.down_proj.weight": "model-00093-of-00101.safetensors", + "model.layers.85.mlp.experts.4.gate_proj.weight": "model-00093-of-00101.safetensors", + "model.layers.85.mlp.experts.4.up_proj.weight": "model-00093-of-00101.safetensors", + "model.layers.85.mlp.experts.4.down_proj.weight": "model-00093-of-00101.safetensors", + "model.layers.85.mlp.experts.5.gate_proj.weight": "model-00093-of-00101.safetensors", + "model.layers.85.mlp.experts.5.up_proj.weight": "model-00093-of-00101.safetensors", + "model.layers.85.mlp.experts.5.down_proj.weight": "model-00093-of-00101.safetensors", + "model.layers.85.mlp.experts.6.gate_proj.weight": "model-00093-of-00101.safetensors", + "model.layers.85.mlp.experts.6.up_proj.weight": "model-00093-of-00101.safetensors", + "model.layers.85.mlp.experts.6.down_proj.weight": "model-00093-of-00101.safetensors", + "model.layers.85.mlp.experts.7.gate_proj.weight": "model-00093-of-00101.safetensors", + "model.layers.85.mlp.experts.7.up_proj.weight": "model-00093-of-00101.safetensors", + "model.layers.85.mlp.experts.7.down_proj.weight": "model-00093-of-00101.safetensors", + "model.layers.85.mlp.experts.8.gate_proj.weight": "model-00093-of-00101.safetensors", + "model.layers.85.mlp.experts.8.up_proj.weight": "model-00093-of-00101.safetensors", + "model.layers.85.mlp.experts.8.down_proj.weight": "model-00093-of-00101.safetensors", + "model.layers.85.mlp.experts.9.gate_proj.weight": "model-00093-of-00101.safetensors", + "model.layers.85.mlp.experts.9.up_proj.weight": "model-00093-of-00101.safetensors", + "model.layers.85.mlp.experts.9.down_proj.weight": "model-00093-of-00101.safetensors", + "model.layers.85.mlp.experts.10.gate_proj.weight": "model-00093-of-00101.safetensors", + "model.layers.85.mlp.experts.10.up_proj.weight": "model-00093-of-00101.safetensors", + "model.layers.85.mlp.experts.10.down_proj.weight": "model-00093-of-00101.safetensors", + "model.layers.85.mlp.experts.11.gate_proj.weight": "model-00093-of-00101.safetensors", + "model.layers.85.mlp.experts.11.up_proj.weight": "model-00093-of-00101.safetensors", + "model.layers.85.mlp.experts.11.down_proj.weight": "model-00093-of-00101.safetensors", + "model.layers.85.mlp.experts.12.gate_proj.weight": "model-00093-of-00101.safetensors", + "model.layers.85.mlp.experts.12.up_proj.weight": "model-00093-of-00101.safetensors", + "model.layers.85.mlp.experts.12.down_proj.weight": "model-00093-of-00101.safetensors", + "model.layers.85.mlp.experts.13.gate_proj.weight": "model-00093-of-00101.safetensors", + "model.layers.85.mlp.experts.13.up_proj.weight": "model-00093-of-00101.safetensors", + "model.layers.85.mlp.experts.13.down_proj.weight": "model-00093-of-00101.safetensors", + "model.layers.85.mlp.experts.14.gate_proj.weight": "model-00093-of-00101.safetensors", + "model.layers.85.mlp.experts.14.up_proj.weight": "model-00093-of-00101.safetensors", + "model.layers.85.mlp.experts.14.down_proj.weight": "model-00093-of-00101.safetensors", + "model.layers.85.mlp.experts.15.gate_proj.weight": "model-00093-of-00101.safetensors", + "model.layers.85.mlp.experts.15.up_proj.weight": "model-00093-of-00101.safetensors", + "model.layers.85.mlp.experts.15.down_proj.weight": "model-00093-of-00101.safetensors", + "model.layers.85.mlp.experts.16.gate_proj.weight": "model-00093-of-00101.safetensors", + "model.layers.85.mlp.experts.16.up_proj.weight": "model-00093-of-00101.safetensors", + "model.layers.85.mlp.experts.16.down_proj.weight": "model-00093-of-00101.safetensors", + "model.layers.85.mlp.experts.17.gate_proj.weight": "model-00093-of-00101.safetensors", + "model.layers.85.mlp.experts.17.up_proj.weight": "model-00093-of-00101.safetensors", + "model.layers.85.mlp.experts.17.down_proj.weight": "model-00093-of-00101.safetensors", + "model.layers.85.mlp.experts.18.gate_proj.weight": "model-00093-of-00101.safetensors", + "model.layers.85.mlp.experts.18.up_proj.weight": "model-00093-of-00101.safetensors", + "model.layers.85.mlp.experts.18.down_proj.weight": "model-00093-of-00101.safetensors", + "model.layers.85.mlp.experts.19.gate_proj.weight": "model-00093-of-00101.safetensors", + "model.layers.85.mlp.experts.19.up_proj.weight": "model-00093-of-00101.safetensors", + "model.layers.85.mlp.experts.19.down_proj.weight": "model-00093-of-00101.safetensors", + "model.layers.85.mlp.experts.20.gate_proj.weight": "model-00093-of-00101.safetensors", + "model.layers.85.mlp.experts.20.up_proj.weight": "model-00093-of-00101.safetensors", + "model.layers.85.mlp.experts.20.down_proj.weight": "model-00093-of-00101.safetensors", + "model.layers.85.mlp.experts.21.gate_proj.weight": "model-00093-of-00101.safetensors", + "model.layers.85.mlp.experts.21.up_proj.weight": "model-00093-of-00101.safetensors", + "model.layers.85.mlp.experts.21.down_proj.weight": "model-00093-of-00101.safetensors", + "model.layers.85.mlp.experts.22.gate_proj.weight": "model-00093-of-00101.safetensors", + "model.layers.85.mlp.experts.22.up_proj.weight": "model-00093-of-00101.safetensors", + "model.layers.85.mlp.experts.22.down_proj.weight": "model-00093-of-00101.safetensors", + "model.layers.85.mlp.experts.23.gate_proj.weight": "model-00093-of-00101.safetensors", + "model.layers.85.mlp.experts.23.up_proj.weight": "model-00093-of-00101.safetensors", + "model.layers.85.mlp.experts.23.down_proj.weight": "model-00093-of-00101.safetensors", + "model.layers.85.mlp.experts.24.gate_proj.weight": "model-00093-of-00101.safetensors", + "model.layers.85.mlp.experts.24.up_proj.weight": "model-00093-of-00101.safetensors", + "model.layers.85.mlp.experts.24.down_proj.weight": "model-00093-of-00101.safetensors", + "model.layers.85.mlp.experts.25.gate_proj.weight": "model-00093-of-00101.safetensors", + "model.layers.85.mlp.experts.25.up_proj.weight": "model-00093-of-00101.safetensors", + "model.layers.85.mlp.experts.25.down_proj.weight": "model-00093-of-00101.safetensors", + "model.layers.85.mlp.experts.26.gate_proj.weight": "model-00093-of-00101.safetensors", + "model.layers.85.mlp.experts.26.up_proj.weight": "model-00093-of-00101.safetensors", + "model.layers.85.mlp.experts.26.down_proj.weight": "model-00093-of-00101.safetensors", + "model.layers.85.mlp.experts.27.gate_proj.weight": "model-00093-of-00101.safetensors", + "model.layers.85.mlp.experts.27.up_proj.weight": "model-00093-of-00101.safetensors", + "model.layers.85.mlp.experts.27.down_proj.weight": "model-00093-of-00101.safetensors", + "model.layers.85.mlp.experts.28.gate_proj.weight": "model-00093-of-00101.safetensors", + "model.layers.85.mlp.experts.28.up_proj.weight": "model-00093-of-00101.safetensors", + "model.layers.85.mlp.experts.28.down_proj.weight": "model-00093-of-00101.safetensors", + "model.layers.85.mlp.experts.29.gate_proj.weight": "model-00093-of-00101.safetensors", + "model.layers.85.mlp.experts.29.up_proj.weight": "model-00093-of-00101.safetensors", + "model.layers.85.mlp.experts.29.down_proj.weight": "model-00093-of-00101.safetensors", + "model.layers.85.mlp.experts.30.gate_proj.weight": "model-00093-of-00101.safetensors", + "model.layers.85.mlp.experts.30.up_proj.weight": "model-00093-of-00101.safetensors", + "model.layers.85.mlp.experts.30.down_proj.weight": "model-00093-of-00101.safetensors", + "model.layers.85.mlp.experts.31.gate_proj.weight": "model-00093-of-00101.safetensors", + "model.layers.85.mlp.experts.31.up_proj.weight": "model-00093-of-00101.safetensors", + "model.layers.85.mlp.experts.31.down_proj.weight": "model-00093-of-00101.safetensors", + "model.layers.85.mlp.experts.32.gate_proj.weight": "model-00093-of-00101.safetensors", + "model.layers.85.mlp.experts.32.up_proj.weight": "model-00093-of-00101.safetensors", + "model.layers.85.mlp.experts.32.down_proj.weight": "model-00093-of-00101.safetensors", + "model.layers.85.mlp.experts.33.gate_proj.weight": "model-00093-of-00101.safetensors", + "model.layers.85.mlp.experts.33.up_proj.weight": "model-00093-of-00101.safetensors", + "model.layers.85.mlp.experts.33.down_proj.weight": "model-00093-of-00101.safetensors", + "model.layers.85.mlp.experts.34.gate_proj.weight": "model-00093-of-00101.safetensors", + "model.layers.85.mlp.experts.34.up_proj.weight": "model-00093-of-00101.safetensors", + "model.layers.85.mlp.experts.34.down_proj.weight": "model-00093-of-00101.safetensors", + "model.layers.85.mlp.experts.35.gate_proj.weight": "model-00093-of-00101.safetensors", + "model.layers.85.mlp.experts.35.up_proj.weight": "model-00093-of-00101.safetensors", + "model.layers.85.mlp.experts.35.down_proj.weight": "model-00093-of-00101.safetensors", + "model.layers.85.mlp.experts.36.gate_proj.weight": "model-00093-of-00101.safetensors", + "model.layers.85.mlp.experts.36.up_proj.weight": "model-00093-of-00101.safetensors", + "model.layers.85.mlp.experts.36.down_proj.weight": "model-00093-of-00101.safetensors", + "model.layers.85.mlp.experts.37.gate_proj.weight": "model-00093-of-00101.safetensors", + "model.layers.85.mlp.experts.37.up_proj.weight": "model-00093-of-00101.safetensors", + "model.layers.85.mlp.experts.37.down_proj.weight": "model-00093-of-00101.safetensors", + "model.layers.85.mlp.experts.38.gate_proj.weight": "model-00093-of-00101.safetensors", + "model.layers.85.mlp.experts.38.up_proj.weight": "model-00093-of-00101.safetensors", + "model.layers.85.mlp.experts.38.down_proj.weight": "model-00093-of-00101.safetensors", + "model.layers.85.mlp.experts.39.gate_proj.weight": "model-00093-of-00101.safetensors", + "model.layers.85.mlp.experts.39.up_proj.weight": "model-00093-of-00101.safetensors", + "model.layers.85.mlp.experts.39.down_proj.weight": "model-00093-of-00101.safetensors", + "model.layers.85.mlp.experts.40.gate_proj.weight": "model-00093-of-00101.safetensors", + "model.layers.85.mlp.experts.40.up_proj.weight": "model-00093-of-00101.safetensors", + "model.layers.85.mlp.experts.40.down_proj.weight": "model-00093-of-00101.safetensors", + "model.layers.85.mlp.experts.41.gate_proj.weight": "model-00093-of-00101.safetensors", + "model.layers.85.mlp.experts.41.up_proj.weight": "model-00093-of-00101.safetensors", + "model.layers.85.mlp.experts.41.down_proj.weight": "model-00093-of-00101.safetensors", + "model.layers.85.mlp.experts.42.gate_proj.weight": "model-00093-of-00101.safetensors", + "model.layers.85.mlp.experts.42.up_proj.weight": "model-00093-of-00101.safetensors", + "model.layers.85.mlp.experts.42.down_proj.weight": "model-00093-of-00101.safetensors", + "model.layers.85.mlp.experts.43.gate_proj.weight": "model-00093-of-00101.safetensors", + "model.layers.85.mlp.experts.43.up_proj.weight": "model-00093-of-00101.safetensors", + "model.layers.85.mlp.experts.43.down_proj.weight": "model-00093-of-00101.safetensors", + "model.layers.85.mlp.experts.44.gate_proj.weight": "model-00093-of-00101.safetensors", + "model.layers.85.mlp.experts.44.up_proj.weight": "model-00093-of-00101.safetensors", + "model.layers.85.mlp.experts.44.down_proj.weight": "model-00093-of-00101.safetensors", + "model.layers.85.mlp.experts.45.gate_proj.weight": "model-00093-of-00101.safetensors", + "model.layers.85.mlp.experts.45.up_proj.weight": "model-00093-of-00101.safetensors", + "model.layers.85.mlp.experts.45.down_proj.weight": "model-00093-of-00101.safetensors", + "model.layers.85.mlp.experts.46.gate_proj.weight": "model-00093-of-00101.safetensors", + "model.layers.85.mlp.experts.46.up_proj.weight": "model-00093-of-00101.safetensors", + "model.layers.85.mlp.experts.46.down_proj.weight": "model-00093-of-00101.safetensors", + "model.layers.85.mlp.experts.47.gate_proj.weight": "model-00093-of-00101.safetensors", + "model.layers.85.mlp.experts.47.up_proj.weight": "model-00093-of-00101.safetensors", + "model.layers.85.mlp.experts.47.down_proj.weight": "model-00093-of-00101.safetensors", + "model.layers.85.mlp.experts.48.gate_proj.weight": "model-00093-of-00101.safetensors", + "model.layers.85.mlp.experts.48.up_proj.weight": "model-00093-of-00101.safetensors", + "model.layers.85.mlp.experts.48.down_proj.weight": "model-00093-of-00101.safetensors", + "model.layers.85.mlp.experts.49.gate_proj.weight": "model-00093-of-00101.safetensors", + "model.layers.85.mlp.experts.49.up_proj.weight": "model-00093-of-00101.safetensors", + "model.layers.85.mlp.experts.49.down_proj.weight": "model-00093-of-00101.safetensors", + "model.layers.85.mlp.experts.50.gate_proj.weight": "model-00093-of-00101.safetensors", + "model.layers.85.mlp.experts.50.up_proj.weight": "model-00093-of-00101.safetensors", + "model.layers.85.mlp.experts.50.down_proj.weight": "model-00093-of-00101.safetensors", + "model.layers.85.mlp.experts.51.gate_proj.weight": "model-00093-of-00101.safetensors", + "model.layers.85.mlp.experts.51.up_proj.weight": "model-00093-of-00101.safetensors", + "model.layers.85.mlp.experts.51.down_proj.weight": "model-00093-of-00101.safetensors", + "model.layers.85.mlp.experts.52.gate_proj.weight": "model-00093-of-00101.safetensors", + "model.layers.85.mlp.experts.52.up_proj.weight": "model-00093-of-00101.safetensors", + "model.layers.85.mlp.experts.52.down_proj.weight": "model-00093-of-00101.safetensors", + "model.layers.85.mlp.experts.53.gate_proj.weight": "model-00093-of-00101.safetensors", + "model.layers.85.mlp.experts.53.up_proj.weight": "model-00093-of-00101.safetensors", + "model.layers.85.mlp.experts.53.down_proj.weight": "model-00093-of-00101.safetensors", + "model.layers.85.mlp.experts.54.gate_proj.weight": "model-00093-of-00101.safetensors", + "model.layers.85.mlp.experts.54.up_proj.weight": "model-00093-of-00101.safetensors", + "model.layers.85.mlp.experts.54.down_proj.weight": "model-00093-of-00101.safetensors", + "model.layers.85.mlp.experts.55.gate_proj.weight": "model-00093-of-00101.safetensors", + "model.layers.85.mlp.experts.55.up_proj.weight": "model-00093-of-00101.safetensors", + "model.layers.85.mlp.experts.55.down_proj.weight": "model-00093-of-00101.safetensors", + "model.layers.85.mlp.experts.56.gate_proj.weight": "model-00093-of-00101.safetensors", + "model.layers.85.mlp.experts.56.up_proj.weight": "model-00093-of-00101.safetensors", + "model.layers.85.mlp.experts.56.down_proj.weight": "model-00093-of-00101.safetensors", + "model.layers.85.mlp.experts.57.gate_proj.weight": "model-00093-of-00101.safetensors", + "model.layers.85.mlp.experts.57.up_proj.weight": "model-00093-of-00101.safetensors", + "model.layers.85.mlp.experts.57.down_proj.weight": "model-00093-of-00101.safetensors", + "model.layers.85.mlp.experts.58.gate_proj.weight": "model-00093-of-00101.safetensors", + "model.layers.85.mlp.experts.58.up_proj.weight": "model-00093-of-00101.safetensors", + "model.layers.85.mlp.experts.58.down_proj.weight": "model-00093-of-00101.safetensors", + "model.layers.85.mlp.experts.59.gate_proj.weight": "model-00093-of-00101.safetensors", + "model.layers.85.mlp.experts.59.up_proj.weight": "model-00093-of-00101.safetensors", + "model.layers.85.mlp.experts.59.down_proj.weight": "model-00093-of-00101.safetensors", + "model.layers.85.mlp.experts.60.gate_proj.weight": "model-00093-of-00101.safetensors", + "model.layers.85.mlp.experts.60.up_proj.weight": "model-00093-of-00101.safetensors", + "model.layers.85.mlp.experts.60.down_proj.weight": "model-00093-of-00101.safetensors", + "model.layers.85.mlp.experts.61.gate_proj.weight": "model-00093-of-00101.safetensors", + "model.layers.85.mlp.experts.61.up_proj.weight": "model-00093-of-00101.safetensors", + "model.layers.85.mlp.experts.61.down_proj.weight": "model-00093-of-00101.safetensors", + "model.layers.85.mlp.experts.62.gate_proj.weight": "model-00093-of-00101.safetensors", + "model.layers.85.mlp.experts.62.up_proj.weight": "model-00093-of-00101.safetensors", + "model.layers.85.mlp.experts.62.down_proj.weight": "model-00093-of-00101.safetensors", + "model.layers.85.mlp.experts.63.gate_proj.weight": "model-00093-of-00101.safetensors", + "model.layers.85.mlp.experts.63.up_proj.weight": "model-00093-of-00101.safetensors", + "model.layers.85.mlp.experts.63.down_proj.weight": "model-00093-of-00101.safetensors", + "model.layers.85.mlp.experts.64.gate_proj.weight": "model-00093-of-00101.safetensors", + "model.layers.85.mlp.experts.64.up_proj.weight": "model-00093-of-00101.safetensors", + "model.layers.85.mlp.experts.64.down_proj.weight": "model-00093-of-00101.safetensors", + "model.layers.85.mlp.experts.65.gate_proj.weight": "model-00093-of-00101.safetensors", + "model.layers.85.mlp.experts.65.up_proj.weight": "model-00093-of-00101.safetensors", + "model.layers.85.mlp.experts.65.down_proj.weight": "model-00093-of-00101.safetensors", + "model.layers.85.mlp.experts.66.gate_proj.weight": "model-00093-of-00101.safetensors", + "model.layers.85.mlp.experts.66.up_proj.weight": "model-00093-of-00101.safetensors", + "model.layers.85.mlp.experts.66.down_proj.weight": "model-00093-of-00101.safetensors", + "model.layers.85.mlp.experts.67.gate_proj.weight": "model-00093-of-00101.safetensors", + "model.layers.85.mlp.experts.67.up_proj.weight": "model-00093-of-00101.safetensors", + "model.layers.85.mlp.experts.67.down_proj.weight": "model-00093-of-00101.safetensors", + "model.layers.85.mlp.experts.68.gate_proj.weight": "model-00093-of-00101.safetensors", + "model.layers.85.mlp.experts.68.up_proj.weight": "model-00093-of-00101.safetensors", + "model.layers.85.mlp.experts.68.down_proj.weight": "model-00093-of-00101.safetensors", + "model.layers.85.mlp.experts.69.gate_proj.weight": "model-00093-of-00101.safetensors", + "model.layers.85.mlp.experts.69.up_proj.weight": "model-00093-of-00101.safetensors", + "model.layers.85.mlp.experts.69.down_proj.weight": "model-00093-of-00101.safetensors", + "model.layers.85.mlp.experts.70.gate_proj.weight": "model-00093-of-00101.safetensors", + "model.layers.85.mlp.experts.70.up_proj.weight": "model-00093-of-00101.safetensors", + "model.layers.85.mlp.experts.70.down_proj.weight": "model-00093-of-00101.safetensors", + "model.layers.85.mlp.experts.71.gate_proj.weight": "model-00093-of-00101.safetensors", + "model.layers.85.mlp.experts.71.up_proj.weight": "model-00093-of-00101.safetensors", + "model.layers.85.mlp.experts.71.down_proj.weight": "model-00093-of-00101.safetensors", + "model.layers.85.mlp.experts.72.gate_proj.weight": "model-00093-of-00101.safetensors", + "model.layers.85.mlp.experts.72.up_proj.weight": "model-00093-of-00101.safetensors", + "model.layers.85.mlp.experts.72.down_proj.weight": "model-00093-of-00101.safetensors", + "model.layers.85.mlp.experts.73.gate_proj.weight": "model-00093-of-00101.safetensors", + "model.layers.85.mlp.experts.73.up_proj.weight": "model-00093-of-00101.safetensors", + "model.layers.85.mlp.experts.73.down_proj.weight": "model-00093-of-00101.safetensors", + "model.layers.85.mlp.experts.74.gate_proj.weight": "model-00093-of-00101.safetensors", + "model.layers.85.mlp.experts.74.up_proj.weight": "model-00093-of-00101.safetensors", + "model.layers.85.mlp.experts.74.down_proj.weight": "model-00094-of-00101.safetensors", + "model.layers.85.mlp.experts.75.gate_proj.weight": "model-00094-of-00101.safetensors", + "model.layers.85.mlp.experts.75.up_proj.weight": "model-00094-of-00101.safetensors", + "model.layers.85.mlp.experts.75.down_proj.weight": "model-00094-of-00101.safetensors", + "model.layers.85.mlp.experts.76.gate_proj.weight": "model-00094-of-00101.safetensors", + "model.layers.85.mlp.experts.76.up_proj.weight": "model-00094-of-00101.safetensors", + "model.layers.85.mlp.experts.76.down_proj.weight": "model-00094-of-00101.safetensors", + "model.layers.85.mlp.experts.77.gate_proj.weight": "model-00094-of-00101.safetensors", + "model.layers.85.mlp.experts.77.up_proj.weight": "model-00094-of-00101.safetensors", + "model.layers.85.mlp.experts.77.down_proj.weight": "model-00094-of-00101.safetensors", + "model.layers.85.mlp.experts.78.gate_proj.weight": "model-00094-of-00101.safetensors", + "model.layers.85.mlp.experts.78.up_proj.weight": "model-00094-of-00101.safetensors", + "model.layers.85.mlp.experts.78.down_proj.weight": "model-00094-of-00101.safetensors", + "model.layers.85.mlp.experts.79.gate_proj.weight": "model-00094-of-00101.safetensors", + "model.layers.85.mlp.experts.79.up_proj.weight": "model-00094-of-00101.safetensors", + "model.layers.85.mlp.experts.79.down_proj.weight": "model-00094-of-00101.safetensors", + "model.layers.85.mlp.experts.80.gate_proj.weight": "model-00094-of-00101.safetensors", + "model.layers.85.mlp.experts.80.up_proj.weight": "model-00094-of-00101.safetensors", + "model.layers.85.mlp.experts.80.down_proj.weight": "model-00094-of-00101.safetensors", + "model.layers.85.mlp.experts.81.gate_proj.weight": "model-00094-of-00101.safetensors", + "model.layers.85.mlp.experts.81.up_proj.weight": "model-00094-of-00101.safetensors", + "model.layers.85.mlp.experts.81.down_proj.weight": "model-00094-of-00101.safetensors", + "model.layers.85.mlp.experts.82.gate_proj.weight": "model-00094-of-00101.safetensors", + "model.layers.85.mlp.experts.82.up_proj.weight": "model-00094-of-00101.safetensors", + "model.layers.85.mlp.experts.82.down_proj.weight": "model-00094-of-00101.safetensors", + "model.layers.85.mlp.experts.83.gate_proj.weight": "model-00094-of-00101.safetensors", + "model.layers.85.mlp.experts.83.up_proj.weight": "model-00094-of-00101.safetensors", + "model.layers.85.mlp.experts.83.down_proj.weight": "model-00094-of-00101.safetensors", + "model.layers.85.mlp.experts.84.gate_proj.weight": "model-00094-of-00101.safetensors", + "model.layers.85.mlp.experts.84.up_proj.weight": "model-00094-of-00101.safetensors", + "model.layers.85.mlp.experts.84.down_proj.weight": "model-00094-of-00101.safetensors", + "model.layers.85.mlp.experts.85.gate_proj.weight": "model-00094-of-00101.safetensors", + "model.layers.85.mlp.experts.85.up_proj.weight": "model-00094-of-00101.safetensors", + "model.layers.85.mlp.experts.85.down_proj.weight": "model-00094-of-00101.safetensors", + "model.layers.85.mlp.experts.86.gate_proj.weight": "model-00094-of-00101.safetensors", + "model.layers.85.mlp.experts.86.up_proj.weight": "model-00094-of-00101.safetensors", + "model.layers.85.mlp.experts.86.down_proj.weight": "model-00094-of-00101.safetensors", + "model.layers.85.mlp.experts.87.gate_proj.weight": "model-00094-of-00101.safetensors", + "model.layers.85.mlp.experts.87.up_proj.weight": "model-00094-of-00101.safetensors", + "model.layers.85.mlp.experts.87.down_proj.weight": "model-00094-of-00101.safetensors", + "model.layers.85.mlp.experts.88.gate_proj.weight": "model-00094-of-00101.safetensors", + "model.layers.85.mlp.experts.88.up_proj.weight": "model-00094-of-00101.safetensors", + "model.layers.85.mlp.experts.88.down_proj.weight": "model-00094-of-00101.safetensors", + "model.layers.85.mlp.experts.89.gate_proj.weight": "model-00094-of-00101.safetensors", + "model.layers.85.mlp.experts.89.up_proj.weight": "model-00094-of-00101.safetensors", + "model.layers.85.mlp.experts.89.down_proj.weight": "model-00094-of-00101.safetensors", + "model.layers.85.mlp.experts.90.gate_proj.weight": "model-00094-of-00101.safetensors", + "model.layers.85.mlp.experts.90.up_proj.weight": "model-00094-of-00101.safetensors", + "model.layers.85.mlp.experts.90.down_proj.weight": "model-00094-of-00101.safetensors", + "model.layers.85.mlp.experts.91.gate_proj.weight": "model-00094-of-00101.safetensors", + "model.layers.85.mlp.experts.91.up_proj.weight": "model-00094-of-00101.safetensors", + "model.layers.85.mlp.experts.91.down_proj.weight": "model-00094-of-00101.safetensors", + "model.layers.85.mlp.experts.92.gate_proj.weight": "model-00094-of-00101.safetensors", + "model.layers.85.mlp.experts.92.up_proj.weight": "model-00094-of-00101.safetensors", + "model.layers.85.mlp.experts.92.down_proj.weight": "model-00094-of-00101.safetensors", + "model.layers.85.mlp.experts.93.gate_proj.weight": "model-00094-of-00101.safetensors", + "model.layers.85.mlp.experts.93.up_proj.weight": "model-00094-of-00101.safetensors", + "model.layers.85.mlp.experts.93.down_proj.weight": "model-00094-of-00101.safetensors", + "model.layers.85.mlp.experts.94.gate_proj.weight": "model-00094-of-00101.safetensors", + "model.layers.85.mlp.experts.94.up_proj.weight": "model-00094-of-00101.safetensors", + "model.layers.85.mlp.experts.94.down_proj.weight": "model-00094-of-00101.safetensors", + "model.layers.85.mlp.experts.95.gate_proj.weight": "model-00094-of-00101.safetensors", + "model.layers.85.mlp.experts.95.up_proj.weight": "model-00094-of-00101.safetensors", + "model.layers.85.mlp.experts.95.down_proj.weight": "model-00094-of-00101.safetensors", + "model.layers.85.mlp.experts.96.gate_proj.weight": "model-00094-of-00101.safetensors", + "model.layers.85.mlp.experts.96.up_proj.weight": "model-00094-of-00101.safetensors", + "model.layers.85.mlp.experts.96.down_proj.weight": "model-00094-of-00101.safetensors", + "model.layers.85.mlp.experts.97.gate_proj.weight": "model-00094-of-00101.safetensors", + "model.layers.85.mlp.experts.97.up_proj.weight": "model-00094-of-00101.safetensors", + "model.layers.85.mlp.experts.97.down_proj.weight": "model-00094-of-00101.safetensors", + "model.layers.85.mlp.experts.98.gate_proj.weight": "model-00094-of-00101.safetensors", + "model.layers.85.mlp.experts.98.up_proj.weight": "model-00094-of-00101.safetensors", + "model.layers.85.mlp.experts.98.down_proj.weight": "model-00094-of-00101.safetensors", + "model.layers.85.mlp.experts.99.gate_proj.weight": "model-00094-of-00101.safetensors", + "model.layers.85.mlp.experts.99.up_proj.weight": "model-00094-of-00101.safetensors", + "model.layers.85.mlp.experts.99.down_proj.weight": "model-00094-of-00101.safetensors", + "model.layers.85.mlp.experts.100.gate_proj.weight": "model-00094-of-00101.safetensors", + "model.layers.85.mlp.experts.100.up_proj.weight": "model-00094-of-00101.safetensors", + "model.layers.85.mlp.experts.100.down_proj.weight": "model-00094-of-00101.safetensors", + "model.layers.85.mlp.experts.101.gate_proj.weight": "model-00094-of-00101.safetensors", + "model.layers.85.mlp.experts.101.up_proj.weight": "model-00094-of-00101.safetensors", + "model.layers.85.mlp.experts.101.down_proj.weight": "model-00094-of-00101.safetensors", + "model.layers.85.mlp.experts.102.gate_proj.weight": "model-00094-of-00101.safetensors", + "model.layers.85.mlp.experts.102.up_proj.weight": "model-00094-of-00101.safetensors", + "model.layers.85.mlp.experts.102.down_proj.weight": "model-00094-of-00101.safetensors", + "model.layers.85.mlp.experts.103.gate_proj.weight": "model-00094-of-00101.safetensors", + "model.layers.85.mlp.experts.103.up_proj.weight": "model-00094-of-00101.safetensors", + "model.layers.85.mlp.experts.103.down_proj.weight": "model-00094-of-00101.safetensors", + "model.layers.85.mlp.experts.104.gate_proj.weight": "model-00094-of-00101.safetensors", + "model.layers.85.mlp.experts.104.up_proj.weight": "model-00094-of-00101.safetensors", + "model.layers.85.mlp.experts.104.down_proj.weight": "model-00094-of-00101.safetensors", + "model.layers.85.mlp.experts.105.gate_proj.weight": "model-00094-of-00101.safetensors", + "model.layers.85.mlp.experts.105.up_proj.weight": "model-00094-of-00101.safetensors", + "model.layers.85.mlp.experts.105.down_proj.weight": "model-00094-of-00101.safetensors", + "model.layers.85.mlp.experts.106.gate_proj.weight": "model-00094-of-00101.safetensors", + "model.layers.85.mlp.experts.106.up_proj.weight": "model-00094-of-00101.safetensors", + "model.layers.85.mlp.experts.106.down_proj.weight": "model-00094-of-00101.safetensors", + "model.layers.85.mlp.experts.107.gate_proj.weight": "model-00094-of-00101.safetensors", + "model.layers.85.mlp.experts.107.up_proj.weight": "model-00094-of-00101.safetensors", + "model.layers.85.mlp.experts.107.down_proj.weight": "model-00094-of-00101.safetensors", + "model.layers.85.mlp.experts.108.gate_proj.weight": "model-00094-of-00101.safetensors", + "model.layers.85.mlp.experts.108.up_proj.weight": "model-00094-of-00101.safetensors", + "model.layers.85.mlp.experts.108.down_proj.weight": "model-00094-of-00101.safetensors", + "model.layers.85.mlp.experts.109.gate_proj.weight": "model-00094-of-00101.safetensors", + "model.layers.85.mlp.experts.109.up_proj.weight": "model-00094-of-00101.safetensors", + "model.layers.85.mlp.experts.109.down_proj.weight": "model-00094-of-00101.safetensors", + "model.layers.85.mlp.experts.110.gate_proj.weight": "model-00094-of-00101.safetensors", + "model.layers.85.mlp.experts.110.up_proj.weight": "model-00094-of-00101.safetensors", + "model.layers.85.mlp.experts.110.down_proj.weight": "model-00094-of-00101.safetensors", + "model.layers.85.mlp.experts.111.gate_proj.weight": "model-00094-of-00101.safetensors", + "model.layers.85.mlp.experts.111.up_proj.weight": "model-00094-of-00101.safetensors", + "model.layers.85.mlp.experts.111.down_proj.weight": "model-00094-of-00101.safetensors", + "model.layers.85.mlp.experts.112.gate_proj.weight": "model-00094-of-00101.safetensors", + "model.layers.85.mlp.experts.112.up_proj.weight": "model-00094-of-00101.safetensors", + "model.layers.85.mlp.experts.112.down_proj.weight": "model-00094-of-00101.safetensors", + "model.layers.85.mlp.experts.113.gate_proj.weight": "model-00094-of-00101.safetensors", + "model.layers.85.mlp.experts.113.up_proj.weight": "model-00094-of-00101.safetensors", + "model.layers.85.mlp.experts.113.down_proj.weight": "model-00094-of-00101.safetensors", + "model.layers.85.mlp.experts.114.gate_proj.weight": "model-00094-of-00101.safetensors", + "model.layers.85.mlp.experts.114.up_proj.weight": "model-00094-of-00101.safetensors", + "model.layers.85.mlp.experts.114.down_proj.weight": "model-00094-of-00101.safetensors", + "model.layers.85.mlp.experts.115.gate_proj.weight": "model-00094-of-00101.safetensors", + "model.layers.85.mlp.experts.115.up_proj.weight": "model-00094-of-00101.safetensors", + "model.layers.85.mlp.experts.115.down_proj.weight": "model-00094-of-00101.safetensors", + "model.layers.85.mlp.experts.116.gate_proj.weight": "model-00094-of-00101.safetensors", + "model.layers.85.mlp.experts.116.up_proj.weight": "model-00094-of-00101.safetensors", + "model.layers.85.mlp.experts.116.down_proj.weight": "model-00094-of-00101.safetensors", + "model.layers.85.mlp.experts.117.gate_proj.weight": "model-00094-of-00101.safetensors", + "model.layers.85.mlp.experts.117.up_proj.weight": "model-00094-of-00101.safetensors", + "model.layers.85.mlp.experts.117.down_proj.weight": "model-00094-of-00101.safetensors", + "model.layers.85.mlp.experts.118.gate_proj.weight": "model-00094-of-00101.safetensors", + "model.layers.85.mlp.experts.118.up_proj.weight": "model-00094-of-00101.safetensors", + "model.layers.85.mlp.experts.118.down_proj.weight": "model-00094-of-00101.safetensors", + "model.layers.85.mlp.experts.119.gate_proj.weight": "model-00094-of-00101.safetensors", + "model.layers.85.mlp.experts.119.up_proj.weight": "model-00094-of-00101.safetensors", + "model.layers.85.mlp.experts.119.down_proj.weight": "model-00094-of-00101.safetensors", + "model.layers.85.mlp.gate.weight": "model-00094-of-00101.safetensors", + "model.layers.85.mlp.gate.e_score_correction_bias": "model-00094-of-00101.safetensors", + "model.layers.85.mlp.shared_experts.gate_proj.weight": "model-00094-of-00101.safetensors", + "model.layers.85.mlp.shared_experts.up_proj.weight": "model-00094-of-00101.safetensors", + "model.layers.85.mlp.shared_experts.down_proj.weight": "model-00094-of-00101.safetensors", + "model.layers.85.input_layernorm.weight": "model-00094-of-00101.safetensors", + "model.layers.85.post_attention_layernorm.weight": "model-00094-of-00101.safetensors", + "model.layers.86.self_attn.q_proj.weight": "model-00094-of-00101.safetensors", + "model.layers.86.self_attn.q_proj.bias": "model-00094-of-00101.safetensors", + "model.layers.86.self_attn.k_proj.weight": "model-00094-of-00101.safetensors", + "model.layers.86.self_attn.k_proj.bias": "model-00094-of-00101.safetensors", + "model.layers.86.self_attn.v_proj.weight": "model-00094-of-00101.safetensors", + "model.layers.86.self_attn.v_proj.bias": "model-00094-of-00101.safetensors", + "model.layers.86.self_attn.o_proj.weight": "model-00094-of-00101.safetensors", + "model.layers.86.self_attn.q_norm.weight": "model-00094-of-00101.safetensors", + "model.layers.86.self_attn.k_norm.weight": "model-00094-of-00101.safetensors", + "model.layers.86.mlp.experts.0.gate_proj.weight": "model-00094-of-00101.safetensors", + "model.layers.86.mlp.experts.0.up_proj.weight": "model-00094-of-00101.safetensors", + "model.layers.86.mlp.experts.0.down_proj.weight": "model-00094-of-00101.safetensors", + "model.layers.86.mlp.experts.1.gate_proj.weight": "model-00094-of-00101.safetensors", + "model.layers.86.mlp.experts.1.up_proj.weight": "model-00094-of-00101.safetensors", + "model.layers.86.mlp.experts.1.down_proj.weight": "model-00094-of-00101.safetensors", + "model.layers.86.mlp.experts.2.gate_proj.weight": "model-00094-of-00101.safetensors", + "model.layers.86.mlp.experts.2.up_proj.weight": "model-00094-of-00101.safetensors", + "model.layers.86.mlp.experts.2.down_proj.weight": "model-00094-of-00101.safetensors", + "model.layers.86.mlp.experts.3.gate_proj.weight": "model-00094-of-00101.safetensors", + "model.layers.86.mlp.experts.3.up_proj.weight": "model-00094-of-00101.safetensors", + "model.layers.86.mlp.experts.3.down_proj.weight": "model-00094-of-00101.safetensors", + "model.layers.86.mlp.experts.4.gate_proj.weight": "model-00094-of-00101.safetensors", + "model.layers.86.mlp.experts.4.up_proj.weight": "model-00094-of-00101.safetensors", + "model.layers.86.mlp.experts.4.down_proj.weight": "model-00094-of-00101.safetensors", + "model.layers.86.mlp.experts.5.gate_proj.weight": "model-00094-of-00101.safetensors", + "model.layers.86.mlp.experts.5.up_proj.weight": "model-00094-of-00101.safetensors", + "model.layers.86.mlp.experts.5.down_proj.weight": "model-00094-of-00101.safetensors", + "model.layers.86.mlp.experts.6.gate_proj.weight": "model-00094-of-00101.safetensors", + "model.layers.86.mlp.experts.6.up_proj.weight": "model-00094-of-00101.safetensors", + "model.layers.86.mlp.experts.6.down_proj.weight": "model-00094-of-00101.safetensors", + "model.layers.86.mlp.experts.7.gate_proj.weight": "model-00094-of-00101.safetensors", + "model.layers.86.mlp.experts.7.up_proj.weight": "model-00094-of-00101.safetensors", + "model.layers.86.mlp.experts.7.down_proj.weight": "model-00094-of-00101.safetensors", + "model.layers.86.mlp.experts.8.gate_proj.weight": "model-00094-of-00101.safetensors", + "model.layers.86.mlp.experts.8.up_proj.weight": "model-00094-of-00101.safetensors", + "model.layers.86.mlp.experts.8.down_proj.weight": "model-00094-of-00101.safetensors", + "model.layers.86.mlp.experts.9.gate_proj.weight": "model-00094-of-00101.safetensors", + "model.layers.86.mlp.experts.9.up_proj.weight": "model-00094-of-00101.safetensors", + "model.layers.86.mlp.experts.9.down_proj.weight": "model-00094-of-00101.safetensors", + "model.layers.86.mlp.experts.10.gate_proj.weight": "model-00094-of-00101.safetensors", + "model.layers.86.mlp.experts.10.up_proj.weight": "model-00094-of-00101.safetensors", + "model.layers.86.mlp.experts.10.down_proj.weight": "model-00094-of-00101.safetensors", + "model.layers.86.mlp.experts.11.gate_proj.weight": "model-00094-of-00101.safetensors", + "model.layers.86.mlp.experts.11.up_proj.weight": "model-00094-of-00101.safetensors", + "model.layers.86.mlp.experts.11.down_proj.weight": "model-00094-of-00101.safetensors", + "model.layers.86.mlp.experts.12.gate_proj.weight": "model-00094-of-00101.safetensors", + "model.layers.86.mlp.experts.12.up_proj.weight": "model-00094-of-00101.safetensors", + "model.layers.86.mlp.experts.12.down_proj.weight": "model-00094-of-00101.safetensors", + "model.layers.86.mlp.experts.13.gate_proj.weight": "model-00094-of-00101.safetensors", + "model.layers.86.mlp.experts.13.up_proj.weight": "model-00094-of-00101.safetensors", + "model.layers.86.mlp.experts.13.down_proj.weight": "model-00094-of-00101.safetensors", + "model.layers.86.mlp.experts.14.gate_proj.weight": "model-00094-of-00101.safetensors", + "model.layers.86.mlp.experts.14.up_proj.weight": "model-00094-of-00101.safetensors", + "model.layers.86.mlp.experts.14.down_proj.weight": "model-00094-of-00101.safetensors", + "model.layers.86.mlp.experts.15.gate_proj.weight": "model-00094-of-00101.safetensors", + "model.layers.86.mlp.experts.15.up_proj.weight": "model-00094-of-00101.safetensors", + "model.layers.86.mlp.experts.15.down_proj.weight": "model-00094-of-00101.safetensors", + "model.layers.86.mlp.experts.16.gate_proj.weight": "model-00094-of-00101.safetensors", + "model.layers.86.mlp.experts.16.up_proj.weight": "model-00094-of-00101.safetensors", + "model.layers.86.mlp.experts.16.down_proj.weight": "model-00094-of-00101.safetensors", + "model.layers.86.mlp.experts.17.gate_proj.weight": "model-00094-of-00101.safetensors", + "model.layers.86.mlp.experts.17.up_proj.weight": "model-00094-of-00101.safetensors", + "model.layers.86.mlp.experts.17.down_proj.weight": "model-00094-of-00101.safetensors", + "model.layers.86.mlp.experts.18.gate_proj.weight": "model-00094-of-00101.safetensors", + "model.layers.86.mlp.experts.18.up_proj.weight": "model-00094-of-00101.safetensors", + "model.layers.86.mlp.experts.18.down_proj.weight": "model-00094-of-00101.safetensors", + "model.layers.86.mlp.experts.19.gate_proj.weight": "model-00094-of-00101.safetensors", + "model.layers.86.mlp.experts.19.up_proj.weight": "model-00094-of-00101.safetensors", + "model.layers.86.mlp.experts.19.down_proj.weight": "model-00094-of-00101.safetensors", + "model.layers.86.mlp.experts.20.gate_proj.weight": "model-00094-of-00101.safetensors", + "model.layers.86.mlp.experts.20.up_proj.weight": "model-00094-of-00101.safetensors", + "model.layers.86.mlp.experts.20.down_proj.weight": "model-00094-of-00101.safetensors", + "model.layers.86.mlp.experts.21.gate_proj.weight": "model-00094-of-00101.safetensors", + "model.layers.86.mlp.experts.21.up_proj.weight": "model-00094-of-00101.safetensors", + "model.layers.86.mlp.experts.21.down_proj.weight": "model-00094-of-00101.safetensors", + "model.layers.86.mlp.experts.22.gate_proj.weight": "model-00094-of-00101.safetensors", + "model.layers.86.mlp.experts.22.up_proj.weight": "model-00094-of-00101.safetensors", + "model.layers.86.mlp.experts.22.down_proj.weight": "model-00094-of-00101.safetensors", + "model.layers.86.mlp.experts.23.gate_proj.weight": "model-00094-of-00101.safetensors", + "model.layers.86.mlp.experts.23.up_proj.weight": "model-00094-of-00101.safetensors", + "model.layers.86.mlp.experts.23.down_proj.weight": "model-00094-of-00101.safetensors", + "model.layers.86.mlp.experts.24.gate_proj.weight": "model-00094-of-00101.safetensors", + "model.layers.86.mlp.experts.24.up_proj.weight": "model-00094-of-00101.safetensors", + "model.layers.86.mlp.experts.24.down_proj.weight": "model-00094-of-00101.safetensors", + "model.layers.86.mlp.experts.25.gate_proj.weight": "model-00094-of-00101.safetensors", + "model.layers.86.mlp.experts.25.up_proj.weight": "model-00094-of-00101.safetensors", + "model.layers.86.mlp.experts.25.down_proj.weight": "model-00094-of-00101.safetensors", + "model.layers.86.mlp.experts.26.gate_proj.weight": "model-00094-of-00101.safetensors", + "model.layers.86.mlp.experts.26.up_proj.weight": "model-00094-of-00101.safetensors", + "model.layers.86.mlp.experts.26.down_proj.weight": "model-00094-of-00101.safetensors", + "model.layers.86.mlp.experts.27.gate_proj.weight": "model-00094-of-00101.safetensors", + "model.layers.86.mlp.experts.27.up_proj.weight": "model-00094-of-00101.safetensors", + "model.layers.86.mlp.experts.27.down_proj.weight": "model-00094-of-00101.safetensors", + "model.layers.86.mlp.experts.28.gate_proj.weight": "model-00094-of-00101.safetensors", + "model.layers.86.mlp.experts.28.up_proj.weight": "model-00094-of-00101.safetensors", + "model.layers.86.mlp.experts.28.down_proj.weight": "model-00094-of-00101.safetensors", + "model.layers.86.mlp.experts.29.gate_proj.weight": "model-00094-of-00101.safetensors", + "model.layers.86.mlp.experts.29.up_proj.weight": "model-00094-of-00101.safetensors", + "model.layers.86.mlp.experts.29.down_proj.weight": "model-00094-of-00101.safetensors", + "model.layers.86.mlp.experts.30.gate_proj.weight": "model-00094-of-00101.safetensors", + "model.layers.86.mlp.experts.30.up_proj.weight": "model-00094-of-00101.safetensors", + "model.layers.86.mlp.experts.30.down_proj.weight": "model-00094-of-00101.safetensors", + "model.layers.86.mlp.experts.31.gate_proj.weight": "model-00094-of-00101.safetensors", + "model.layers.86.mlp.experts.31.up_proj.weight": "model-00094-of-00101.safetensors", + "model.layers.86.mlp.experts.31.down_proj.weight": "model-00094-of-00101.safetensors", + "model.layers.86.mlp.experts.32.gate_proj.weight": "model-00094-of-00101.safetensors", + "model.layers.86.mlp.experts.32.up_proj.weight": "model-00094-of-00101.safetensors", + "model.layers.86.mlp.experts.32.down_proj.weight": "model-00094-of-00101.safetensors", + "model.layers.86.mlp.experts.33.gate_proj.weight": "model-00094-of-00101.safetensors", + "model.layers.86.mlp.experts.33.up_proj.weight": "model-00094-of-00101.safetensors", + "model.layers.86.mlp.experts.33.down_proj.weight": "model-00094-of-00101.safetensors", + "model.layers.86.mlp.experts.34.gate_proj.weight": "model-00094-of-00101.safetensors", + "model.layers.86.mlp.experts.34.up_proj.weight": "model-00094-of-00101.safetensors", + "model.layers.86.mlp.experts.34.down_proj.weight": "model-00094-of-00101.safetensors", + "model.layers.86.mlp.experts.35.gate_proj.weight": "model-00094-of-00101.safetensors", + "model.layers.86.mlp.experts.35.up_proj.weight": "model-00094-of-00101.safetensors", + "model.layers.86.mlp.experts.35.down_proj.weight": "model-00094-of-00101.safetensors", + "model.layers.86.mlp.experts.36.gate_proj.weight": "model-00094-of-00101.safetensors", + "model.layers.86.mlp.experts.36.up_proj.weight": "model-00094-of-00101.safetensors", + "model.layers.86.mlp.experts.36.down_proj.weight": "model-00094-of-00101.safetensors", + "model.layers.86.mlp.experts.37.gate_proj.weight": "model-00094-of-00101.safetensors", + "model.layers.86.mlp.experts.37.up_proj.weight": "model-00094-of-00101.safetensors", + "model.layers.86.mlp.experts.37.down_proj.weight": "model-00094-of-00101.safetensors", + "model.layers.86.mlp.experts.38.gate_proj.weight": "model-00094-of-00101.safetensors", + "model.layers.86.mlp.experts.38.up_proj.weight": "model-00094-of-00101.safetensors", + "model.layers.86.mlp.experts.38.down_proj.weight": "model-00094-of-00101.safetensors", + "model.layers.86.mlp.experts.39.gate_proj.weight": "model-00094-of-00101.safetensors", + "model.layers.86.mlp.experts.39.up_proj.weight": "model-00094-of-00101.safetensors", + "model.layers.86.mlp.experts.39.down_proj.weight": "model-00094-of-00101.safetensors", + "model.layers.86.mlp.experts.40.gate_proj.weight": "model-00094-of-00101.safetensors", + "model.layers.86.mlp.experts.40.up_proj.weight": "model-00094-of-00101.safetensors", + "model.layers.86.mlp.experts.40.down_proj.weight": "model-00094-of-00101.safetensors", + "model.layers.86.mlp.experts.41.gate_proj.weight": "model-00094-of-00101.safetensors", + "model.layers.86.mlp.experts.41.up_proj.weight": "model-00094-of-00101.safetensors", + "model.layers.86.mlp.experts.41.down_proj.weight": "model-00094-of-00101.safetensors", + "model.layers.86.mlp.experts.42.gate_proj.weight": "model-00094-of-00101.safetensors", + "model.layers.86.mlp.experts.42.up_proj.weight": "model-00094-of-00101.safetensors", + "model.layers.86.mlp.experts.42.down_proj.weight": "model-00094-of-00101.safetensors", + "model.layers.86.mlp.experts.43.gate_proj.weight": "model-00094-of-00101.safetensors", + "model.layers.86.mlp.experts.43.up_proj.weight": "model-00094-of-00101.safetensors", + "model.layers.86.mlp.experts.43.down_proj.weight": "model-00094-of-00101.safetensors", + "model.layers.86.mlp.experts.44.gate_proj.weight": "model-00094-of-00101.safetensors", + "model.layers.86.mlp.experts.44.up_proj.weight": "model-00094-of-00101.safetensors", + "model.layers.86.mlp.experts.44.down_proj.weight": "model-00094-of-00101.safetensors", + "model.layers.86.mlp.experts.45.gate_proj.weight": "model-00094-of-00101.safetensors", + "model.layers.86.mlp.experts.45.up_proj.weight": "model-00094-of-00101.safetensors", + "model.layers.86.mlp.experts.45.down_proj.weight": "model-00094-of-00101.safetensors", + "model.layers.86.mlp.experts.46.gate_proj.weight": "model-00094-of-00101.safetensors", + "model.layers.86.mlp.experts.46.up_proj.weight": "model-00094-of-00101.safetensors", + "model.layers.86.mlp.experts.46.down_proj.weight": "model-00094-of-00101.safetensors", + "model.layers.86.mlp.experts.47.gate_proj.weight": "model-00094-of-00101.safetensors", + "model.layers.86.mlp.experts.47.up_proj.weight": "model-00094-of-00101.safetensors", + "model.layers.86.mlp.experts.47.down_proj.weight": "model-00094-of-00101.safetensors", + "model.layers.86.mlp.experts.48.gate_proj.weight": "model-00094-of-00101.safetensors", + "model.layers.86.mlp.experts.48.up_proj.weight": "model-00094-of-00101.safetensors", + "model.layers.86.mlp.experts.48.down_proj.weight": "model-00094-of-00101.safetensors", + "model.layers.86.mlp.experts.49.gate_proj.weight": "model-00094-of-00101.safetensors", + "model.layers.86.mlp.experts.49.up_proj.weight": "model-00094-of-00101.safetensors", + "model.layers.86.mlp.experts.49.down_proj.weight": "model-00094-of-00101.safetensors", + "model.layers.86.mlp.experts.50.gate_proj.weight": "model-00094-of-00101.safetensors", + "model.layers.86.mlp.experts.50.up_proj.weight": "model-00094-of-00101.safetensors", + "model.layers.86.mlp.experts.50.down_proj.weight": "model-00094-of-00101.safetensors", + "model.layers.86.mlp.experts.51.gate_proj.weight": "model-00094-of-00101.safetensors", + "model.layers.86.mlp.experts.51.up_proj.weight": "model-00094-of-00101.safetensors", + "model.layers.86.mlp.experts.51.down_proj.weight": "model-00094-of-00101.safetensors", + "model.layers.86.mlp.experts.52.gate_proj.weight": "model-00094-of-00101.safetensors", + "model.layers.86.mlp.experts.52.up_proj.weight": "model-00094-of-00101.safetensors", + "model.layers.86.mlp.experts.52.down_proj.weight": "model-00094-of-00101.safetensors", + "model.layers.86.mlp.experts.53.gate_proj.weight": "model-00094-of-00101.safetensors", + "model.layers.86.mlp.experts.53.up_proj.weight": "model-00094-of-00101.safetensors", + "model.layers.86.mlp.experts.53.down_proj.weight": "model-00094-of-00101.safetensors", + "model.layers.86.mlp.experts.54.gate_proj.weight": "model-00094-of-00101.safetensors", + "model.layers.86.mlp.experts.54.up_proj.weight": "model-00094-of-00101.safetensors", + "model.layers.86.mlp.experts.54.down_proj.weight": "model-00094-of-00101.safetensors", + "model.layers.86.mlp.experts.55.gate_proj.weight": "model-00094-of-00101.safetensors", + "model.layers.86.mlp.experts.55.up_proj.weight": "model-00094-of-00101.safetensors", + "model.layers.86.mlp.experts.55.down_proj.weight": "model-00094-of-00101.safetensors", + "model.layers.86.mlp.experts.56.gate_proj.weight": "model-00094-of-00101.safetensors", + "model.layers.86.mlp.experts.56.up_proj.weight": "model-00094-of-00101.safetensors", + "model.layers.86.mlp.experts.56.down_proj.weight": "model-00094-of-00101.safetensors", + "model.layers.86.mlp.experts.57.gate_proj.weight": "model-00094-of-00101.safetensors", + "model.layers.86.mlp.experts.57.up_proj.weight": "model-00094-of-00101.safetensors", + "model.layers.86.mlp.experts.57.down_proj.weight": "model-00094-of-00101.safetensors", + "model.layers.86.mlp.experts.58.gate_proj.weight": "model-00094-of-00101.safetensors", + "model.layers.86.mlp.experts.58.up_proj.weight": "model-00094-of-00101.safetensors", + "model.layers.86.mlp.experts.58.down_proj.weight": "model-00094-of-00101.safetensors", + "model.layers.86.mlp.experts.59.gate_proj.weight": "model-00094-of-00101.safetensors", + "model.layers.86.mlp.experts.59.up_proj.weight": "model-00094-of-00101.safetensors", + "model.layers.86.mlp.experts.59.down_proj.weight": "model-00094-of-00101.safetensors", + "model.layers.86.mlp.experts.60.gate_proj.weight": "model-00094-of-00101.safetensors", + "model.layers.86.mlp.experts.60.up_proj.weight": "model-00094-of-00101.safetensors", + "model.layers.86.mlp.experts.60.down_proj.weight": "model-00094-of-00101.safetensors", + "model.layers.86.mlp.experts.61.gate_proj.weight": "model-00094-of-00101.safetensors", + "model.layers.86.mlp.experts.61.up_proj.weight": "model-00095-of-00101.safetensors", + "model.layers.86.mlp.experts.61.down_proj.weight": "model-00095-of-00101.safetensors", + "model.layers.86.mlp.experts.62.gate_proj.weight": "model-00095-of-00101.safetensors", + "model.layers.86.mlp.experts.62.up_proj.weight": "model-00095-of-00101.safetensors", + "model.layers.86.mlp.experts.62.down_proj.weight": "model-00095-of-00101.safetensors", + "model.layers.86.mlp.experts.63.gate_proj.weight": "model-00095-of-00101.safetensors", + "model.layers.86.mlp.experts.63.up_proj.weight": "model-00095-of-00101.safetensors", + "model.layers.86.mlp.experts.63.down_proj.weight": "model-00095-of-00101.safetensors", + "model.layers.86.mlp.experts.64.gate_proj.weight": "model-00095-of-00101.safetensors", + "model.layers.86.mlp.experts.64.up_proj.weight": "model-00095-of-00101.safetensors", + "model.layers.86.mlp.experts.64.down_proj.weight": "model-00095-of-00101.safetensors", + "model.layers.86.mlp.experts.65.gate_proj.weight": "model-00095-of-00101.safetensors", + "model.layers.86.mlp.experts.65.up_proj.weight": "model-00095-of-00101.safetensors", + "model.layers.86.mlp.experts.65.down_proj.weight": "model-00095-of-00101.safetensors", + "model.layers.86.mlp.experts.66.gate_proj.weight": "model-00095-of-00101.safetensors", + "model.layers.86.mlp.experts.66.up_proj.weight": "model-00095-of-00101.safetensors", + "model.layers.86.mlp.experts.66.down_proj.weight": "model-00095-of-00101.safetensors", + "model.layers.86.mlp.experts.67.gate_proj.weight": "model-00095-of-00101.safetensors", + "model.layers.86.mlp.experts.67.up_proj.weight": "model-00095-of-00101.safetensors", + "model.layers.86.mlp.experts.67.down_proj.weight": "model-00095-of-00101.safetensors", + "model.layers.86.mlp.experts.68.gate_proj.weight": "model-00095-of-00101.safetensors", + "model.layers.86.mlp.experts.68.up_proj.weight": "model-00095-of-00101.safetensors", + "model.layers.86.mlp.experts.68.down_proj.weight": "model-00095-of-00101.safetensors", + "model.layers.86.mlp.experts.69.gate_proj.weight": "model-00095-of-00101.safetensors", + "model.layers.86.mlp.experts.69.up_proj.weight": "model-00095-of-00101.safetensors", + "model.layers.86.mlp.experts.69.down_proj.weight": "model-00095-of-00101.safetensors", + "model.layers.86.mlp.experts.70.gate_proj.weight": "model-00095-of-00101.safetensors", + "model.layers.86.mlp.experts.70.up_proj.weight": "model-00095-of-00101.safetensors", + "model.layers.86.mlp.experts.70.down_proj.weight": "model-00095-of-00101.safetensors", + "model.layers.86.mlp.experts.71.gate_proj.weight": "model-00095-of-00101.safetensors", + "model.layers.86.mlp.experts.71.up_proj.weight": "model-00095-of-00101.safetensors", + "model.layers.86.mlp.experts.71.down_proj.weight": "model-00095-of-00101.safetensors", + "model.layers.86.mlp.experts.72.gate_proj.weight": "model-00095-of-00101.safetensors", + "model.layers.86.mlp.experts.72.up_proj.weight": "model-00095-of-00101.safetensors", + "model.layers.86.mlp.experts.72.down_proj.weight": "model-00095-of-00101.safetensors", + "model.layers.86.mlp.experts.73.gate_proj.weight": "model-00095-of-00101.safetensors", + "model.layers.86.mlp.experts.73.up_proj.weight": "model-00095-of-00101.safetensors", + "model.layers.86.mlp.experts.73.down_proj.weight": "model-00095-of-00101.safetensors", + "model.layers.86.mlp.experts.74.gate_proj.weight": "model-00095-of-00101.safetensors", + "model.layers.86.mlp.experts.74.up_proj.weight": "model-00095-of-00101.safetensors", + "model.layers.86.mlp.experts.74.down_proj.weight": "model-00095-of-00101.safetensors", + "model.layers.86.mlp.experts.75.gate_proj.weight": "model-00095-of-00101.safetensors", + "model.layers.86.mlp.experts.75.up_proj.weight": "model-00095-of-00101.safetensors", + "model.layers.86.mlp.experts.75.down_proj.weight": "model-00095-of-00101.safetensors", + "model.layers.86.mlp.experts.76.gate_proj.weight": "model-00095-of-00101.safetensors", + "model.layers.86.mlp.experts.76.up_proj.weight": "model-00095-of-00101.safetensors", + "model.layers.86.mlp.experts.76.down_proj.weight": "model-00095-of-00101.safetensors", + "model.layers.86.mlp.experts.77.gate_proj.weight": "model-00095-of-00101.safetensors", + "model.layers.86.mlp.experts.77.up_proj.weight": "model-00095-of-00101.safetensors", + "model.layers.86.mlp.experts.77.down_proj.weight": "model-00095-of-00101.safetensors", + "model.layers.86.mlp.experts.78.gate_proj.weight": "model-00095-of-00101.safetensors", + "model.layers.86.mlp.experts.78.up_proj.weight": "model-00095-of-00101.safetensors", + "model.layers.86.mlp.experts.78.down_proj.weight": "model-00095-of-00101.safetensors", + "model.layers.86.mlp.experts.79.gate_proj.weight": "model-00095-of-00101.safetensors", + "model.layers.86.mlp.experts.79.up_proj.weight": "model-00095-of-00101.safetensors", + "model.layers.86.mlp.experts.79.down_proj.weight": "model-00095-of-00101.safetensors", + "model.layers.86.mlp.experts.80.gate_proj.weight": "model-00095-of-00101.safetensors", + "model.layers.86.mlp.experts.80.up_proj.weight": "model-00095-of-00101.safetensors", + "model.layers.86.mlp.experts.80.down_proj.weight": "model-00095-of-00101.safetensors", + "model.layers.86.mlp.experts.81.gate_proj.weight": "model-00095-of-00101.safetensors", + "model.layers.86.mlp.experts.81.up_proj.weight": "model-00095-of-00101.safetensors", + "model.layers.86.mlp.experts.81.down_proj.weight": "model-00095-of-00101.safetensors", + "model.layers.86.mlp.experts.82.gate_proj.weight": "model-00095-of-00101.safetensors", + "model.layers.86.mlp.experts.82.up_proj.weight": "model-00095-of-00101.safetensors", + "model.layers.86.mlp.experts.82.down_proj.weight": "model-00095-of-00101.safetensors", + "model.layers.86.mlp.experts.83.gate_proj.weight": "model-00095-of-00101.safetensors", + "model.layers.86.mlp.experts.83.up_proj.weight": "model-00095-of-00101.safetensors", + "model.layers.86.mlp.experts.83.down_proj.weight": "model-00095-of-00101.safetensors", + "model.layers.86.mlp.experts.84.gate_proj.weight": "model-00095-of-00101.safetensors", + "model.layers.86.mlp.experts.84.up_proj.weight": "model-00095-of-00101.safetensors", + "model.layers.86.mlp.experts.84.down_proj.weight": "model-00095-of-00101.safetensors", + "model.layers.86.mlp.experts.85.gate_proj.weight": "model-00095-of-00101.safetensors", + "model.layers.86.mlp.experts.85.up_proj.weight": "model-00095-of-00101.safetensors", + "model.layers.86.mlp.experts.85.down_proj.weight": "model-00095-of-00101.safetensors", + "model.layers.86.mlp.experts.86.gate_proj.weight": "model-00095-of-00101.safetensors", + "model.layers.86.mlp.experts.86.up_proj.weight": "model-00095-of-00101.safetensors", + "model.layers.86.mlp.experts.86.down_proj.weight": "model-00095-of-00101.safetensors", + "model.layers.86.mlp.experts.87.gate_proj.weight": "model-00095-of-00101.safetensors", + "model.layers.86.mlp.experts.87.up_proj.weight": "model-00095-of-00101.safetensors", + "model.layers.86.mlp.experts.87.down_proj.weight": "model-00095-of-00101.safetensors", + "model.layers.86.mlp.experts.88.gate_proj.weight": "model-00095-of-00101.safetensors", + "model.layers.86.mlp.experts.88.up_proj.weight": "model-00095-of-00101.safetensors", + "model.layers.86.mlp.experts.88.down_proj.weight": "model-00095-of-00101.safetensors", + "model.layers.86.mlp.experts.89.gate_proj.weight": "model-00095-of-00101.safetensors", + "model.layers.86.mlp.experts.89.up_proj.weight": "model-00095-of-00101.safetensors", + "model.layers.86.mlp.experts.89.down_proj.weight": "model-00095-of-00101.safetensors", + "model.layers.86.mlp.experts.90.gate_proj.weight": "model-00095-of-00101.safetensors", + "model.layers.86.mlp.experts.90.up_proj.weight": "model-00095-of-00101.safetensors", + "model.layers.86.mlp.experts.90.down_proj.weight": "model-00095-of-00101.safetensors", + "model.layers.86.mlp.experts.91.gate_proj.weight": "model-00095-of-00101.safetensors", + "model.layers.86.mlp.experts.91.up_proj.weight": "model-00095-of-00101.safetensors", + "model.layers.86.mlp.experts.91.down_proj.weight": "model-00095-of-00101.safetensors", + "model.layers.86.mlp.experts.92.gate_proj.weight": "model-00095-of-00101.safetensors", + "model.layers.86.mlp.experts.92.up_proj.weight": "model-00095-of-00101.safetensors", + "model.layers.86.mlp.experts.92.down_proj.weight": "model-00095-of-00101.safetensors", + "model.layers.86.mlp.experts.93.gate_proj.weight": "model-00095-of-00101.safetensors", + "model.layers.86.mlp.experts.93.up_proj.weight": "model-00095-of-00101.safetensors", + "model.layers.86.mlp.experts.93.down_proj.weight": "model-00095-of-00101.safetensors", + "model.layers.86.mlp.experts.94.gate_proj.weight": "model-00095-of-00101.safetensors", + "model.layers.86.mlp.experts.94.up_proj.weight": "model-00095-of-00101.safetensors", + "model.layers.86.mlp.experts.94.down_proj.weight": "model-00095-of-00101.safetensors", + "model.layers.86.mlp.experts.95.gate_proj.weight": "model-00095-of-00101.safetensors", + "model.layers.86.mlp.experts.95.up_proj.weight": "model-00095-of-00101.safetensors", + "model.layers.86.mlp.experts.95.down_proj.weight": "model-00095-of-00101.safetensors", + "model.layers.86.mlp.experts.96.gate_proj.weight": "model-00095-of-00101.safetensors", + "model.layers.86.mlp.experts.96.up_proj.weight": "model-00095-of-00101.safetensors", + "model.layers.86.mlp.experts.96.down_proj.weight": "model-00095-of-00101.safetensors", + "model.layers.86.mlp.experts.97.gate_proj.weight": "model-00095-of-00101.safetensors", + "model.layers.86.mlp.experts.97.up_proj.weight": "model-00095-of-00101.safetensors", + "model.layers.86.mlp.experts.97.down_proj.weight": "model-00095-of-00101.safetensors", + "model.layers.86.mlp.experts.98.gate_proj.weight": "model-00095-of-00101.safetensors", + "model.layers.86.mlp.experts.98.up_proj.weight": "model-00095-of-00101.safetensors", + "model.layers.86.mlp.experts.98.down_proj.weight": "model-00095-of-00101.safetensors", + "model.layers.86.mlp.experts.99.gate_proj.weight": "model-00095-of-00101.safetensors", + "model.layers.86.mlp.experts.99.up_proj.weight": "model-00095-of-00101.safetensors", + "model.layers.86.mlp.experts.99.down_proj.weight": "model-00095-of-00101.safetensors", + "model.layers.86.mlp.experts.100.gate_proj.weight": "model-00095-of-00101.safetensors", + "model.layers.86.mlp.experts.100.up_proj.weight": "model-00095-of-00101.safetensors", + "model.layers.86.mlp.experts.100.down_proj.weight": "model-00095-of-00101.safetensors", + "model.layers.86.mlp.experts.101.gate_proj.weight": "model-00095-of-00101.safetensors", + "model.layers.86.mlp.experts.101.up_proj.weight": "model-00095-of-00101.safetensors", + "model.layers.86.mlp.experts.101.down_proj.weight": "model-00095-of-00101.safetensors", + "model.layers.86.mlp.experts.102.gate_proj.weight": "model-00095-of-00101.safetensors", + "model.layers.86.mlp.experts.102.up_proj.weight": "model-00095-of-00101.safetensors", + "model.layers.86.mlp.experts.102.down_proj.weight": "model-00095-of-00101.safetensors", + "model.layers.86.mlp.experts.103.gate_proj.weight": "model-00095-of-00101.safetensors", + "model.layers.86.mlp.experts.103.up_proj.weight": "model-00095-of-00101.safetensors", + "model.layers.86.mlp.experts.103.down_proj.weight": "model-00095-of-00101.safetensors", + "model.layers.86.mlp.experts.104.gate_proj.weight": "model-00095-of-00101.safetensors", + "model.layers.86.mlp.experts.104.up_proj.weight": "model-00095-of-00101.safetensors", + "model.layers.86.mlp.experts.104.down_proj.weight": "model-00095-of-00101.safetensors", + "model.layers.86.mlp.experts.105.gate_proj.weight": "model-00095-of-00101.safetensors", + "model.layers.86.mlp.experts.105.up_proj.weight": "model-00095-of-00101.safetensors", + "model.layers.86.mlp.experts.105.down_proj.weight": "model-00095-of-00101.safetensors", + "model.layers.86.mlp.experts.106.gate_proj.weight": "model-00095-of-00101.safetensors", + "model.layers.86.mlp.experts.106.up_proj.weight": "model-00095-of-00101.safetensors", + "model.layers.86.mlp.experts.106.down_proj.weight": "model-00095-of-00101.safetensors", + "model.layers.86.mlp.experts.107.gate_proj.weight": "model-00095-of-00101.safetensors", + "model.layers.86.mlp.experts.107.up_proj.weight": "model-00095-of-00101.safetensors", + "model.layers.86.mlp.experts.107.down_proj.weight": "model-00095-of-00101.safetensors", + "model.layers.86.mlp.experts.108.gate_proj.weight": "model-00095-of-00101.safetensors", + "model.layers.86.mlp.experts.108.up_proj.weight": "model-00095-of-00101.safetensors", + "model.layers.86.mlp.experts.108.down_proj.weight": "model-00095-of-00101.safetensors", + "model.layers.86.mlp.experts.109.gate_proj.weight": "model-00095-of-00101.safetensors", + "model.layers.86.mlp.experts.109.up_proj.weight": "model-00095-of-00101.safetensors", + "model.layers.86.mlp.experts.109.down_proj.weight": "model-00095-of-00101.safetensors", + "model.layers.86.mlp.experts.110.gate_proj.weight": "model-00095-of-00101.safetensors", + "model.layers.86.mlp.experts.110.up_proj.weight": "model-00095-of-00101.safetensors", + "model.layers.86.mlp.experts.110.down_proj.weight": "model-00095-of-00101.safetensors", + "model.layers.86.mlp.experts.111.gate_proj.weight": "model-00095-of-00101.safetensors", + "model.layers.86.mlp.experts.111.up_proj.weight": "model-00095-of-00101.safetensors", + "model.layers.86.mlp.experts.111.down_proj.weight": "model-00095-of-00101.safetensors", + "model.layers.86.mlp.experts.112.gate_proj.weight": "model-00095-of-00101.safetensors", + "model.layers.86.mlp.experts.112.up_proj.weight": "model-00095-of-00101.safetensors", + "model.layers.86.mlp.experts.112.down_proj.weight": "model-00095-of-00101.safetensors", + "model.layers.86.mlp.experts.113.gate_proj.weight": "model-00095-of-00101.safetensors", + "model.layers.86.mlp.experts.113.up_proj.weight": "model-00095-of-00101.safetensors", + "model.layers.86.mlp.experts.113.down_proj.weight": "model-00095-of-00101.safetensors", + "model.layers.86.mlp.experts.114.gate_proj.weight": "model-00095-of-00101.safetensors", + "model.layers.86.mlp.experts.114.up_proj.weight": "model-00095-of-00101.safetensors", + "model.layers.86.mlp.experts.114.down_proj.weight": "model-00095-of-00101.safetensors", + "model.layers.86.mlp.experts.115.gate_proj.weight": "model-00095-of-00101.safetensors", + "model.layers.86.mlp.experts.115.up_proj.weight": "model-00095-of-00101.safetensors", + "model.layers.86.mlp.experts.115.down_proj.weight": "model-00095-of-00101.safetensors", + "model.layers.86.mlp.experts.116.gate_proj.weight": "model-00095-of-00101.safetensors", + "model.layers.86.mlp.experts.116.up_proj.weight": "model-00095-of-00101.safetensors", + "model.layers.86.mlp.experts.116.down_proj.weight": "model-00095-of-00101.safetensors", + "model.layers.86.mlp.experts.117.gate_proj.weight": "model-00095-of-00101.safetensors", + "model.layers.86.mlp.experts.117.up_proj.weight": "model-00095-of-00101.safetensors", + "model.layers.86.mlp.experts.117.down_proj.weight": "model-00095-of-00101.safetensors", + "model.layers.86.mlp.experts.118.gate_proj.weight": "model-00095-of-00101.safetensors", + "model.layers.86.mlp.experts.118.up_proj.weight": "model-00095-of-00101.safetensors", + "model.layers.86.mlp.experts.118.down_proj.weight": "model-00095-of-00101.safetensors", + "model.layers.86.mlp.experts.119.gate_proj.weight": "model-00095-of-00101.safetensors", + "model.layers.86.mlp.experts.119.up_proj.weight": "model-00095-of-00101.safetensors", + "model.layers.86.mlp.experts.119.down_proj.weight": "model-00095-of-00101.safetensors", + "model.layers.86.mlp.gate.weight": "model-00095-of-00101.safetensors", + "model.layers.86.mlp.gate.e_score_correction_bias": "model-00095-of-00101.safetensors", + "model.layers.86.mlp.shared_experts.gate_proj.weight": "model-00095-of-00101.safetensors", + "model.layers.86.mlp.shared_experts.up_proj.weight": "model-00095-of-00101.safetensors", + "model.layers.86.mlp.shared_experts.down_proj.weight": "model-00095-of-00101.safetensors", + "model.layers.86.input_layernorm.weight": "model-00095-of-00101.safetensors", + "model.layers.86.post_attention_layernorm.weight": "model-00095-of-00101.safetensors", + "model.layers.87.self_attn.q_proj.weight": "model-00095-of-00101.safetensors", + "model.layers.87.self_attn.q_proj.bias": "model-00095-of-00101.safetensors", + "model.layers.87.self_attn.k_proj.weight": "model-00095-of-00101.safetensors", + "model.layers.87.self_attn.k_proj.bias": "model-00095-of-00101.safetensors", + "model.layers.87.self_attn.v_proj.weight": "model-00095-of-00101.safetensors", + "model.layers.87.self_attn.v_proj.bias": "model-00095-of-00101.safetensors", + "model.layers.87.self_attn.o_proj.weight": "model-00095-of-00101.safetensors", + "model.layers.87.self_attn.q_norm.weight": "model-00095-of-00101.safetensors", + "model.layers.87.self_attn.k_norm.weight": "model-00095-of-00101.safetensors", + "model.layers.87.mlp.experts.0.gate_proj.weight": "model-00095-of-00101.safetensors", + "model.layers.87.mlp.experts.0.up_proj.weight": "model-00095-of-00101.safetensors", + "model.layers.87.mlp.experts.0.down_proj.weight": "model-00095-of-00101.safetensors", + "model.layers.87.mlp.experts.1.gate_proj.weight": "model-00095-of-00101.safetensors", + "model.layers.87.mlp.experts.1.up_proj.weight": "model-00095-of-00101.safetensors", + "model.layers.87.mlp.experts.1.down_proj.weight": "model-00095-of-00101.safetensors", + "model.layers.87.mlp.experts.2.gate_proj.weight": "model-00095-of-00101.safetensors", + "model.layers.87.mlp.experts.2.up_proj.weight": "model-00095-of-00101.safetensors", + "model.layers.87.mlp.experts.2.down_proj.weight": "model-00095-of-00101.safetensors", + "model.layers.87.mlp.experts.3.gate_proj.weight": "model-00095-of-00101.safetensors", + "model.layers.87.mlp.experts.3.up_proj.weight": "model-00095-of-00101.safetensors", + "model.layers.87.mlp.experts.3.down_proj.weight": "model-00095-of-00101.safetensors", + "model.layers.87.mlp.experts.4.gate_proj.weight": "model-00095-of-00101.safetensors", + "model.layers.87.mlp.experts.4.up_proj.weight": "model-00095-of-00101.safetensors", + "model.layers.87.mlp.experts.4.down_proj.weight": "model-00095-of-00101.safetensors", + "model.layers.87.mlp.experts.5.gate_proj.weight": "model-00095-of-00101.safetensors", + "model.layers.87.mlp.experts.5.up_proj.weight": "model-00095-of-00101.safetensors", + "model.layers.87.mlp.experts.5.down_proj.weight": "model-00095-of-00101.safetensors", + "model.layers.87.mlp.experts.6.gate_proj.weight": "model-00095-of-00101.safetensors", + "model.layers.87.mlp.experts.6.up_proj.weight": "model-00095-of-00101.safetensors", + "model.layers.87.mlp.experts.6.down_proj.weight": "model-00095-of-00101.safetensors", + "model.layers.87.mlp.experts.7.gate_proj.weight": "model-00095-of-00101.safetensors", + "model.layers.87.mlp.experts.7.up_proj.weight": "model-00095-of-00101.safetensors", + "model.layers.87.mlp.experts.7.down_proj.weight": "model-00095-of-00101.safetensors", + "model.layers.87.mlp.experts.8.gate_proj.weight": "model-00095-of-00101.safetensors", + "model.layers.87.mlp.experts.8.up_proj.weight": "model-00095-of-00101.safetensors", + "model.layers.87.mlp.experts.8.down_proj.weight": "model-00095-of-00101.safetensors", + "model.layers.87.mlp.experts.9.gate_proj.weight": "model-00095-of-00101.safetensors", + "model.layers.87.mlp.experts.9.up_proj.weight": "model-00095-of-00101.safetensors", + "model.layers.87.mlp.experts.9.down_proj.weight": "model-00095-of-00101.safetensors", + "model.layers.87.mlp.experts.10.gate_proj.weight": "model-00095-of-00101.safetensors", + "model.layers.87.mlp.experts.10.up_proj.weight": "model-00095-of-00101.safetensors", + "model.layers.87.mlp.experts.10.down_proj.weight": "model-00095-of-00101.safetensors", + "model.layers.87.mlp.experts.11.gate_proj.weight": "model-00095-of-00101.safetensors", + "model.layers.87.mlp.experts.11.up_proj.weight": "model-00095-of-00101.safetensors", + "model.layers.87.mlp.experts.11.down_proj.weight": "model-00095-of-00101.safetensors", + "model.layers.87.mlp.experts.12.gate_proj.weight": "model-00095-of-00101.safetensors", + "model.layers.87.mlp.experts.12.up_proj.weight": "model-00095-of-00101.safetensors", + "model.layers.87.mlp.experts.12.down_proj.weight": "model-00095-of-00101.safetensors", + "model.layers.87.mlp.experts.13.gate_proj.weight": "model-00095-of-00101.safetensors", + "model.layers.87.mlp.experts.13.up_proj.weight": "model-00095-of-00101.safetensors", + "model.layers.87.mlp.experts.13.down_proj.weight": "model-00095-of-00101.safetensors", + "model.layers.87.mlp.experts.14.gate_proj.weight": "model-00095-of-00101.safetensors", + "model.layers.87.mlp.experts.14.up_proj.weight": "model-00095-of-00101.safetensors", + "model.layers.87.mlp.experts.14.down_proj.weight": "model-00095-of-00101.safetensors", + "model.layers.87.mlp.experts.15.gate_proj.weight": "model-00095-of-00101.safetensors", + "model.layers.87.mlp.experts.15.up_proj.weight": "model-00095-of-00101.safetensors", + "model.layers.87.mlp.experts.15.down_proj.weight": "model-00095-of-00101.safetensors", + "model.layers.87.mlp.experts.16.gate_proj.weight": "model-00095-of-00101.safetensors", + "model.layers.87.mlp.experts.16.up_proj.weight": "model-00095-of-00101.safetensors", + "model.layers.87.mlp.experts.16.down_proj.weight": "model-00095-of-00101.safetensors", + "model.layers.87.mlp.experts.17.gate_proj.weight": "model-00095-of-00101.safetensors", + "model.layers.87.mlp.experts.17.up_proj.weight": "model-00095-of-00101.safetensors", + "model.layers.87.mlp.experts.17.down_proj.weight": "model-00095-of-00101.safetensors", + "model.layers.87.mlp.experts.18.gate_proj.weight": "model-00095-of-00101.safetensors", + "model.layers.87.mlp.experts.18.up_proj.weight": "model-00095-of-00101.safetensors", + "model.layers.87.mlp.experts.18.down_proj.weight": "model-00095-of-00101.safetensors", + "model.layers.87.mlp.experts.19.gate_proj.weight": "model-00095-of-00101.safetensors", + "model.layers.87.mlp.experts.19.up_proj.weight": "model-00095-of-00101.safetensors", + "model.layers.87.mlp.experts.19.down_proj.weight": "model-00095-of-00101.safetensors", + "model.layers.87.mlp.experts.20.gate_proj.weight": "model-00095-of-00101.safetensors", + "model.layers.87.mlp.experts.20.up_proj.weight": "model-00095-of-00101.safetensors", + "model.layers.87.mlp.experts.20.down_proj.weight": "model-00095-of-00101.safetensors", + "model.layers.87.mlp.experts.21.gate_proj.weight": "model-00095-of-00101.safetensors", + "model.layers.87.mlp.experts.21.up_proj.weight": "model-00095-of-00101.safetensors", + "model.layers.87.mlp.experts.21.down_proj.weight": "model-00095-of-00101.safetensors", + "model.layers.87.mlp.experts.22.gate_proj.weight": "model-00095-of-00101.safetensors", + "model.layers.87.mlp.experts.22.up_proj.weight": "model-00095-of-00101.safetensors", + "model.layers.87.mlp.experts.22.down_proj.weight": "model-00095-of-00101.safetensors", + "model.layers.87.mlp.experts.23.gate_proj.weight": "model-00095-of-00101.safetensors", + "model.layers.87.mlp.experts.23.up_proj.weight": "model-00095-of-00101.safetensors", + "model.layers.87.mlp.experts.23.down_proj.weight": "model-00095-of-00101.safetensors", + "model.layers.87.mlp.experts.24.gate_proj.weight": "model-00095-of-00101.safetensors", + "model.layers.87.mlp.experts.24.up_proj.weight": "model-00095-of-00101.safetensors", + "model.layers.87.mlp.experts.24.down_proj.weight": "model-00095-of-00101.safetensors", + "model.layers.87.mlp.experts.25.gate_proj.weight": "model-00095-of-00101.safetensors", + "model.layers.87.mlp.experts.25.up_proj.weight": "model-00095-of-00101.safetensors", + "model.layers.87.mlp.experts.25.down_proj.weight": "model-00095-of-00101.safetensors", + "model.layers.87.mlp.experts.26.gate_proj.weight": "model-00095-of-00101.safetensors", + "model.layers.87.mlp.experts.26.up_proj.weight": "model-00095-of-00101.safetensors", + "model.layers.87.mlp.experts.26.down_proj.weight": "model-00095-of-00101.safetensors", + "model.layers.87.mlp.experts.27.gate_proj.weight": "model-00095-of-00101.safetensors", + "model.layers.87.mlp.experts.27.up_proj.weight": "model-00095-of-00101.safetensors", + "model.layers.87.mlp.experts.27.down_proj.weight": "model-00095-of-00101.safetensors", + "model.layers.87.mlp.experts.28.gate_proj.weight": "model-00095-of-00101.safetensors", + "model.layers.87.mlp.experts.28.up_proj.weight": "model-00095-of-00101.safetensors", + "model.layers.87.mlp.experts.28.down_proj.weight": "model-00095-of-00101.safetensors", + "model.layers.87.mlp.experts.29.gate_proj.weight": "model-00095-of-00101.safetensors", + "model.layers.87.mlp.experts.29.up_proj.weight": "model-00095-of-00101.safetensors", + "model.layers.87.mlp.experts.29.down_proj.weight": "model-00095-of-00101.safetensors", + "model.layers.87.mlp.experts.30.gate_proj.weight": "model-00095-of-00101.safetensors", + "model.layers.87.mlp.experts.30.up_proj.weight": "model-00095-of-00101.safetensors", + "model.layers.87.mlp.experts.30.down_proj.weight": "model-00095-of-00101.safetensors", + "model.layers.87.mlp.experts.31.gate_proj.weight": "model-00095-of-00101.safetensors", + "model.layers.87.mlp.experts.31.up_proj.weight": "model-00095-of-00101.safetensors", + "model.layers.87.mlp.experts.31.down_proj.weight": "model-00095-of-00101.safetensors", + "model.layers.87.mlp.experts.32.gate_proj.weight": "model-00095-of-00101.safetensors", + "model.layers.87.mlp.experts.32.up_proj.weight": "model-00095-of-00101.safetensors", + "model.layers.87.mlp.experts.32.down_proj.weight": "model-00095-of-00101.safetensors", + "model.layers.87.mlp.experts.33.gate_proj.weight": "model-00095-of-00101.safetensors", + "model.layers.87.mlp.experts.33.up_proj.weight": "model-00095-of-00101.safetensors", + "model.layers.87.mlp.experts.33.down_proj.weight": "model-00095-of-00101.safetensors", + "model.layers.87.mlp.experts.34.gate_proj.weight": "model-00095-of-00101.safetensors", + "model.layers.87.mlp.experts.34.up_proj.weight": "model-00095-of-00101.safetensors", + "model.layers.87.mlp.experts.34.down_proj.weight": "model-00095-of-00101.safetensors", + "model.layers.87.mlp.experts.35.gate_proj.weight": "model-00095-of-00101.safetensors", + "model.layers.87.mlp.experts.35.up_proj.weight": "model-00095-of-00101.safetensors", + "model.layers.87.mlp.experts.35.down_proj.weight": "model-00095-of-00101.safetensors", + "model.layers.87.mlp.experts.36.gate_proj.weight": "model-00095-of-00101.safetensors", + "model.layers.87.mlp.experts.36.up_proj.weight": "model-00095-of-00101.safetensors", + "model.layers.87.mlp.experts.36.down_proj.weight": "model-00095-of-00101.safetensors", + "model.layers.87.mlp.experts.37.gate_proj.weight": "model-00095-of-00101.safetensors", + "model.layers.87.mlp.experts.37.up_proj.weight": "model-00095-of-00101.safetensors", + "model.layers.87.mlp.experts.37.down_proj.weight": "model-00095-of-00101.safetensors", + "model.layers.87.mlp.experts.38.gate_proj.weight": "model-00095-of-00101.safetensors", + "model.layers.87.mlp.experts.38.up_proj.weight": "model-00095-of-00101.safetensors", + "model.layers.87.mlp.experts.38.down_proj.weight": "model-00095-of-00101.safetensors", + "model.layers.87.mlp.experts.39.gate_proj.weight": "model-00095-of-00101.safetensors", + "model.layers.87.mlp.experts.39.up_proj.weight": "model-00095-of-00101.safetensors", + "model.layers.87.mlp.experts.39.down_proj.weight": "model-00095-of-00101.safetensors", + "model.layers.87.mlp.experts.40.gate_proj.weight": "model-00095-of-00101.safetensors", + "model.layers.87.mlp.experts.40.up_proj.weight": "model-00095-of-00101.safetensors", + "model.layers.87.mlp.experts.40.down_proj.weight": "model-00095-of-00101.safetensors", + "model.layers.87.mlp.experts.41.gate_proj.weight": "model-00095-of-00101.safetensors", + "model.layers.87.mlp.experts.41.up_proj.weight": "model-00095-of-00101.safetensors", + "model.layers.87.mlp.experts.41.down_proj.weight": "model-00095-of-00101.safetensors", + "model.layers.87.mlp.experts.42.gate_proj.weight": "model-00095-of-00101.safetensors", + "model.layers.87.mlp.experts.42.up_proj.weight": "model-00095-of-00101.safetensors", + "model.layers.87.mlp.experts.42.down_proj.weight": "model-00095-of-00101.safetensors", + "model.layers.87.mlp.experts.43.gate_proj.weight": "model-00095-of-00101.safetensors", + "model.layers.87.mlp.experts.43.up_proj.weight": "model-00095-of-00101.safetensors", + "model.layers.87.mlp.experts.43.down_proj.weight": "model-00095-of-00101.safetensors", + "model.layers.87.mlp.experts.44.gate_proj.weight": "model-00095-of-00101.safetensors", + "model.layers.87.mlp.experts.44.up_proj.weight": "model-00095-of-00101.safetensors", + "model.layers.87.mlp.experts.44.down_proj.weight": "model-00095-of-00101.safetensors", + "model.layers.87.mlp.experts.45.gate_proj.weight": "model-00095-of-00101.safetensors", + "model.layers.87.mlp.experts.45.up_proj.weight": "model-00095-of-00101.safetensors", + "model.layers.87.mlp.experts.45.down_proj.weight": "model-00095-of-00101.safetensors", + "model.layers.87.mlp.experts.46.gate_proj.weight": "model-00095-of-00101.safetensors", + "model.layers.87.mlp.experts.46.up_proj.weight": "model-00095-of-00101.safetensors", + "model.layers.87.mlp.experts.46.down_proj.weight": "model-00095-of-00101.safetensors", + "model.layers.87.mlp.experts.47.gate_proj.weight": "model-00095-of-00101.safetensors", + "model.layers.87.mlp.experts.47.up_proj.weight": "model-00095-of-00101.safetensors", + "model.layers.87.mlp.experts.47.down_proj.weight": "model-00095-of-00101.safetensors", + "model.layers.87.mlp.experts.48.gate_proj.weight": "model-00096-of-00101.safetensors", + "model.layers.87.mlp.experts.48.up_proj.weight": "model-00096-of-00101.safetensors", + "model.layers.87.mlp.experts.48.down_proj.weight": "model-00096-of-00101.safetensors", + "model.layers.87.mlp.experts.49.gate_proj.weight": "model-00096-of-00101.safetensors", + "model.layers.87.mlp.experts.49.up_proj.weight": "model-00096-of-00101.safetensors", + "model.layers.87.mlp.experts.49.down_proj.weight": "model-00096-of-00101.safetensors", + "model.layers.87.mlp.experts.50.gate_proj.weight": "model-00096-of-00101.safetensors", + "model.layers.87.mlp.experts.50.up_proj.weight": "model-00096-of-00101.safetensors", + "model.layers.87.mlp.experts.50.down_proj.weight": "model-00096-of-00101.safetensors", + "model.layers.87.mlp.experts.51.gate_proj.weight": "model-00096-of-00101.safetensors", + "model.layers.87.mlp.experts.51.up_proj.weight": "model-00096-of-00101.safetensors", + "model.layers.87.mlp.experts.51.down_proj.weight": "model-00096-of-00101.safetensors", + "model.layers.87.mlp.experts.52.gate_proj.weight": "model-00096-of-00101.safetensors", + "model.layers.87.mlp.experts.52.up_proj.weight": "model-00096-of-00101.safetensors", + "model.layers.87.mlp.experts.52.down_proj.weight": "model-00096-of-00101.safetensors", + "model.layers.87.mlp.experts.53.gate_proj.weight": "model-00096-of-00101.safetensors", + "model.layers.87.mlp.experts.53.up_proj.weight": "model-00096-of-00101.safetensors", + "model.layers.87.mlp.experts.53.down_proj.weight": "model-00096-of-00101.safetensors", + "model.layers.87.mlp.experts.54.gate_proj.weight": "model-00096-of-00101.safetensors", + "model.layers.87.mlp.experts.54.up_proj.weight": "model-00096-of-00101.safetensors", + "model.layers.87.mlp.experts.54.down_proj.weight": "model-00096-of-00101.safetensors", + "model.layers.87.mlp.experts.55.gate_proj.weight": "model-00096-of-00101.safetensors", + "model.layers.87.mlp.experts.55.up_proj.weight": "model-00096-of-00101.safetensors", + "model.layers.87.mlp.experts.55.down_proj.weight": "model-00096-of-00101.safetensors", + "model.layers.87.mlp.experts.56.gate_proj.weight": "model-00096-of-00101.safetensors", + "model.layers.87.mlp.experts.56.up_proj.weight": "model-00096-of-00101.safetensors", + "model.layers.87.mlp.experts.56.down_proj.weight": "model-00096-of-00101.safetensors", + "model.layers.87.mlp.experts.57.gate_proj.weight": "model-00096-of-00101.safetensors", + "model.layers.87.mlp.experts.57.up_proj.weight": "model-00096-of-00101.safetensors", + "model.layers.87.mlp.experts.57.down_proj.weight": "model-00096-of-00101.safetensors", + "model.layers.87.mlp.experts.58.gate_proj.weight": "model-00096-of-00101.safetensors", + "model.layers.87.mlp.experts.58.up_proj.weight": "model-00096-of-00101.safetensors", + "model.layers.87.mlp.experts.58.down_proj.weight": "model-00096-of-00101.safetensors", + "model.layers.87.mlp.experts.59.gate_proj.weight": "model-00096-of-00101.safetensors", + "model.layers.87.mlp.experts.59.up_proj.weight": "model-00096-of-00101.safetensors", + "model.layers.87.mlp.experts.59.down_proj.weight": "model-00096-of-00101.safetensors", + "model.layers.87.mlp.experts.60.gate_proj.weight": "model-00096-of-00101.safetensors", + "model.layers.87.mlp.experts.60.up_proj.weight": "model-00096-of-00101.safetensors", + "model.layers.87.mlp.experts.60.down_proj.weight": "model-00096-of-00101.safetensors", + "model.layers.87.mlp.experts.61.gate_proj.weight": "model-00096-of-00101.safetensors", + "model.layers.87.mlp.experts.61.up_proj.weight": "model-00096-of-00101.safetensors", + "model.layers.87.mlp.experts.61.down_proj.weight": "model-00096-of-00101.safetensors", + "model.layers.87.mlp.experts.62.gate_proj.weight": "model-00096-of-00101.safetensors", + "model.layers.87.mlp.experts.62.up_proj.weight": "model-00096-of-00101.safetensors", + "model.layers.87.mlp.experts.62.down_proj.weight": "model-00096-of-00101.safetensors", + "model.layers.87.mlp.experts.63.gate_proj.weight": "model-00096-of-00101.safetensors", + "model.layers.87.mlp.experts.63.up_proj.weight": "model-00096-of-00101.safetensors", + "model.layers.87.mlp.experts.63.down_proj.weight": "model-00096-of-00101.safetensors", + "model.layers.87.mlp.experts.64.gate_proj.weight": "model-00096-of-00101.safetensors", + "model.layers.87.mlp.experts.64.up_proj.weight": "model-00096-of-00101.safetensors", + "model.layers.87.mlp.experts.64.down_proj.weight": "model-00096-of-00101.safetensors", + "model.layers.87.mlp.experts.65.gate_proj.weight": "model-00096-of-00101.safetensors", + "model.layers.87.mlp.experts.65.up_proj.weight": "model-00096-of-00101.safetensors", + "model.layers.87.mlp.experts.65.down_proj.weight": "model-00096-of-00101.safetensors", + "model.layers.87.mlp.experts.66.gate_proj.weight": "model-00096-of-00101.safetensors", + "model.layers.87.mlp.experts.66.up_proj.weight": "model-00096-of-00101.safetensors", + "model.layers.87.mlp.experts.66.down_proj.weight": "model-00096-of-00101.safetensors", + "model.layers.87.mlp.experts.67.gate_proj.weight": "model-00096-of-00101.safetensors", + "model.layers.87.mlp.experts.67.up_proj.weight": "model-00096-of-00101.safetensors", + "model.layers.87.mlp.experts.67.down_proj.weight": "model-00096-of-00101.safetensors", + "model.layers.87.mlp.experts.68.gate_proj.weight": "model-00096-of-00101.safetensors", + "model.layers.87.mlp.experts.68.up_proj.weight": "model-00096-of-00101.safetensors", + "model.layers.87.mlp.experts.68.down_proj.weight": "model-00096-of-00101.safetensors", + "model.layers.87.mlp.experts.69.gate_proj.weight": "model-00096-of-00101.safetensors", + "model.layers.87.mlp.experts.69.up_proj.weight": "model-00096-of-00101.safetensors", + "model.layers.87.mlp.experts.69.down_proj.weight": "model-00096-of-00101.safetensors", + "model.layers.87.mlp.experts.70.gate_proj.weight": "model-00096-of-00101.safetensors", + "model.layers.87.mlp.experts.70.up_proj.weight": "model-00096-of-00101.safetensors", + "model.layers.87.mlp.experts.70.down_proj.weight": "model-00096-of-00101.safetensors", + "model.layers.87.mlp.experts.71.gate_proj.weight": "model-00096-of-00101.safetensors", + "model.layers.87.mlp.experts.71.up_proj.weight": "model-00096-of-00101.safetensors", + "model.layers.87.mlp.experts.71.down_proj.weight": "model-00096-of-00101.safetensors", + "model.layers.87.mlp.experts.72.gate_proj.weight": "model-00096-of-00101.safetensors", + "model.layers.87.mlp.experts.72.up_proj.weight": "model-00096-of-00101.safetensors", + "model.layers.87.mlp.experts.72.down_proj.weight": "model-00096-of-00101.safetensors", + "model.layers.87.mlp.experts.73.gate_proj.weight": "model-00096-of-00101.safetensors", + "model.layers.87.mlp.experts.73.up_proj.weight": "model-00096-of-00101.safetensors", + "model.layers.87.mlp.experts.73.down_proj.weight": "model-00096-of-00101.safetensors", + "model.layers.87.mlp.experts.74.gate_proj.weight": "model-00096-of-00101.safetensors", + "model.layers.87.mlp.experts.74.up_proj.weight": "model-00096-of-00101.safetensors", + "model.layers.87.mlp.experts.74.down_proj.weight": "model-00096-of-00101.safetensors", + "model.layers.87.mlp.experts.75.gate_proj.weight": "model-00096-of-00101.safetensors", + "model.layers.87.mlp.experts.75.up_proj.weight": "model-00096-of-00101.safetensors", + "model.layers.87.mlp.experts.75.down_proj.weight": "model-00096-of-00101.safetensors", + "model.layers.87.mlp.experts.76.gate_proj.weight": "model-00096-of-00101.safetensors", + "model.layers.87.mlp.experts.76.up_proj.weight": "model-00096-of-00101.safetensors", + "model.layers.87.mlp.experts.76.down_proj.weight": "model-00096-of-00101.safetensors", + "model.layers.87.mlp.experts.77.gate_proj.weight": "model-00096-of-00101.safetensors", + "model.layers.87.mlp.experts.77.up_proj.weight": "model-00096-of-00101.safetensors", + "model.layers.87.mlp.experts.77.down_proj.weight": "model-00096-of-00101.safetensors", + "model.layers.87.mlp.experts.78.gate_proj.weight": "model-00096-of-00101.safetensors", + "model.layers.87.mlp.experts.78.up_proj.weight": "model-00096-of-00101.safetensors", + "model.layers.87.mlp.experts.78.down_proj.weight": "model-00096-of-00101.safetensors", + "model.layers.87.mlp.experts.79.gate_proj.weight": "model-00096-of-00101.safetensors", + "model.layers.87.mlp.experts.79.up_proj.weight": "model-00096-of-00101.safetensors", + "model.layers.87.mlp.experts.79.down_proj.weight": "model-00096-of-00101.safetensors", + "model.layers.87.mlp.experts.80.gate_proj.weight": "model-00096-of-00101.safetensors", + "model.layers.87.mlp.experts.80.up_proj.weight": "model-00096-of-00101.safetensors", + "model.layers.87.mlp.experts.80.down_proj.weight": "model-00096-of-00101.safetensors", + "model.layers.87.mlp.experts.81.gate_proj.weight": "model-00096-of-00101.safetensors", + "model.layers.87.mlp.experts.81.up_proj.weight": "model-00096-of-00101.safetensors", + "model.layers.87.mlp.experts.81.down_proj.weight": "model-00096-of-00101.safetensors", + "model.layers.87.mlp.experts.82.gate_proj.weight": "model-00096-of-00101.safetensors", + "model.layers.87.mlp.experts.82.up_proj.weight": "model-00096-of-00101.safetensors", + "model.layers.87.mlp.experts.82.down_proj.weight": "model-00096-of-00101.safetensors", + "model.layers.87.mlp.experts.83.gate_proj.weight": "model-00096-of-00101.safetensors", + "model.layers.87.mlp.experts.83.up_proj.weight": "model-00096-of-00101.safetensors", + "model.layers.87.mlp.experts.83.down_proj.weight": "model-00096-of-00101.safetensors", + "model.layers.87.mlp.experts.84.gate_proj.weight": "model-00096-of-00101.safetensors", + "model.layers.87.mlp.experts.84.up_proj.weight": "model-00096-of-00101.safetensors", + "model.layers.87.mlp.experts.84.down_proj.weight": "model-00096-of-00101.safetensors", + "model.layers.87.mlp.experts.85.gate_proj.weight": "model-00096-of-00101.safetensors", + "model.layers.87.mlp.experts.85.up_proj.weight": "model-00096-of-00101.safetensors", + "model.layers.87.mlp.experts.85.down_proj.weight": "model-00096-of-00101.safetensors", + "model.layers.87.mlp.experts.86.gate_proj.weight": "model-00096-of-00101.safetensors", + "model.layers.87.mlp.experts.86.up_proj.weight": "model-00096-of-00101.safetensors", + "model.layers.87.mlp.experts.86.down_proj.weight": "model-00096-of-00101.safetensors", + "model.layers.87.mlp.experts.87.gate_proj.weight": "model-00096-of-00101.safetensors", + "model.layers.87.mlp.experts.87.up_proj.weight": "model-00096-of-00101.safetensors", + "model.layers.87.mlp.experts.87.down_proj.weight": "model-00096-of-00101.safetensors", + "model.layers.87.mlp.experts.88.gate_proj.weight": "model-00096-of-00101.safetensors", + "model.layers.87.mlp.experts.88.up_proj.weight": "model-00096-of-00101.safetensors", + "model.layers.87.mlp.experts.88.down_proj.weight": "model-00096-of-00101.safetensors", + "model.layers.87.mlp.experts.89.gate_proj.weight": "model-00096-of-00101.safetensors", + "model.layers.87.mlp.experts.89.up_proj.weight": "model-00096-of-00101.safetensors", + "model.layers.87.mlp.experts.89.down_proj.weight": "model-00096-of-00101.safetensors", + "model.layers.87.mlp.experts.90.gate_proj.weight": "model-00096-of-00101.safetensors", + "model.layers.87.mlp.experts.90.up_proj.weight": "model-00096-of-00101.safetensors", + "model.layers.87.mlp.experts.90.down_proj.weight": "model-00096-of-00101.safetensors", + "model.layers.87.mlp.experts.91.gate_proj.weight": "model-00096-of-00101.safetensors", + "model.layers.87.mlp.experts.91.up_proj.weight": "model-00096-of-00101.safetensors", + "model.layers.87.mlp.experts.91.down_proj.weight": "model-00096-of-00101.safetensors", + "model.layers.87.mlp.experts.92.gate_proj.weight": "model-00096-of-00101.safetensors", + "model.layers.87.mlp.experts.92.up_proj.weight": "model-00096-of-00101.safetensors", + "model.layers.87.mlp.experts.92.down_proj.weight": "model-00096-of-00101.safetensors", + "model.layers.87.mlp.experts.93.gate_proj.weight": "model-00096-of-00101.safetensors", + "model.layers.87.mlp.experts.93.up_proj.weight": "model-00096-of-00101.safetensors", + "model.layers.87.mlp.experts.93.down_proj.weight": "model-00096-of-00101.safetensors", + "model.layers.87.mlp.experts.94.gate_proj.weight": "model-00096-of-00101.safetensors", + "model.layers.87.mlp.experts.94.up_proj.weight": "model-00096-of-00101.safetensors", + "model.layers.87.mlp.experts.94.down_proj.weight": "model-00096-of-00101.safetensors", + "model.layers.87.mlp.experts.95.gate_proj.weight": "model-00096-of-00101.safetensors", + "model.layers.87.mlp.experts.95.up_proj.weight": "model-00096-of-00101.safetensors", + "model.layers.87.mlp.experts.95.down_proj.weight": "model-00096-of-00101.safetensors", + "model.layers.87.mlp.experts.96.gate_proj.weight": "model-00096-of-00101.safetensors", + "model.layers.87.mlp.experts.96.up_proj.weight": "model-00096-of-00101.safetensors", + "model.layers.87.mlp.experts.96.down_proj.weight": "model-00096-of-00101.safetensors", + "model.layers.87.mlp.experts.97.gate_proj.weight": "model-00096-of-00101.safetensors", + "model.layers.87.mlp.experts.97.up_proj.weight": "model-00096-of-00101.safetensors", + "model.layers.87.mlp.experts.97.down_proj.weight": "model-00096-of-00101.safetensors", + "model.layers.87.mlp.experts.98.gate_proj.weight": "model-00096-of-00101.safetensors", + "model.layers.87.mlp.experts.98.up_proj.weight": "model-00096-of-00101.safetensors", + "model.layers.87.mlp.experts.98.down_proj.weight": "model-00096-of-00101.safetensors", + "model.layers.87.mlp.experts.99.gate_proj.weight": "model-00096-of-00101.safetensors", + "model.layers.87.mlp.experts.99.up_proj.weight": "model-00096-of-00101.safetensors", + "model.layers.87.mlp.experts.99.down_proj.weight": "model-00096-of-00101.safetensors", + "model.layers.87.mlp.experts.100.gate_proj.weight": "model-00096-of-00101.safetensors", + "model.layers.87.mlp.experts.100.up_proj.weight": "model-00096-of-00101.safetensors", + "model.layers.87.mlp.experts.100.down_proj.weight": "model-00096-of-00101.safetensors", + "model.layers.87.mlp.experts.101.gate_proj.weight": "model-00096-of-00101.safetensors", + "model.layers.87.mlp.experts.101.up_proj.weight": "model-00096-of-00101.safetensors", + "model.layers.87.mlp.experts.101.down_proj.weight": "model-00096-of-00101.safetensors", + "model.layers.87.mlp.experts.102.gate_proj.weight": "model-00096-of-00101.safetensors", + "model.layers.87.mlp.experts.102.up_proj.weight": "model-00096-of-00101.safetensors", + "model.layers.87.mlp.experts.102.down_proj.weight": "model-00096-of-00101.safetensors", + "model.layers.87.mlp.experts.103.gate_proj.weight": "model-00096-of-00101.safetensors", + "model.layers.87.mlp.experts.103.up_proj.weight": "model-00096-of-00101.safetensors", + "model.layers.87.mlp.experts.103.down_proj.weight": "model-00096-of-00101.safetensors", + "model.layers.87.mlp.experts.104.gate_proj.weight": "model-00096-of-00101.safetensors", + "model.layers.87.mlp.experts.104.up_proj.weight": "model-00096-of-00101.safetensors", + "model.layers.87.mlp.experts.104.down_proj.weight": "model-00096-of-00101.safetensors", + "model.layers.87.mlp.experts.105.gate_proj.weight": "model-00096-of-00101.safetensors", + "model.layers.87.mlp.experts.105.up_proj.weight": "model-00096-of-00101.safetensors", + "model.layers.87.mlp.experts.105.down_proj.weight": "model-00096-of-00101.safetensors", + "model.layers.87.mlp.experts.106.gate_proj.weight": "model-00096-of-00101.safetensors", + "model.layers.87.mlp.experts.106.up_proj.weight": "model-00096-of-00101.safetensors", + "model.layers.87.mlp.experts.106.down_proj.weight": "model-00096-of-00101.safetensors", + "model.layers.87.mlp.experts.107.gate_proj.weight": "model-00096-of-00101.safetensors", + "model.layers.87.mlp.experts.107.up_proj.weight": "model-00096-of-00101.safetensors", + "model.layers.87.mlp.experts.107.down_proj.weight": "model-00096-of-00101.safetensors", + "model.layers.87.mlp.experts.108.gate_proj.weight": "model-00096-of-00101.safetensors", + "model.layers.87.mlp.experts.108.up_proj.weight": "model-00096-of-00101.safetensors", + "model.layers.87.mlp.experts.108.down_proj.weight": "model-00096-of-00101.safetensors", + "model.layers.87.mlp.experts.109.gate_proj.weight": "model-00096-of-00101.safetensors", + "model.layers.87.mlp.experts.109.up_proj.weight": "model-00096-of-00101.safetensors", + "model.layers.87.mlp.experts.109.down_proj.weight": "model-00096-of-00101.safetensors", + "model.layers.87.mlp.experts.110.gate_proj.weight": "model-00096-of-00101.safetensors", + "model.layers.87.mlp.experts.110.up_proj.weight": "model-00096-of-00101.safetensors", + "model.layers.87.mlp.experts.110.down_proj.weight": "model-00096-of-00101.safetensors", + "model.layers.87.mlp.experts.111.gate_proj.weight": "model-00096-of-00101.safetensors", + "model.layers.87.mlp.experts.111.up_proj.weight": "model-00096-of-00101.safetensors", + "model.layers.87.mlp.experts.111.down_proj.weight": "model-00096-of-00101.safetensors", + "model.layers.87.mlp.experts.112.gate_proj.weight": "model-00096-of-00101.safetensors", + "model.layers.87.mlp.experts.112.up_proj.weight": "model-00096-of-00101.safetensors", + "model.layers.87.mlp.experts.112.down_proj.weight": "model-00096-of-00101.safetensors", + "model.layers.87.mlp.experts.113.gate_proj.weight": "model-00096-of-00101.safetensors", + "model.layers.87.mlp.experts.113.up_proj.weight": "model-00096-of-00101.safetensors", + "model.layers.87.mlp.experts.113.down_proj.weight": "model-00096-of-00101.safetensors", + "model.layers.87.mlp.experts.114.gate_proj.weight": "model-00096-of-00101.safetensors", + "model.layers.87.mlp.experts.114.up_proj.weight": "model-00096-of-00101.safetensors", + "model.layers.87.mlp.experts.114.down_proj.weight": "model-00096-of-00101.safetensors", + "model.layers.87.mlp.experts.115.gate_proj.weight": "model-00096-of-00101.safetensors", + "model.layers.87.mlp.experts.115.up_proj.weight": "model-00096-of-00101.safetensors", + "model.layers.87.mlp.experts.115.down_proj.weight": "model-00096-of-00101.safetensors", + "model.layers.87.mlp.experts.116.gate_proj.weight": "model-00096-of-00101.safetensors", + "model.layers.87.mlp.experts.116.up_proj.weight": "model-00096-of-00101.safetensors", + "model.layers.87.mlp.experts.116.down_proj.weight": "model-00096-of-00101.safetensors", + "model.layers.87.mlp.experts.117.gate_proj.weight": "model-00096-of-00101.safetensors", + "model.layers.87.mlp.experts.117.up_proj.weight": "model-00096-of-00101.safetensors", + "model.layers.87.mlp.experts.117.down_proj.weight": "model-00096-of-00101.safetensors", + "model.layers.87.mlp.experts.118.gate_proj.weight": "model-00096-of-00101.safetensors", + "model.layers.87.mlp.experts.118.up_proj.weight": "model-00096-of-00101.safetensors", + "model.layers.87.mlp.experts.118.down_proj.weight": "model-00096-of-00101.safetensors", + "model.layers.87.mlp.experts.119.gate_proj.weight": "model-00096-of-00101.safetensors", + "model.layers.87.mlp.experts.119.up_proj.weight": "model-00096-of-00101.safetensors", + "model.layers.87.mlp.experts.119.down_proj.weight": "model-00096-of-00101.safetensors", + "model.layers.87.mlp.gate.weight": "model-00096-of-00101.safetensors", + "model.layers.87.mlp.gate.e_score_correction_bias": "model-00096-of-00101.safetensors", + "model.layers.87.mlp.shared_experts.gate_proj.weight": "model-00096-of-00101.safetensors", + "model.layers.87.mlp.shared_experts.up_proj.weight": "model-00096-of-00101.safetensors", + "model.layers.87.mlp.shared_experts.down_proj.weight": "model-00096-of-00101.safetensors", + "model.layers.87.input_layernorm.weight": "model-00096-of-00101.safetensors", + "model.layers.87.post_attention_layernorm.weight": "model-00096-of-00101.safetensors", + "model.layers.88.self_attn.q_proj.weight": "model-00096-of-00101.safetensors", + "model.layers.88.self_attn.q_proj.bias": "model-00096-of-00101.safetensors", + "model.layers.88.self_attn.k_proj.weight": "model-00096-of-00101.safetensors", + "model.layers.88.self_attn.k_proj.bias": "model-00096-of-00101.safetensors", + "model.layers.88.self_attn.v_proj.weight": "model-00096-of-00101.safetensors", + "model.layers.88.self_attn.v_proj.bias": "model-00096-of-00101.safetensors", + "model.layers.88.self_attn.o_proj.weight": "model-00096-of-00101.safetensors", + "model.layers.88.self_attn.q_norm.weight": "model-00096-of-00101.safetensors", + "model.layers.88.self_attn.k_norm.weight": "model-00096-of-00101.safetensors", + "model.layers.88.mlp.experts.0.gate_proj.weight": "model-00096-of-00101.safetensors", + "model.layers.88.mlp.experts.0.up_proj.weight": "model-00096-of-00101.safetensors", + "model.layers.88.mlp.experts.0.down_proj.weight": "model-00096-of-00101.safetensors", + "model.layers.88.mlp.experts.1.gate_proj.weight": "model-00096-of-00101.safetensors", + "model.layers.88.mlp.experts.1.up_proj.weight": "model-00096-of-00101.safetensors", + "model.layers.88.mlp.experts.1.down_proj.weight": "model-00096-of-00101.safetensors", + "model.layers.88.mlp.experts.2.gate_proj.weight": "model-00096-of-00101.safetensors", + "model.layers.88.mlp.experts.2.up_proj.weight": "model-00096-of-00101.safetensors", + "model.layers.88.mlp.experts.2.down_proj.weight": "model-00096-of-00101.safetensors", + "model.layers.88.mlp.experts.3.gate_proj.weight": "model-00096-of-00101.safetensors", + "model.layers.88.mlp.experts.3.up_proj.weight": "model-00096-of-00101.safetensors", + "model.layers.88.mlp.experts.3.down_proj.weight": "model-00096-of-00101.safetensors", + "model.layers.88.mlp.experts.4.gate_proj.weight": "model-00096-of-00101.safetensors", + "model.layers.88.mlp.experts.4.up_proj.weight": "model-00096-of-00101.safetensors", + "model.layers.88.mlp.experts.4.down_proj.weight": "model-00096-of-00101.safetensors", + "model.layers.88.mlp.experts.5.gate_proj.weight": "model-00096-of-00101.safetensors", + "model.layers.88.mlp.experts.5.up_proj.weight": "model-00096-of-00101.safetensors", + "model.layers.88.mlp.experts.5.down_proj.weight": "model-00096-of-00101.safetensors", + "model.layers.88.mlp.experts.6.gate_proj.weight": "model-00096-of-00101.safetensors", + "model.layers.88.mlp.experts.6.up_proj.weight": "model-00096-of-00101.safetensors", + "model.layers.88.mlp.experts.6.down_proj.weight": "model-00096-of-00101.safetensors", + "model.layers.88.mlp.experts.7.gate_proj.weight": "model-00096-of-00101.safetensors", + "model.layers.88.mlp.experts.7.up_proj.weight": "model-00096-of-00101.safetensors", + "model.layers.88.mlp.experts.7.down_proj.weight": "model-00096-of-00101.safetensors", + "model.layers.88.mlp.experts.8.gate_proj.weight": "model-00096-of-00101.safetensors", + "model.layers.88.mlp.experts.8.up_proj.weight": "model-00096-of-00101.safetensors", + "model.layers.88.mlp.experts.8.down_proj.weight": "model-00096-of-00101.safetensors", + "model.layers.88.mlp.experts.9.gate_proj.weight": "model-00096-of-00101.safetensors", + "model.layers.88.mlp.experts.9.up_proj.weight": "model-00096-of-00101.safetensors", + "model.layers.88.mlp.experts.9.down_proj.weight": "model-00096-of-00101.safetensors", + "model.layers.88.mlp.experts.10.gate_proj.weight": "model-00096-of-00101.safetensors", + "model.layers.88.mlp.experts.10.up_proj.weight": "model-00096-of-00101.safetensors", + "model.layers.88.mlp.experts.10.down_proj.weight": "model-00096-of-00101.safetensors", + "model.layers.88.mlp.experts.11.gate_proj.weight": "model-00096-of-00101.safetensors", + "model.layers.88.mlp.experts.11.up_proj.weight": "model-00096-of-00101.safetensors", + "model.layers.88.mlp.experts.11.down_proj.weight": "model-00096-of-00101.safetensors", + "model.layers.88.mlp.experts.12.gate_proj.weight": "model-00096-of-00101.safetensors", + "model.layers.88.mlp.experts.12.up_proj.weight": "model-00096-of-00101.safetensors", + "model.layers.88.mlp.experts.12.down_proj.weight": "model-00096-of-00101.safetensors", + "model.layers.88.mlp.experts.13.gate_proj.weight": "model-00096-of-00101.safetensors", + "model.layers.88.mlp.experts.13.up_proj.weight": "model-00096-of-00101.safetensors", + "model.layers.88.mlp.experts.13.down_proj.weight": "model-00096-of-00101.safetensors", + "model.layers.88.mlp.experts.14.gate_proj.weight": "model-00096-of-00101.safetensors", + "model.layers.88.mlp.experts.14.up_proj.weight": "model-00096-of-00101.safetensors", + "model.layers.88.mlp.experts.14.down_proj.weight": "model-00096-of-00101.safetensors", + "model.layers.88.mlp.experts.15.gate_proj.weight": "model-00096-of-00101.safetensors", + "model.layers.88.mlp.experts.15.up_proj.weight": "model-00096-of-00101.safetensors", + "model.layers.88.mlp.experts.15.down_proj.weight": "model-00096-of-00101.safetensors", + "model.layers.88.mlp.experts.16.gate_proj.weight": "model-00096-of-00101.safetensors", + "model.layers.88.mlp.experts.16.up_proj.weight": "model-00096-of-00101.safetensors", + "model.layers.88.mlp.experts.16.down_proj.weight": "model-00096-of-00101.safetensors", + "model.layers.88.mlp.experts.17.gate_proj.weight": "model-00096-of-00101.safetensors", + "model.layers.88.mlp.experts.17.up_proj.weight": "model-00096-of-00101.safetensors", + "model.layers.88.mlp.experts.17.down_proj.weight": "model-00096-of-00101.safetensors", + "model.layers.88.mlp.experts.18.gate_proj.weight": "model-00096-of-00101.safetensors", + "model.layers.88.mlp.experts.18.up_proj.weight": "model-00096-of-00101.safetensors", + "model.layers.88.mlp.experts.18.down_proj.weight": "model-00096-of-00101.safetensors", + "model.layers.88.mlp.experts.19.gate_proj.weight": "model-00096-of-00101.safetensors", + "model.layers.88.mlp.experts.19.up_proj.weight": "model-00096-of-00101.safetensors", + "model.layers.88.mlp.experts.19.down_proj.weight": "model-00096-of-00101.safetensors", + "model.layers.88.mlp.experts.20.gate_proj.weight": "model-00096-of-00101.safetensors", + "model.layers.88.mlp.experts.20.up_proj.weight": "model-00096-of-00101.safetensors", + "model.layers.88.mlp.experts.20.down_proj.weight": "model-00096-of-00101.safetensors", + "model.layers.88.mlp.experts.21.gate_proj.weight": "model-00096-of-00101.safetensors", + "model.layers.88.mlp.experts.21.up_proj.weight": "model-00096-of-00101.safetensors", + "model.layers.88.mlp.experts.21.down_proj.weight": "model-00096-of-00101.safetensors", + "model.layers.88.mlp.experts.22.gate_proj.weight": "model-00096-of-00101.safetensors", + "model.layers.88.mlp.experts.22.up_proj.weight": "model-00096-of-00101.safetensors", + "model.layers.88.mlp.experts.22.down_proj.weight": "model-00096-of-00101.safetensors", + "model.layers.88.mlp.experts.23.gate_proj.weight": "model-00096-of-00101.safetensors", + "model.layers.88.mlp.experts.23.up_proj.weight": "model-00096-of-00101.safetensors", + "model.layers.88.mlp.experts.23.down_proj.weight": "model-00096-of-00101.safetensors", + "model.layers.88.mlp.experts.24.gate_proj.weight": "model-00096-of-00101.safetensors", + "model.layers.88.mlp.experts.24.up_proj.weight": "model-00096-of-00101.safetensors", + "model.layers.88.mlp.experts.24.down_proj.weight": "model-00096-of-00101.safetensors", + "model.layers.88.mlp.experts.25.gate_proj.weight": "model-00096-of-00101.safetensors", + "model.layers.88.mlp.experts.25.up_proj.weight": "model-00096-of-00101.safetensors", + "model.layers.88.mlp.experts.25.down_proj.weight": "model-00096-of-00101.safetensors", + "model.layers.88.mlp.experts.26.gate_proj.weight": "model-00096-of-00101.safetensors", + "model.layers.88.mlp.experts.26.up_proj.weight": "model-00096-of-00101.safetensors", + "model.layers.88.mlp.experts.26.down_proj.weight": "model-00096-of-00101.safetensors", + "model.layers.88.mlp.experts.27.gate_proj.weight": "model-00096-of-00101.safetensors", + "model.layers.88.mlp.experts.27.up_proj.weight": "model-00096-of-00101.safetensors", + "model.layers.88.mlp.experts.27.down_proj.weight": "model-00096-of-00101.safetensors", + "model.layers.88.mlp.experts.28.gate_proj.weight": "model-00096-of-00101.safetensors", + "model.layers.88.mlp.experts.28.up_proj.weight": "model-00096-of-00101.safetensors", + "model.layers.88.mlp.experts.28.down_proj.weight": "model-00096-of-00101.safetensors", + "model.layers.88.mlp.experts.29.gate_proj.weight": "model-00096-of-00101.safetensors", + "model.layers.88.mlp.experts.29.up_proj.weight": "model-00096-of-00101.safetensors", + "model.layers.88.mlp.experts.29.down_proj.weight": "model-00096-of-00101.safetensors", + "model.layers.88.mlp.experts.30.gate_proj.weight": "model-00096-of-00101.safetensors", + "model.layers.88.mlp.experts.30.up_proj.weight": "model-00096-of-00101.safetensors", + "model.layers.88.mlp.experts.30.down_proj.weight": "model-00096-of-00101.safetensors", + "model.layers.88.mlp.experts.31.gate_proj.weight": "model-00096-of-00101.safetensors", + "model.layers.88.mlp.experts.31.up_proj.weight": "model-00096-of-00101.safetensors", + "model.layers.88.mlp.experts.31.down_proj.weight": "model-00096-of-00101.safetensors", + "model.layers.88.mlp.experts.32.gate_proj.weight": "model-00096-of-00101.safetensors", + "model.layers.88.mlp.experts.32.up_proj.weight": "model-00096-of-00101.safetensors", + "model.layers.88.mlp.experts.32.down_proj.weight": "model-00096-of-00101.safetensors", + "model.layers.88.mlp.experts.33.gate_proj.weight": "model-00096-of-00101.safetensors", + "model.layers.88.mlp.experts.33.up_proj.weight": "model-00096-of-00101.safetensors", + "model.layers.88.mlp.experts.33.down_proj.weight": "model-00096-of-00101.safetensors", + "model.layers.88.mlp.experts.34.gate_proj.weight": "model-00096-of-00101.safetensors", + "model.layers.88.mlp.experts.34.up_proj.weight": "model-00096-of-00101.safetensors", + "model.layers.88.mlp.experts.34.down_proj.weight": "model-00097-of-00101.safetensors", + "model.layers.88.mlp.experts.35.gate_proj.weight": "model-00097-of-00101.safetensors", + "model.layers.88.mlp.experts.35.up_proj.weight": "model-00097-of-00101.safetensors", + "model.layers.88.mlp.experts.35.down_proj.weight": "model-00097-of-00101.safetensors", + "model.layers.88.mlp.experts.36.gate_proj.weight": "model-00097-of-00101.safetensors", + "model.layers.88.mlp.experts.36.up_proj.weight": "model-00097-of-00101.safetensors", + "model.layers.88.mlp.experts.36.down_proj.weight": "model-00097-of-00101.safetensors", + "model.layers.88.mlp.experts.37.gate_proj.weight": "model-00097-of-00101.safetensors", + "model.layers.88.mlp.experts.37.up_proj.weight": "model-00097-of-00101.safetensors", + "model.layers.88.mlp.experts.37.down_proj.weight": "model-00097-of-00101.safetensors", + "model.layers.88.mlp.experts.38.gate_proj.weight": "model-00097-of-00101.safetensors", + "model.layers.88.mlp.experts.38.up_proj.weight": "model-00097-of-00101.safetensors", + "model.layers.88.mlp.experts.38.down_proj.weight": "model-00097-of-00101.safetensors", + "model.layers.88.mlp.experts.39.gate_proj.weight": "model-00097-of-00101.safetensors", + "model.layers.88.mlp.experts.39.up_proj.weight": "model-00097-of-00101.safetensors", + "model.layers.88.mlp.experts.39.down_proj.weight": "model-00097-of-00101.safetensors", + "model.layers.88.mlp.experts.40.gate_proj.weight": "model-00097-of-00101.safetensors", + "model.layers.88.mlp.experts.40.up_proj.weight": "model-00097-of-00101.safetensors", + "model.layers.88.mlp.experts.40.down_proj.weight": "model-00097-of-00101.safetensors", + "model.layers.88.mlp.experts.41.gate_proj.weight": "model-00097-of-00101.safetensors", + "model.layers.88.mlp.experts.41.up_proj.weight": "model-00097-of-00101.safetensors", + "model.layers.88.mlp.experts.41.down_proj.weight": "model-00097-of-00101.safetensors", + "model.layers.88.mlp.experts.42.gate_proj.weight": "model-00097-of-00101.safetensors", + "model.layers.88.mlp.experts.42.up_proj.weight": "model-00097-of-00101.safetensors", + "model.layers.88.mlp.experts.42.down_proj.weight": "model-00097-of-00101.safetensors", + "model.layers.88.mlp.experts.43.gate_proj.weight": "model-00097-of-00101.safetensors", + "model.layers.88.mlp.experts.43.up_proj.weight": "model-00097-of-00101.safetensors", + "model.layers.88.mlp.experts.43.down_proj.weight": "model-00097-of-00101.safetensors", + "model.layers.88.mlp.experts.44.gate_proj.weight": "model-00097-of-00101.safetensors", + "model.layers.88.mlp.experts.44.up_proj.weight": "model-00097-of-00101.safetensors", + "model.layers.88.mlp.experts.44.down_proj.weight": "model-00097-of-00101.safetensors", + "model.layers.88.mlp.experts.45.gate_proj.weight": "model-00097-of-00101.safetensors", + "model.layers.88.mlp.experts.45.up_proj.weight": "model-00097-of-00101.safetensors", + "model.layers.88.mlp.experts.45.down_proj.weight": "model-00097-of-00101.safetensors", + "model.layers.88.mlp.experts.46.gate_proj.weight": "model-00097-of-00101.safetensors", + "model.layers.88.mlp.experts.46.up_proj.weight": "model-00097-of-00101.safetensors", + "model.layers.88.mlp.experts.46.down_proj.weight": "model-00097-of-00101.safetensors", + "model.layers.88.mlp.experts.47.gate_proj.weight": "model-00097-of-00101.safetensors", + "model.layers.88.mlp.experts.47.up_proj.weight": "model-00097-of-00101.safetensors", + "model.layers.88.mlp.experts.47.down_proj.weight": "model-00097-of-00101.safetensors", + "model.layers.88.mlp.experts.48.gate_proj.weight": "model-00097-of-00101.safetensors", + "model.layers.88.mlp.experts.48.up_proj.weight": "model-00097-of-00101.safetensors", + "model.layers.88.mlp.experts.48.down_proj.weight": "model-00097-of-00101.safetensors", + "model.layers.88.mlp.experts.49.gate_proj.weight": "model-00097-of-00101.safetensors", + "model.layers.88.mlp.experts.49.up_proj.weight": "model-00097-of-00101.safetensors", + "model.layers.88.mlp.experts.49.down_proj.weight": "model-00097-of-00101.safetensors", + "model.layers.88.mlp.experts.50.gate_proj.weight": "model-00097-of-00101.safetensors", + "model.layers.88.mlp.experts.50.up_proj.weight": "model-00097-of-00101.safetensors", + "model.layers.88.mlp.experts.50.down_proj.weight": "model-00097-of-00101.safetensors", + "model.layers.88.mlp.experts.51.gate_proj.weight": "model-00097-of-00101.safetensors", + "model.layers.88.mlp.experts.51.up_proj.weight": "model-00097-of-00101.safetensors", + "model.layers.88.mlp.experts.51.down_proj.weight": "model-00097-of-00101.safetensors", + "model.layers.88.mlp.experts.52.gate_proj.weight": "model-00097-of-00101.safetensors", + "model.layers.88.mlp.experts.52.up_proj.weight": "model-00097-of-00101.safetensors", + "model.layers.88.mlp.experts.52.down_proj.weight": "model-00097-of-00101.safetensors", + "model.layers.88.mlp.experts.53.gate_proj.weight": "model-00097-of-00101.safetensors", + "model.layers.88.mlp.experts.53.up_proj.weight": "model-00097-of-00101.safetensors", + "model.layers.88.mlp.experts.53.down_proj.weight": "model-00097-of-00101.safetensors", + "model.layers.88.mlp.experts.54.gate_proj.weight": "model-00097-of-00101.safetensors", + "model.layers.88.mlp.experts.54.up_proj.weight": "model-00097-of-00101.safetensors", + "model.layers.88.mlp.experts.54.down_proj.weight": "model-00097-of-00101.safetensors", + "model.layers.88.mlp.experts.55.gate_proj.weight": "model-00097-of-00101.safetensors", + "model.layers.88.mlp.experts.55.up_proj.weight": "model-00097-of-00101.safetensors", + "model.layers.88.mlp.experts.55.down_proj.weight": "model-00097-of-00101.safetensors", + "model.layers.88.mlp.experts.56.gate_proj.weight": "model-00097-of-00101.safetensors", + "model.layers.88.mlp.experts.56.up_proj.weight": "model-00097-of-00101.safetensors", + "model.layers.88.mlp.experts.56.down_proj.weight": "model-00097-of-00101.safetensors", + "model.layers.88.mlp.experts.57.gate_proj.weight": "model-00097-of-00101.safetensors", + "model.layers.88.mlp.experts.57.up_proj.weight": "model-00097-of-00101.safetensors", + "model.layers.88.mlp.experts.57.down_proj.weight": "model-00097-of-00101.safetensors", + "model.layers.88.mlp.experts.58.gate_proj.weight": "model-00097-of-00101.safetensors", + "model.layers.88.mlp.experts.58.up_proj.weight": "model-00097-of-00101.safetensors", + "model.layers.88.mlp.experts.58.down_proj.weight": "model-00097-of-00101.safetensors", + "model.layers.88.mlp.experts.59.gate_proj.weight": "model-00097-of-00101.safetensors", + "model.layers.88.mlp.experts.59.up_proj.weight": "model-00097-of-00101.safetensors", + "model.layers.88.mlp.experts.59.down_proj.weight": "model-00097-of-00101.safetensors", + "model.layers.88.mlp.experts.60.gate_proj.weight": "model-00097-of-00101.safetensors", + "model.layers.88.mlp.experts.60.up_proj.weight": "model-00097-of-00101.safetensors", + "model.layers.88.mlp.experts.60.down_proj.weight": "model-00097-of-00101.safetensors", + "model.layers.88.mlp.experts.61.gate_proj.weight": "model-00097-of-00101.safetensors", + "model.layers.88.mlp.experts.61.up_proj.weight": "model-00097-of-00101.safetensors", + "model.layers.88.mlp.experts.61.down_proj.weight": "model-00097-of-00101.safetensors", + "model.layers.88.mlp.experts.62.gate_proj.weight": "model-00097-of-00101.safetensors", + "model.layers.88.mlp.experts.62.up_proj.weight": "model-00097-of-00101.safetensors", + "model.layers.88.mlp.experts.62.down_proj.weight": "model-00097-of-00101.safetensors", + "model.layers.88.mlp.experts.63.gate_proj.weight": "model-00097-of-00101.safetensors", + "model.layers.88.mlp.experts.63.up_proj.weight": "model-00097-of-00101.safetensors", + "model.layers.88.mlp.experts.63.down_proj.weight": "model-00097-of-00101.safetensors", + "model.layers.88.mlp.experts.64.gate_proj.weight": "model-00097-of-00101.safetensors", + "model.layers.88.mlp.experts.64.up_proj.weight": "model-00097-of-00101.safetensors", + "model.layers.88.mlp.experts.64.down_proj.weight": "model-00097-of-00101.safetensors", + "model.layers.88.mlp.experts.65.gate_proj.weight": "model-00097-of-00101.safetensors", + "model.layers.88.mlp.experts.65.up_proj.weight": "model-00097-of-00101.safetensors", + "model.layers.88.mlp.experts.65.down_proj.weight": "model-00097-of-00101.safetensors", + "model.layers.88.mlp.experts.66.gate_proj.weight": "model-00097-of-00101.safetensors", + "model.layers.88.mlp.experts.66.up_proj.weight": "model-00097-of-00101.safetensors", + "model.layers.88.mlp.experts.66.down_proj.weight": "model-00097-of-00101.safetensors", + "model.layers.88.mlp.experts.67.gate_proj.weight": "model-00097-of-00101.safetensors", + "model.layers.88.mlp.experts.67.up_proj.weight": "model-00097-of-00101.safetensors", + "model.layers.88.mlp.experts.67.down_proj.weight": "model-00097-of-00101.safetensors", + "model.layers.88.mlp.experts.68.gate_proj.weight": "model-00097-of-00101.safetensors", + "model.layers.88.mlp.experts.68.up_proj.weight": "model-00097-of-00101.safetensors", + "model.layers.88.mlp.experts.68.down_proj.weight": "model-00097-of-00101.safetensors", + "model.layers.88.mlp.experts.69.gate_proj.weight": "model-00097-of-00101.safetensors", + "model.layers.88.mlp.experts.69.up_proj.weight": "model-00097-of-00101.safetensors", + "model.layers.88.mlp.experts.69.down_proj.weight": "model-00097-of-00101.safetensors", + "model.layers.88.mlp.experts.70.gate_proj.weight": "model-00097-of-00101.safetensors", + "model.layers.88.mlp.experts.70.up_proj.weight": "model-00097-of-00101.safetensors", + "model.layers.88.mlp.experts.70.down_proj.weight": "model-00097-of-00101.safetensors", + "model.layers.88.mlp.experts.71.gate_proj.weight": "model-00097-of-00101.safetensors", + "model.layers.88.mlp.experts.71.up_proj.weight": "model-00097-of-00101.safetensors", + "model.layers.88.mlp.experts.71.down_proj.weight": "model-00097-of-00101.safetensors", + "model.layers.88.mlp.experts.72.gate_proj.weight": "model-00097-of-00101.safetensors", + "model.layers.88.mlp.experts.72.up_proj.weight": "model-00097-of-00101.safetensors", + "model.layers.88.mlp.experts.72.down_proj.weight": "model-00097-of-00101.safetensors", + "model.layers.88.mlp.experts.73.gate_proj.weight": "model-00097-of-00101.safetensors", + "model.layers.88.mlp.experts.73.up_proj.weight": "model-00097-of-00101.safetensors", + "model.layers.88.mlp.experts.73.down_proj.weight": "model-00097-of-00101.safetensors", + "model.layers.88.mlp.experts.74.gate_proj.weight": "model-00097-of-00101.safetensors", + "model.layers.88.mlp.experts.74.up_proj.weight": "model-00097-of-00101.safetensors", + "model.layers.88.mlp.experts.74.down_proj.weight": "model-00097-of-00101.safetensors", + "model.layers.88.mlp.experts.75.gate_proj.weight": "model-00097-of-00101.safetensors", + "model.layers.88.mlp.experts.75.up_proj.weight": "model-00097-of-00101.safetensors", + "model.layers.88.mlp.experts.75.down_proj.weight": "model-00097-of-00101.safetensors", + "model.layers.88.mlp.experts.76.gate_proj.weight": "model-00097-of-00101.safetensors", + "model.layers.88.mlp.experts.76.up_proj.weight": "model-00097-of-00101.safetensors", + "model.layers.88.mlp.experts.76.down_proj.weight": "model-00097-of-00101.safetensors", + "model.layers.88.mlp.experts.77.gate_proj.weight": "model-00097-of-00101.safetensors", + "model.layers.88.mlp.experts.77.up_proj.weight": "model-00097-of-00101.safetensors", + "model.layers.88.mlp.experts.77.down_proj.weight": "model-00097-of-00101.safetensors", + "model.layers.88.mlp.experts.78.gate_proj.weight": "model-00097-of-00101.safetensors", + "model.layers.88.mlp.experts.78.up_proj.weight": "model-00097-of-00101.safetensors", + "model.layers.88.mlp.experts.78.down_proj.weight": "model-00097-of-00101.safetensors", + "model.layers.88.mlp.experts.79.gate_proj.weight": "model-00097-of-00101.safetensors", + "model.layers.88.mlp.experts.79.up_proj.weight": "model-00097-of-00101.safetensors", + "model.layers.88.mlp.experts.79.down_proj.weight": "model-00097-of-00101.safetensors", + "model.layers.88.mlp.experts.80.gate_proj.weight": "model-00097-of-00101.safetensors", + "model.layers.88.mlp.experts.80.up_proj.weight": "model-00097-of-00101.safetensors", + "model.layers.88.mlp.experts.80.down_proj.weight": "model-00097-of-00101.safetensors", + "model.layers.88.mlp.experts.81.gate_proj.weight": "model-00097-of-00101.safetensors", + "model.layers.88.mlp.experts.81.up_proj.weight": "model-00097-of-00101.safetensors", + "model.layers.88.mlp.experts.81.down_proj.weight": "model-00097-of-00101.safetensors", + "model.layers.88.mlp.experts.82.gate_proj.weight": "model-00097-of-00101.safetensors", + "model.layers.88.mlp.experts.82.up_proj.weight": "model-00097-of-00101.safetensors", + "model.layers.88.mlp.experts.82.down_proj.weight": "model-00097-of-00101.safetensors", + "model.layers.88.mlp.experts.83.gate_proj.weight": "model-00097-of-00101.safetensors", + "model.layers.88.mlp.experts.83.up_proj.weight": "model-00097-of-00101.safetensors", + "model.layers.88.mlp.experts.83.down_proj.weight": "model-00097-of-00101.safetensors", + "model.layers.88.mlp.experts.84.gate_proj.weight": "model-00097-of-00101.safetensors", + "model.layers.88.mlp.experts.84.up_proj.weight": "model-00097-of-00101.safetensors", + "model.layers.88.mlp.experts.84.down_proj.weight": "model-00097-of-00101.safetensors", + "model.layers.88.mlp.experts.85.gate_proj.weight": "model-00097-of-00101.safetensors", + "model.layers.88.mlp.experts.85.up_proj.weight": "model-00097-of-00101.safetensors", + "model.layers.88.mlp.experts.85.down_proj.weight": "model-00097-of-00101.safetensors", + "model.layers.88.mlp.experts.86.gate_proj.weight": "model-00097-of-00101.safetensors", + "model.layers.88.mlp.experts.86.up_proj.weight": "model-00097-of-00101.safetensors", + "model.layers.88.mlp.experts.86.down_proj.weight": "model-00097-of-00101.safetensors", + "model.layers.88.mlp.experts.87.gate_proj.weight": "model-00097-of-00101.safetensors", + "model.layers.88.mlp.experts.87.up_proj.weight": "model-00097-of-00101.safetensors", + "model.layers.88.mlp.experts.87.down_proj.weight": "model-00097-of-00101.safetensors", + "model.layers.88.mlp.experts.88.gate_proj.weight": "model-00097-of-00101.safetensors", + "model.layers.88.mlp.experts.88.up_proj.weight": "model-00097-of-00101.safetensors", + "model.layers.88.mlp.experts.88.down_proj.weight": "model-00097-of-00101.safetensors", + "model.layers.88.mlp.experts.89.gate_proj.weight": "model-00097-of-00101.safetensors", + "model.layers.88.mlp.experts.89.up_proj.weight": "model-00097-of-00101.safetensors", + "model.layers.88.mlp.experts.89.down_proj.weight": "model-00097-of-00101.safetensors", + "model.layers.88.mlp.experts.90.gate_proj.weight": "model-00097-of-00101.safetensors", + "model.layers.88.mlp.experts.90.up_proj.weight": "model-00097-of-00101.safetensors", + "model.layers.88.mlp.experts.90.down_proj.weight": "model-00097-of-00101.safetensors", + "model.layers.88.mlp.experts.91.gate_proj.weight": "model-00097-of-00101.safetensors", + "model.layers.88.mlp.experts.91.up_proj.weight": "model-00097-of-00101.safetensors", + "model.layers.88.mlp.experts.91.down_proj.weight": "model-00097-of-00101.safetensors", + "model.layers.88.mlp.experts.92.gate_proj.weight": "model-00097-of-00101.safetensors", + "model.layers.88.mlp.experts.92.up_proj.weight": "model-00097-of-00101.safetensors", + "model.layers.88.mlp.experts.92.down_proj.weight": "model-00097-of-00101.safetensors", + "model.layers.88.mlp.experts.93.gate_proj.weight": "model-00097-of-00101.safetensors", + "model.layers.88.mlp.experts.93.up_proj.weight": "model-00097-of-00101.safetensors", + "model.layers.88.mlp.experts.93.down_proj.weight": "model-00097-of-00101.safetensors", + "model.layers.88.mlp.experts.94.gate_proj.weight": "model-00097-of-00101.safetensors", + "model.layers.88.mlp.experts.94.up_proj.weight": "model-00097-of-00101.safetensors", + "model.layers.88.mlp.experts.94.down_proj.weight": "model-00097-of-00101.safetensors", + "model.layers.88.mlp.experts.95.gate_proj.weight": "model-00097-of-00101.safetensors", + "model.layers.88.mlp.experts.95.up_proj.weight": "model-00097-of-00101.safetensors", + "model.layers.88.mlp.experts.95.down_proj.weight": "model-00097-of-00101.safetensors", + "model.layers.88.mlp.experts.96.gate_proj.weight": "model-00097-of-00101.safetensors", + "model.layers.88.mlp.experts.96.up_proj.weight": "model-00097-of-00101.safetensors", + "model.layers.88.mlp.experts.96.down_proj.weight": "model-00097-of-00101.safetensors", + "model.layers.88.mlp.experts.97.gate_proj.weight": "model-00097-of-00101.safetensors", + "model.layers.88.mlp.experts.97.up_proj.weight": "model-00097-of-00101.safetensors", + "model.layers.88.mlp.experts.97.down_proj.weight": "model-00097-of-00101.safetensors", + "model.layers.88.mlp.experts.98.gate_proj.weight": "model-00097-of-00101.safetensors", + "model.layers.88.mlp.experts.98.up_proj.weight": "model-00097-of-00101.safetensors", + "model.layers.88.mlp.experts.98.down_proj.weight": "model-00097-of-00101.safetensors", + "model.layers.88.mlp.experts.99.gate_proj.weight": "model-00097-of-00101.safetensors", + "model.layers.88.mlp.experts.99.up_proj.weight": "model-00097-of-00101.safetensors", + "model.layers.88.mlp.experts.99.down_proj.weight": "model-00097-of-00101.safetensors", + "model.layers.88.mlp.experts.100.gate_proj.weight": "model-00097-of-00101.safetensors", + "model.layers.88.mlp.experts.100.up_proj.weight": "model-00097-of-00101.safetensors", + "model.layers.88.mlp.experts.100.down_proj.weight": "model-00097-of-00101.safetensors", + "model.layers.88.mlp.experts.101.gate_proj.weight": "model-00097-of-00101.safetensors", + "model.layers.88.mlp.experts.101.up_proj.weight": "model-00097-of-00101.safetensors", + "model.layers.88.mlp.experts.101.down_proj.weight": "model-00097-of-00101.safetensors", + "model.layers.88.mlp.experts.102.gate_proj.weight": "model-00097-of-00101.safetensors", + "model.layers.88.mlp.experts.102.up_proj.weight": "model-00097-of-00101.safetensors", + "model.layers.88.mlp.experts.102.down_proj.weight": "model-00097-of-00101.safetensors", + "model.layers.88.mlp.experts.103.gate_proj.weight": "model-00097-of-00101.safetensors", + "model.layers.88.mlp.experts.103.up_proj.weight": "model-00097-of-00101.safetensors", + "model.layers.88.mlp.experts.103.down_proj.weight": "model-00097-of-00101.safetensors", + "model.layers.88.mlp.experts.104.gate_proj.weight": "model-00097-of-00101.safetensors", + "model.layers.88.mlp.experts.104.up_proj.weight": "model-00097-of-00101.safetensors", + "model.layers.88.mlp.experts.104.down_proj.weight": "model-00097-of-00101.safetensors", + "model.layers.88.mlp.experts.105.gate_proj.weight": "model-00097-of-00101.safetensors", + "model.layers.88.mlp.experts.105.up_proj.weight": "model-00097-of-00101.safetensors", + "model.layers.88.mlp.experts.105.down_proj.weight": "model-00097-of-00101.safetensors", + "model.layers.88.mlp.experts.106.gate_proj.weight": "model-00097-of-00101.safetensors", + "model.layers.88.mlp.experts.106.up_proj.weight": "model-00097-of-00101.safetensors", + "model.layers.88.mlp.experts.106.down_proj.weight": "model-00097-of-00101.safetensors", + "model.layers.88.mlp.experts.107.gate_proj.weight": "model-00097-of-00101.safetensors", + "model.layers.88.mlp.experts.107.up_proj.weight": "model-00097-of-00101.safetensors", + "model.layers.88.mlp.experts.107.down_proj.weight": "model-00097-of-00101.safetensors", + "model.layers.88.mlp.experts.108.gate_proj.weight": "model-00097-of-00101.safetensors", + "model.layers.88.mlp.experts.108.up_proj.weight": "model-00097-of-00101.safetensors", + "model.layers.88.mlp.experts.108.down_proj.weight": "model-00097-of-00101.safetensors", + "model.layers.88.mlp.experts.109.gate_proj.weight": "model-00097-of-00101.safetensors", + "model.layers.88.mlp.experts.109.up_proj.weight": "model-00097-of-00101.safetensors", + "model.layers.88.mlp.experts.109.down_proj.weight": "model-00097-of-00101.safetensors", + "model.layers.88.mlp.experts.110.gate_proj.weight": "model-00097-of-00101.safetensors", + "model.layers.88.mlp.experts.110.up_proj.weight": "model-00097-of-00101.safetensors", + "model.layers.88.mlp.experts.110.down_proj.weight": "model-00097-of-00101.safetensors", + "model.layers.88.mlp.experts.111.gate_proj.weight": "model-00097-of-00101.safetensors", + "model.layers.88.mlp.experts.111.up_proj.weight": "model-00097-of-00101.safetensors", + "model.layers.88.mlp.experts.111.down_proj.weight": "model-00097-of-00101.safetensors", + "model.layers.88.mlp.experts.112.gate_proj.weight": "model-00097-of-00101.safetensors", + "model.layers.88.mlp.experts.112.up_proj.weight": "model-00097-of-00101.safetensors", + "model.layers.88.mlp.experts.112.down_proj.weight": "model-00097-of-00101.safetensors", + "model.layers.88.mlp.experts.113.gate_proj.weight": "model-00097-of-00101.safetensors", + "model.layers.88.mlp.experts.113.up_proj.weight": "model-00097-of-00101.safetensors", + "model.layers.88.mlp.experts.113.down_proj.weight": "model-00097-of-00101.safetensors", + "model.layers.88.mlp.experts.114.gate_proj.weight": "model-00097-of-00101.safetensors", + "model.layers.88.mlp.experts.114.up_proj.weight": "model-00097-of-00101.safetensors", + "model.layers.88.mlp.experts.114.down_proj.weight": "model-00097-of-00101.safetensors", + "model.layers.88.mlp.experts.115.gate_proj.weight": "model-00097-of-00101.safetensors", + "model.layers.88.mlp.experts.115.up_proj.weight": "model-00097-of-00101.safetensors", + "model.layers.88.mlp.experts.115.down_proj.weight": "model-00097-of-00101.safetensors", + "model.layers.88.mlp.experts.116.gate_proj.weight": "model-00097-of-00101.safetensors", + "model.layers.88.mlp.experts.116.up_proj.weight": "model-00097-of-00101.safetensors", + "model.layers.88.mlp.experts.116.down_proj.weight": "model-00097-of-00101.safetensors", + "model.layers.88.mlp.experts.117.gate_proj.weight": "model-00097-of-00101.safetensors", + "model.layers.88.mlp.experts.117.up_proj.weight": "model-00097-of-00101.safetensors", + "model.layers.88.mlp.experts.117.down_proj.weight": "model-00097-of-00101.safetensors", + "model.layers.88.mlp.experts.118.gate_proj.weight": "model-00097-of-00101.safetensors", + "model.layers.88.mlp.experts.118.up_proj.weight": "model-00097-of-00101.safetensors", + "model.layers.88.mlp.experts.118.down_proj.weight": "model-00097-of-00101.safetensors", + "model.layers.88.mlp.experts.119.gate_proj.weight": "model-00097-of-00101.safetensors", + "model.layers.88.mlp.experts.119.up_proj.weight": "model-00097-of-00101.safetensors", + "model.layers.88.mlp.experts.119.down_proj.weight": "model-00097-of-00101.safetensors", + "model.layers.88.mlp.gate.weight": "model-00097-of-00101.safetensors", + "model.layers.88.mlp.gate.e_score_correction_bias": "model-00097-of-00101.safetensors", + "model.layers.88.mlp.shared_experts.gate_proj.weight": "model-00097-of-00101.safetensors", + "model.layers.88.mlp.shared_experts.up_proj.weight": "model-00097-of-00101.safetensors", + "model.layers.88.mlp.shared_experts.down_proj.weight": "model-00097-of-00101.safetensors", + "model.layers.88.input_layernorm.weight": "model-00097-of-00101.safetensors", + "model.layers.88.post_attention_layernorm.weight": "model-00097-of-00101.safetensors", + "model.layers.89.self_attn.q_proj.weight": "model-00097-of-00101.safetensors", + "model.layers.89.self_attn.q_proj.bias": "model-00097-of-00101.safetensors", + "model.layers.89.self_attn.k_proj.weight": "model-00097-of-00101.safetensors", + "model.layers.89.self_attn.k_proj.bias": "model-00097-of-00101.safetensors", + "model.layers.89.self_attn.v_proj.weight": "model-00097-of-00101.safetensors", + "model.layers.89.self_attn.v_proj.bias": "model-00097-of-00101.safetensors", + "model.layers.89.self_attn.o_proj.weight": "model-00097-of-00101.safetensors", + "model.layers.89.self_attn.q_norm.weight": "model-00097-of-00101.safetensors", + "model.layers.89.self_attn.k_norm.weight": "model-00097-of-00101.safetensors", + "model.layers.89.mlp.experts.0.gate_proj.weight": "model-00097-of-00101.safetensors", + "model.layers.89.mlp.experts.0.up_proj.weight": "model-00097-of-00101.safetensors", + "model.layers.89.mlp.experts.0.down_proj.weight": "model-00097-of-00101.safetensors", + "model.layers.89.mlp.experts.1.gate_proj.weight": "model-00097-of-00101.safetensors", + "model.layers.89.mlp.experts.1.up_proj.weight": "model-00097-of-00101.safetensors", + "model.layers.89.mlp.experts.1.down_proj.weight": "model-00097-of-00101.safetensors", + "model.layers.89.mlp.experts.2.gate_proj.weight": "model-00097-of-00101.safetensors", + "model.layers.89.mlp.experts.2.up_proj.weight": "model-00097-of-00101.safetensors", + "model.layers.89.mlp.experts.2.down_proj.weight": "model-00097-of-00101.safetensors", + "model.layers.89.mlp.experts.3.gate_proj.weight": "model-00097-of-00101.safetensors", + "model.layers.89.mlp.experts.3.up_proj.weight": "model-00097-of-00101.safetensors", + "model.layers.89.mlp.experts.3.down_proj.weight": "model-00097-of-00101.safetensors", + "model.layers.89.mlp.experts.4.gate_proj.weight": "model-00097-of-00101.safetensors", + "model.layers.89.mlp.experts.4.up_proj.weight": "model-00097-of-00101.safetensors", + "model.layers.89.mlp.experts.4.down_proj.weight": "model-00097-of-00101.safetensors", + "model.layers.89.mlp.experts.5.gate_proj.weight": "model-00097-of-00101.safetensors", + "model.layers.89.mlp.experts.5.up_proj.weight": "model-00097-of-00101.safetensors", + "model.layers.89.mlp.experts.5.down_proj.weight": "model-00097-of-00101.safetensors", + "model.layers.89.mlp.experts.6.gate_proj.weight": "model-00097-of-00101.safetensors", + "model.layers.89.mlp.experts.6.up_proj.weight": "model-00097-of-00101.safetensors", + "model.layers.89.mlp.experts.6.down_proj.weight": "model-00097-of-00101.safetensors", + "model.layers.89.mlp.experts.7.gate_proj.weight": "model-00097-of-00101.safetensors", + "model.layers.89.mlp.experts.7.up_proj.weight": "model-00097-of-00101.safetensors", + "model.layers.89.mlp.experts.7.down_proj.weight": "model-00097-of-00101.safetensors", + "model.layers.89.mlp.experts.8.gate_proj.weight": "model-00097-of-00101.safetensors", + "model.layers.89.mlp.experts.8.up_proj.weight": "model-00097-of-00101.safetensors", + "model.layers.89.mlp.experts.8.down_proj.weight": "model-00097-of-00101.safetensors", + "model.layers.89.mlp.experts.9.gate_proj.weight": "model-00097-of-00101.safetensors", + "model.layers.89.mlp.experts.9.up_proj.weight": "model-00097-of-00101.safetensors", + "model.layers.89.mlp.experts.9.down_proj.weight": "model-00097-of-00101.safetensors", + "model.layers.89.mlp.experts.10.gate_proj.weight": "model-00097-of-00101.safetensors", + "model.layers.89.mlp.experts.10.up_proj.weight": "model-00097-of-00101.safetensors", + "model.layers.89.mlp.experts.10.down_proj.weight": "model-00097-of-00101.safetensors", + "model.layers.89.mlp.experts.11.gate_proj.weight": "model-00097-of-00101.safetensors", + "model.layers.89.mlp.experts.11.up_proj.weight": "model-00097-of-00101.safetensors", + "model.layers.89.mlp.experts.11.down_proj.weight": "model-00097-of-00101.safetensors", + "model.layers.89.mlp.experts.12.gate_proj.weight": "model-00097-of-00101.safetensors", + "model.layers.89.mlp.experts.12.up_proj.weight": "model-00097-of-00101.safetensors", + "model.layers.89.mlp.experts.12.down_proj.weight": "model-00097-of-00101.safetensors", + "model.layers.89.mlp.experts.13.gate_proj.weight": "model-00097-of-00101.safetensors", + "model.layers.89.mlp.experts.13.up_proj.weight": "model-00097-of-00101.safetensors", + "model.layers.89.mlp.experts.13.down_proj.weight": "model-00097-of-00101.safetensors", + "model.layers.89.mlp.experts.14.gate_proj.weight": "model-00097-of-00101.safetensors", + "model.layers.89.mlp.experts.14.up_proj.weight": "model-00097-of-00101.safetensors", + "model.layers.89.mlp.experts.14.down_proj.weight": "model-00097-of-00101.safetensors", + "model.layers.89.mlp.experts.15.gate_proj.weight": "model-00097-of-00101.safetensors", + "model.layers.89.mlp.experts.15.up_proj.weight": "model-00097-of-00101.safetensors", + "model.layers.89.mlp.experts.15.down_proj.weight": "model-00097-of-00101.safetensors", + "model.layers.89.mlp.experts.16.gate_proj.weight": "model-00097-of-00101.safetensors", + "model.layers.89.mlp.experts.16.up_proj.weight": "model-00097-of-00101.safetensors", + "model.layers.89.mlp.experts.16.down_proj.weight": "model-00097-of-00101.safetensors", + "model.layers.89.mlp.experts.17.gate_proj.weight": "model-00097-of-00101.safetensors", + "model.layers.89.mlp.experts.17.up_proj.weight": "model-00097-of-00101.safetensors", + "model.layers.89.mlp.experts.17.down_proj.weight": "model-00097-of-00101.safetensors", + "model.layers.89.mlp.experts.18.gate_proj.weight": "model-00097-of-00101.safetensors", + "model.layers.89.mlp.experts.18.up_proj.weight": "model-00097-of-00101.safetensors", + "model.layers.89.mlp.experts.18.down_proj.weight": "model-00097-of-00101.safetensors", + "model.layers.89.mlp.experts.19.gate_proj.weight": "model-00097-of-00101.safetensors", + "model.layers.89.mlp.experts.19.up_proj.weight": "model-00097-of-00101.safetensors", + "model.layers.89.mlp.experts.19.down_proj.weight": "model-00097-of-00101.safetensors", + "model.layers.89.mlp.experts.20.gate_proj.weight": "model-00097-of-00101.safetensors", + "model.layers.89.mlp.experts.20.up_proj.weight": "model-00097-of-00101.safetensors", + "model.layers.89.mlp.experts.20.down_proj.weight": "model-00097-of-00101.safetensors", + "model.layers.89.mlp.experts.21.gate_proj.weight": "model-00097-of-00101.safetensors", + "model.layers.89.mlp.experts.21.up_proj.weight": "model-00098-of-00101.safetensors", + "model.layers.89.mlp.experts.21.down_proj.weight": "model-00098-of-00101.safetensors", + "model.layers.89.mlp.experts.22.gate_proj.weight": "model-00098-of-00101.safetensors", + "model.layers.89.mlp.experts.22.up_proj.weight": "model-00098-of-00101.safetensors", + "model.layers.89.mlp.experts.22.down_proj.weight": "model-00098-of-00101.safetensors", + "model.layers.89.mlp.experts.23.gate_proj.weight": "model-00098-of-00101.safetensors", + "model.layers.89.mlp.experts.23.up_proj.weight": "model-00098-of-00101.safetensors", + "model.layers.89.mlp.experts.23.down_proj.weight": "model-00098-of-00101.safetensors", + "model.layers.89.mlp.experts.24.gate_proj.weight": "model-00098-of-00101.safetensors", + "model.layers.89.mlp.experts.24.up_proj.weight": "model-00098-of-00101.safetensors", + "model.layers.89.mlp.experts.24.down_proj.weight": "model-00098-of-00101.safetensors", + "model.layers.89.mlp.experts.25.gate_proj.weight": "model-00098-of-00101.safetensors", + "model.layers.89.mlp.experts.25.up_proj.weight": "model-00098-of-00101.safetensors", + "model.layers.89.mlp.experts.25.down_proj.weight": "model-00098-of-00101.safetensors", + "model.layers.89.mlp.experts.26.gate_proj.weight": "model-00098-of-00101.safetensors", + "model.layers.89.mlp.experts.26.up_proj.weight": "model-00098-of-00101.safetensors", + "model.layers.89.mlp.experts.26.down_proj.weight": "model-00098-of-00101.safetensors", + "model.layers.89.mlp.experts.27.gate_proj.weight": "model-00098-of-00101.safetensors", + "model.layers.89.mlp.experts.27.up_proj.weight": "model-00098-of-00101.safetensors", + "model.layers.89.mlp.experts.27.down_proj.weight": "model-00098-of-00101.safetensors", + "model.layers.89.mlp.experts.28.gate_proj.weight": "model-00098-of-00101.safetensors", + "model.layers.89.mlp.experts.28.up_proj.weight": "model-00098-of-00101.safetensors", + "model.layers.89.mlp.experts.28.down_proj.weight": "model-00098-of-00101.safetensors", + "model.layers.89.mlp.experts.29.gate_proj.weight": "model-00098-of-00101.safetensors", + "model.layers.89.mlp.experts.29.up_proj.weight": "model-00098-of-00101.safetensors", + "model.layers.89.mlp.experts.29.down_proj.weight": "model-00098-of-00101.safetensors", + "model.layers.89.mlp.experts.30.gate_proj.weight": "model-00098-of-00101.safetensors", + "model.layers.89.mlp.experts.30.up_proj.weight": "model-00098-of-00101.safetensors", + "model.layers.89.mlp.experts.30.down_proj.weight": "model-00098-of-00101.safetensors", + "model.layers.89.mlp.experts.31.gate_proj.weight": "model-00098-of-00101.safetensors", + "model.layers.89.mlp.experts.31.up_proj.weight": "model-00098-of-00101.safetensors", + "model.layers.89.mlp.experts.31.down_proj.weight": "model-00098-of-00101.safetensors", + "model.layers.89.mlp.experts.32.gate_proj.weight": "model-00098-of-00101.safetensors", + "model.layers.89.mlp.experts.32.up_proj.weight": "model-00098-of-00101.safetensors", + "model.layers.89.mlp.experts.32.down_proj.weight": "model-00098-of-00101.safetensors", + "model.layers.89.mlp.experts.33.gate_proj.weight": "model-00098-of-00101.safetensors", + "model.layers.89.mlp.experts.33.up_proj.weight": "model-00098-of-00101.safetensors", + "model.layers.89.mlp.experts.33.down_proj.weight": "model-00098-of-00101.safetensors", + "model.layers.89.mlp.experts.34.gate_proj.weight": "model-00098-of-00101.safetensors", + "model.layers.89.mlp.experts.34.up_proj.weight": "model-00098-of-00101.safetensors", + "model.layers.89.mlp.experts.34.down_proj.weight": "model-00098-of-00101.safetensors", + "model.layers.89.mlp.experts.35.gate_proj.weight": "model-00098-of-00101.safetensors", + "model.layers.89.mlp.experts.35.up_proj.weight": "model-00098-of-00101.safetensors", + "model.layers.89.mlp.experts.35.down_proj.weight": "model-00098-of-00101.safetensors", + "model.layers.89.mlp.experts.36.gate_proj.weight": "model-00098-of-00101.safetensors", + "model.layers.89.mlp.experts.36.up_proj.weight": "model-00098-of-00101.safetensors", + "model.layers.89.mlp.experts.36.down_proj.weight": "model-00098-of-00101.safetensors", + "model.layers.89.mlp.experts.37.gate_proj.weight": "model-00098-of-00101.safetensors", + "model.layers.89.mlp.experts.37.up_proj.weight": "model-00098-of-00101.safetensors", + "model.layers.89.mlp.experts.37.down_proj.weight": "model-00098-of-00101.safetensors", + "model.layers.89.mlp.experts.38.gate_proj.weight": "model-00098-of-00101.safetensors", + "model.layers.89.mlp.experts.38.up_proj.weight": "model-00098-of-00101.safetensors", + "model.layers.89.mlp.experts.38.down_proj.weight": "model-00098-of-00101.safetensors", + "model.layers.89.mlp.experts.39.gate_proj.weight": "model-00098-of-00101.safetensors", + "model.layers.89.mlp.experts.39.up_proj.weight": "model-00098-of-00101.safetensors", + "model.layers.89.mlp.experts.39.down_proj.weight": "model-00098-of-00101.safetensors", + "model.layers.89.mlp.experts.40.gate_proj.weight": "model-00098-of-00101.safetensors", + "model.layers.89.mlp.experts.40.up_proj.weight": "model-00098-of-00101.safetensors", + "model.layers.89.mlp.experts.40.down_proj.weight": "model-00098-of-00101.safetensors", + "model.layers.89.mlp.experts.41.gate_proj.weight": "model-00098-of-00101.safetensors", + "model.layers.89.mlp.experts.41.up_proj.weight": "model-00098-of-00101.safetensors", + "model.layers.89.mlp.experts.41.down_proj.weight": "model-00098-of-00101.safetensors", + "model.layers.89.mlp.experts.42.gate_proj.weight": "model-00098-of-00101.safetensors", + "model.layers.89.mlp.experts.42.up_proj.weight": "model-00098-of-00101.safetensors", + "model.layers.89.mlp.experts.42.down_proj.weight": "model-00098-of-00101.safetensors", + "model.layers.89.mlp.experts.43.gate_proj.weight": "model-00098-of-00101.safetensors", + "model.layers.89.mlp.experts.43.up_proj.weight": "model-00098-of-00101.safetensors", + "model.layers.89.mlp.experts.43.down_proj.weight": "model-00098-of-00101.safetensors", + "model.layers.89.mlp.experts.44.gate_proj.weight": "model-00098-of-00101.safetensors", + "model.layers.89.mlp.experts.44.up_proj.weight": "model-00098-of-00101.safetensors", + "model.layers.89.mlp.experts.44.down_proj.weight": "model-00098-of-00101.safetensors", + "model.layers.89.mlp.experts.45.gate_proj.weight": "model-00098-of-00101.safetensors", + "model.layers.89.mlp.experts.45.up_proj.weight": "model-00098-of-00101.safetensors", + "model.layers.89.mlp.experts.45.down_proj.weight": "model-00098-of-00101.safetensors", + "model.layers.89.mlp.experts.46.gate_proj.weight": "model-00098-of-00101.safetensors", + "model.layers.89.mlp.experts.46.up_proj.weight": "model-00098-of-00101.safetensors", + "model.layers.89.mlp.experts.46.down_proj.weight": "model-00098-of-00101.safetensors", + "model.layers.89.mlp.experts.47.gate_proj.weight": "model-00098-of-00101.safetensors", + "model.layers.89.mlp.experts.47.up_proj.weight": "model-00098-of-00101.safetensors", + "model.layers.89.mlp.experts.47.down_proj.weight": "model-00098-of-00101.safetensors", + "model.layers.89.mlp.experts.48.gate_proj.weight": "model-00098-of-00101.safetensors", + "model.layers.89.mlp.experts.48.up_proj.weight": "model-00098-of-00101.safetensors", + "model.layers.89.mlp.experts.48.down_proj.weight": "model-00098-of-00101.safetensors", + "model.layers.89.mlp.experts.49.gate_proj.weight": "model-00098-of-00101.safetensors", + "model.layers.89.mlp.experts.49.up_proj.weight": "model-00098-of-00101.safetensors", + "model.layers.89.mlp.experts.49.down_proj.weight": "model-00098-of-00101.safetensors", + "model.layers.89.mlp.experts.50.gate_proj.weight": "model-00098-of-00101.safetensors", + "model.layers.89.mlp.experts.50.up_proj.weight": "model-00098-of-00101.safetensors", + "model.layers.89.mlp.experts.50.down_proj.weight": "model-00098-of-00101.safetensors", + "model.layers.89.mlp.experts.51.gate_proj.weight": "model-00098-of-00101.safetensors", + "model.layers.89.mlp.experts.51.up_proj.weight": "model-00098-of-00101.safetensors", + "model.layers.89.mlp.experts.51.down_proj.weight": "model-00098-of-00101.safetensors", + "model.layers.89.mlp.experts.52.gate_proj.weight": "model-00098-of-00101.safetensors", + "model.layers.89.mlp.experts.52.up_proj.weight": "model-00098-of-00101.safetensors", + "model.layers.89.mlp.experts.52.down_proj.weight": "model-00098-of-00101.safetensors", + "model.layers.89.mlp.experts.53.gate_proj.weight": "model-00098-of-00101.safetensors", + "model.layers.89.mlp.experts.53.up_proj.weight": "model-00098-of-00101.safetensors", + "model.layers.89.mlp.experts.53.down_proj.weight": "model-00098-of-00101.safetensors", + "model.layers.89.mlp.experts.54.gate_proj.weight": "model-00098-of-00101.safetensors", + "model.layers.89.mlp.experts.54.up_proj.weight": "model-00098-of-00101.safetensors", + "model.layers.89.mlp.experts.54.down_proj.weight": "model-00098-of-00101.safetensors", + "model.layers.89.mlp.experts.55.gate_proj.weight": "model-00098-of-00101.safetensors", + "model.layers.89.mlp.experts.55.up_proj.weight": "model-00098-of-00101.safetensors", + "model.layers.89.mlp.experts.55.down_proj.weight": "model-00098-of-00101.safetensors", + "model.layers.89.mlp.experts.56.gate_proj.weight": "model-00098-of-00101.safetensors", + "model.layers.89.mlp.experts.56.up_proj.weight": "model-00098-of-00101.safetensors", + "model.layers.89.mlp.experts.56.down_proj.weight": "model-00098-of-00101.safetensors", + "model.layers.89.mlp.experts.57.gate_proj.weight": "model-00098-of-00101.safetensors", + "model.layers.89.mlp.experts.57.up_proj.weight": "model-00098-of-00101.safetensors", + "model.layers.89.mlp.experts.57.down_proj.weight": "model-00098-of-00101.safetensors", + "model.layers.89.mlp.experts.58.gate_proj.weight": "model-00098-of-00101.safetensors", + "model.layers.89.mlp.experts.58.up_proj.weight": "model-00098-of-00101.safetensors", + "model.layers.89.mlp.experts.58.down_proj.weight": "model-00098-of-00101.safetensors", + "model.layers.89.mlp.experts.59.gate_proj.weight": "model-00098-of-00101.safetensors", + "model.layers.89.mlp.experts.59.up_proj.weight": "model-00098-of-00101.safetensors", + "model.layers.89.mlp.experts.59.down_proj.weight": "model-00098-of-00101.safetensors", + "model.layers.89.mlp.experts.60.gate_proj.weight": "model-00098-of-00101.safetensors", + "model.layers.89.mlp.experts.60.up_proj.weight": "model-00098-of-00101.safetensors", + "model.layers.89.mlp.experts.60.down_proj.weight": "model-00098-of-00101.safetensors", + "model.layers.89.mlp.experts.61.gate_proj.weight": "model-00098-of-00101.safetensors", + "model.layers.89.mlp.experts.61.up_proj.weight": "model-00098-of-00101.safetensors", + "model.layers.89.mlp.experts.61.down_proj.weight": "model-00098-of-00101.safetensors", + "model.layers.89.mlp.experts.62.gate_proj.weight": "model-00098-of-00101.safetensors", + "model.layers.89.mlp.experts.62.up_proj.weight": "model-00098-of-00101.safetensors", + "model.layers.89.mlp.experts.62.down_proj.weight": "model-00098-of-00101.safetensors", + "model.layers.89.mlp.experts.63.gate_proj.weight": "model-00098-of-00101.safetensors", + "model.layers.89.mlp.experts.63.up_proj.weight": "model-00098-of-00101.safetensors", + "model.layers.89.mlp.experts.63.down_proj.weight": "model-00098-of-00101.safetensors", + "model.layers.89.mlp.experts.64.gate_proj.weight": "model-00098-of-00101.safetensors", + "model.layers.89.mlp.experts.64.up_proj.weight": "model-00098-of-00101.safetensors", + "model.layers.89.mlp.experts.64.down_proj.weight": "model-00098-of-00101.safetensors", + "model.layers.89.mlp.experts.65.gate_proj.weight": "model-00098-of-00101.safetensors", + "model.layers.89.mlp.experts.65.up_proj.weight": "model-00098-of-00101.safetensors", + "model.layers.89.mlp.experts.65.down_proj.weight": "model-00098-of-00101.safetensors", + "model.layers.89.mlp.experts.66.gate_proj.weight": "model-00098-of-00101.safetensors", + "model.layers.89.mlp.experts.66.up_proj.weight": "model-00098-of-00101.safetensors", + "model.layers.89.mlp.experts.66.down_proj.weight": "model-00098-of-00101.safetensors", + "model.layers.89.mlp.experts.67.gate_proj.weight": "model-00098-of-00101.safetensors", + "model.layers.89.mlp.experts.67.up_proj.weight": "model-00098-of-00101.safetensors", + "model.layers.89.mlp.experts.67.down_proj.weight": "model-00098-of-00101.safetensors", + "model.layers.89.mlp.experts.68.gate_proj.weight": "model-00098-of-00101.safetensors", + "model.layers.89.mlp.experts.68.up_proj.weight": "model-00098-of-00101.safetensors", + "model.layers.89.mlp.experts.68.down_proj.weight": "model-00098-of-00101.safetensors", + "model.layers.89.mlp.experts.69.gate_proj.weight": "model-00098-of-00101.safetensors", + "model.layers.89.mlp.experts.69.up_proj.weight": "model-00098-of-00101.safetensors", + "model.layers.89.mlp.experts.69.down_proj.weight": "model-00098-of-00101.safetensors", + "model.layers.89.mlp.experts.70.gate_proj.weight": "model-00098-of-00101.safetensors", + "model.layers.89.mlp.experts.70.up_proj.weight": "model-00098-of-00101.safetensors", + "model.layers.89.mlp.experts.70.down_proj.weight": "model-00098-of-00101.safetensors", + "model.layers.89.mlp.experts.71.gate_proj.weight": "model-00098-of-00101.safetensors", + "model.layers.89.mlp.experts.71.up_proj.weight": "model-00098-of-00101.safetensors", + "model.layers.89.mlp.experts.71.down_proj.weight": "model-00098-of-00101.safetensors", + "model.layers.89.mlp.experts.72.gate_proj.weight": "model-00098-of-00101.safetensors", + "model.layers.89.mlp.experts.72.up_proj.weight": "model-00098-of-00101.safetensors", + "model.layers.89.mlp.experts.72.down_proj.weight": "model-00098-of-00101.safetensors", + "model.layers.89.mlp.experts.73.gate_proj.weight": "model-00098-of-00101.safetensors", + "model.layers.89.mlp.experts.73.up_proj.weight": "model-00098-of-00101.safetensors", + "model.layers.89.mlp.experts.73.down_proj.weight": "model-00098-of-00101.safetensors", + "model.layers.89.mlp.experts.74.gate_proj.weight": "model-00098-of-00101.safetensors", + "model.layers.89.mlp.experts.74.up_proj.weight": "model-00098-of-00101.safetensors", + "model.layers.89.mlp.experts.74.down_proj.weight": "model-00098-of-00101.safetensors", + "model.layers.89.mlp.experts.75.gate_proj.weight": "model-00098-of-00101.safetensors", + "model.layers.89.mlp.experts.75.up_proj.weight": "model-00098-of-00101.safetensors", + "model.layers.89.mlp.experts.75.down_proj.weight": "model-00098-of-00101.safetensors", + "model.layers.89.mlp.experts.76.gate_proj.weight": "model-00098-of-00101.safetensors", + "model.layers.89.mlp.experts.76.up_proj.weight": "model-00098-of-00101.safetensors", + "model.layers.89.mlp.experts.76.down_proj.weight": "model-00098-of-00101.safetensors", + "model.layers.89.mlp.experts.77.gate_proj.weight": "model-00098-of-00101.safetensors", + "model.layers.89.mlp.experts.77.up_proj.weight": "model-00098-of-00101.safetensors", + "model.layers.89.mlp.experts.77.down_proj.weight": "model-00098-of-00101.safetensors", + "model.layers.89.mlp.experts.78.gate_proj.weight": "model-00098-of-00101.safetensors", + "model.layers.89.mlp.experts.78.up_proj.weight": "model-00098-of-00101.safetensors", + "model.layers.89.mlp.experts.78.down_proj.weight": "model-00098-of-00101.safetensors", + "model.layers.89.mlp.experts.79.gate_proj.weight": "model-00098-of-00101.safetensors", + "model.layers.89.mlp.experts.79.up_proj.weight": "model-00098-of-00101.safetensors", + "model.layers.89.mlp.experts.79.down_proj.weight": "model-00098-of-00101.safetensors", + "model.layers.89.mlp.experts.80.gate_proj.weight": "model-00098-of-00101.safetensors", + "model.layers.89.mlp.experts.80.up_proj.weight": "model-00098-of-00101.safetensors", + "model.layers.89.mlp.experts.80.down_proj.weight": "model-00098-of-00101.safetensors", + "model.layers.89.mlp.experts.81.gate_proj.weight": "model-00098-of-00101.safetensors", + "model.layers.89.mlp.experts.81.up_proj.weight": "model-00098-of-00101.safetensors", + "model.layers.89.mlp.experts.81.down_proj.weight": "model-00098-of-00101.safetensors", + "model.layers.89.mlp.experts.82.gate_proj.weight": "model-00098-of-00101.safetensors", + "model.layers.89.mlp.experts.82.up_proj.weight": "model-00098-of-00101.safetensors", + "model.layers.89.mlp.experts.82.down_proj.weight": "model-00098-of-00101.safetensors", + "model.layers.89.mlp.experts.83.gate_proj.weight": "model-00098-of-00101.safetensors", + "model.layers.89.mlp.experts.83.up_proj.weight": "model-00098-of-00101.safetensors", + "model.layers.89.mlp.experts.83.down_proj.weight": "model-00098-of-00101.safetensors", + "model.layers.89.mlp.experts.84.gate_proj.weight": "model-00098-of-00101.safetensors", + "model.layers.89.mlp.experts.84.up_proj.weight": "model-00098-of-00101.safetensors", + "model.layers.89.mlp.experts.84.down_proj.weight": "model-00098-of-00101.safetensors", + "model.layers.89.mlp.experts.85.gate_proj.weight": "model-00098-of-00101.safetensors", + "model.layers.89.mlp.experts.85.up_proj.weight": "model-00098-of-00101.safetensors", + "model.layers.89.mlp.experts.85.down_proj.weight": "model-00098-of-00101.safetensors", + "model.layers.89.mlp.experts.86.gate_proj.weight": "model-00098-of-00101.safetensors", + "model.layers.89.mlp.experts.86.up_proj.weight": "model-00098-of-00101.safetensors", + "model.layers.89.mlp.experts.86.down_proj.weight": "model-00098-of-00101.safetensors", + "model.layers.89.mlp.experts.87.gate_proj.weight": "model-00098-of-00101.safetensors", + "model.layers.89.mlp.experts.87.up_proj.weight": "model-00098-of-00101.safetensors", + "model.layers.89.mlp.experts.87.down_proj.weight": "model-00098-of-00101.safetensors", + "model.layers.89.mlp.experts.88.gate_proj.weight": "model-00098-of-00101.safetensors", + "model.layers.89.mlp.experts.88.up_proj.weight": "model-00098-of-00101.safetensors", + "model.layers.89.mlp.experts.88.down_proj.weight": "model-00098-of-00101.safetensors", + "model.layers.89.mlp.experts.89.gate_proj.weight": "model-00098-of-00101.safetensors", + "model.layers.89.mlp.experts.89.up_proj.weight": "model-00098-of-00101.safetensors", + "model.layers.89.mlp.experts.89.down_proj.weight": "model-00098-of-00101.safetensors", + "model.layers.89.mlp.experts.90.gate_proj.weight": "model-00098-of-00101.safetensors", + "model.layers.89.mlp.experts.90.up_proj.weight": "model-00098-of-00101.safetensors", + "model.layers.89.mlp.experts.90.down_proj.weight": "model-00098-of-00101.safetensors", + "model.layers.89.mlp.experts.91.gate_proj.weight": "model-00098-of-00101.safetensors", + "model.layers.89.mlp.experts.91.up_proj.weight": "model-00098-of-00101.safetensors", + "model.layers.89.mlp.experts.91.down_proj.weight": "model-00098-of-00101.safetensors", + "model.layers.89.mlp.experts.92.gate_proj.weight": "model-00098-of-00101.safetensors", + "model.layers.89.mlp.experts.92.up_proj.weight": "model-00098-of-00101.safetensors", + "model.layers.89.mlp.experts.92.down_proj.weight": "model-00098-of-00101.safetensors", + "model.layers.89.mlp.experts.93.gate_proj.weight": "model-00098-of-00101.safetensors", + "model.layers.89.mlp.experts.93.up_proj.weight": "model-00098-of-00101.safetensors", + "model.layers.89.mlp.experts.93.down_proj.weight": "model-00098-of-00101.safetensors", + "model.layers.89.mlp.experts.94.gate_proj.weight": "model-00098-of-00101.safetensors", + "model.layers.89.mlp.experts.94.up_proj.weight": "model-00098-of-00101.safetensors", + "model.layers.89.mlp.experts.94.down_proj.weight": "model-00098-of-00101.safetensors", + "model.layers.89.mlp.experts.95.gate_proj.weight": "model-00098-of-00101.safetensors", + "model.layers.89.mlp.experts.95.up_proj.weight": "model-00098-of-00101.safetensors", + "model.layers.89.mlp.experts.95.down_proj.weight": "model-00098-of-00101.safetensors", + "model.layers.89.mlp.experts.96.gate_proj.weight": "model-00098-of-00101.safetensors", + "model.layers.89.mlp.experts.96.up_proj.weight": "model-00098-of-00101.safetensors", + "model.layers.89.mlp.experts.96.down_proj.weight": "model-00098-of-00101.safetensors", + "model.layers.89.mlp.experts.97.gate_proj.weight": "model-00098-of-00101.safetensors", + "model.layers.89.mlp.experts.97.up_proj.weight": "model-00098-of-00101.safetensors", + "model.layers.89.mlp.experts.97.down_proj.weight": "model-00098-of-00101.safetensors", + "model.layers.89.mlp.experts.98.gate_proj.weight": "model-00098-of-00101.safetensors", + "model.layers.89.mlp.experts.98.up_proj.weight": "model-00098-of-00101.safetensors", + "model.layers.89.mlp.experts.98.down_proj.weight": "model-00098-of-00101.safetensors", + "model.layers.89.mlp.experts.99.gate_proj.weight": "model-00098-of-00101.safetensors", + "model.layers.89.mlp.experts.99.up_proj.weight": "model-00098-of-00101.safetensors", + "model.layers.89.mlp.experts.99.down_proj.weight": "model-00098-of-00101.safetensors", + "model.layers.89.mlp.experts.100.gate_proj.weight": "model-00098-of-00101.safetensors", + "model.layers.89.mlp.experts.100.up_proj.weight": "model-00098-of-00101.safetensors", + "model.layers.89.mlp.experts.100.down_proj.weight": "model-00098-of-00101.safetensors", + "model.layers.89.mlp.experts.101.gate_proj.weight": "model-00098-of-00101.safetensors", + "model.layers.89.mlp.experts.101.up_proj.weight": "model-00098-of-00101.safetensors", + "model.layers.89.mlp.experts.101.down_proj.weight": "model-00098-of-00101.safetensors", + "model.layers.89.mlp.experts.102.gate_proj.weight": "model-00098-of-00101.safetensors", + "model.layers.89.mlp.experts.102.up_proj.weight": "model-00098-of-00101.safetensors", + "model.layers.89.mlp.experts.102.down_proj.weight": "model-00098-of-00101.safetensors", + "model.layers.89.mlp.experts.103.gate_proj.weight": "model-00098-of-00101.safetensors", + "model.layers.89.mlp.experts.103.up_proj.weight": "model-00098-of-00101.safetensors", + "model.layers.89.mlp.experts.103.down_proj.weight": "model-00098-of-00101.safetensors", + "model.layers.89.mlp.experts.104.gate_proj.weight": "model-00098-of-00101.safetensors", + "model.layers.89.mlp.experts.104.up_proj.weight": "model-00098-of-00101.safetensors", + "model.layers.89.mlp.experts.104.down_proj.weight": "model-00098-of-00101.safetensors", + "model.layers.89.mlp.experts.105.gate_proj.weight": "model-00098-of-00101.safetensors", + "model.layers.89.mlp.experts.105.up_proj.weight": "model-00098-of-00101.safetensors", + "model.layers.89.mlp.experts.105.down_proj.weight": "model-00098-of-00101.safetensors", + "model.layers.89.mlp.experts.106.gate_proj.weight": "model-00098-of-00101.safetensors", + "model.layers.89.mlp.experts.106.up_proj.weight": "model-00098-of-00101.safetensors", + "model.layers.89.mlp.experts.106.down_proj.weight": "model-00098-of-00101.safetensors", + "model.layers.89.mlp.experts.107.gate_proj.weight": "model-00098-of-00101.safetensors", + "model.layers.89.mlp.experts.107.up_proj.weight": "model-00098-of-00101.safetensors", + "model.layers.89.mlp.experts.107.down_proj.weight": "model-00098-of-00101.safetensors", + "model.layers.89.mlp.experts.108.gate_proj.weight": "model-00098-of-00101.safetensors", + "model.layers.89.mlp.experts.108.up_proj.weight": "model-00098-of-00101.safetensors", + "model.layers.89.mlp.experts.108.down_proj.weight": "model-00098-of-00101.safetensors", + "model.layers.89.mlp.experts.109.gate_proj.weight": "model-00098-of-00101.safetensors", + "model.layers.89.mlp.experts.109.up_proj.weight": "model-00098-of-00101.safetensors", + "model.layers.89.mlp.experts.109.down_proj.weight": "model-00098-of-00101.safetensors", + "model.layers.89.mlp.experts.110.gate_proj.weight": "model-00098-of-00101.safetensors", + "model.layers.89.mlp.experts.110.up_proj.weight": "model-00098-of-00101.safetensors", + "model.layers.89.mlp.experts.110.down_proj.weight": "model-00098-of-00101.safetensors", + "model.layers.89.mlp.experts.111.gate_proj.weight": "model-00098-of-00101.safetensors", + "model.layers.89.mlp.experts.111.up_proj.weight": "model-00098-of-00101.safetensors", + "model.layers.89.mlp.experts.111.down_proj.weight": "model-00098-of-00101.safetensors", + "model.layers.89.mlp.experts.112.gate_proj.weight": "model-00098-of-00101.safetensors", + "model.layers.89.mlp.experts.112.up_proj.weight": "model-00098-of-00101.safetensors", + "model.layers.89.mlp.experts.112.down_proj.weight": "model-00098-of-00101.safetensors", + "model.layers.89.mlp.experts.113.gate_proj.weight": "model-00098-of-00101.safetensors", + "model.layers.89.mlp.experts.113.up_proj.weight": "model-00098-of-00101.safetensors", + "model.layers.89.mlp.experts.113.down_proj.weight": "model-00098-of-00101.safetensors", + "model.layers.89.mlp.experts.114.gate_proj.weight": "model-00098-of-00101.safetensors", + "model.layers.89.mlp.experts.114.up_proj.weight": "model-00098-of-00101.safetensors", + "model.layers.89.mlp.experts.114.down_proj.weight": "model-00098-of-00101.safetensors", + "model.layers.89.mlp.experts.115.gate_proj.weight": "model-00098-of-00101.safetensors", + "model.layers.89.mlp.experts.115.up_proj.weight": "model-00098-of-00101.safetensors", + "model.layers.89.mlp.experts.115.down_proj.weight": "model-00098-of-00101.safetensors", + "model.layers.89.mlp.experts.116.gate_proj.weight": "model-00098-of-00101.safetensors", + "model.layers.89.mlp.experts.116.up_proj.weight": "model-00098-of-00101.safetensors", + "model.layers.89.mlp.experts.116.down_proj.weight": "model-00098-of-00101.safetensors", + "model.layers.89.mlp.experts.117.gate_proj.weight": "model-00098-of-00101.safetensors", + "model.layers.89.mlp.experts.117.up_proj.weight": "model-00098-of-00101.safetensors", + "model.layers.89.mlp.experts.117.down_proj.weight": "model-00098-of-00101.safetensors", + "model.layers.89.mlp.experts.118.gate_proj.weight": "model-00098-of-00101.safetensors", + "model.layers.89.mlp.experts.118.up_proj.weight": "model-00098-of-00101.safetensors", + "model.layers.89.mlp.experts.118.down_proj.weight": "model-00098-of-00101.safetensors", + "model.layers.89.mlp.experts.119.gate_proj.weight": "model-00098-of-00101.safetensors", + "model.layers.89.mlp.experts.119.up_proj.weight": "model-00098-of-00101.safetensors", + "model.layers.89.mlp.experts.119.down_proj.weight": "model-00098-of-00101.safetensors", + "model.layers.89.mlp.gate.weight": "model-00098-of-00101.safetensors", + "model.layers.89.mlp.gate.e_score_correction_bias": "model-00098-of-00101.safetensors", + "model.layers.89.mlp.shared_experts.gate_proj.weight": "model-00098-of-00101.safetensors", + "model.layers.89.mlp.shared_experts.up_proj.weight": "model-00098-of-00101.safetensors", + "model.layers.89.mlp.shared_experts.down_proj.weight": "model-00098-of-00101.safetensors", + "model.layers.89.input_layernorm.weight": "model-00098-of-00101.safetensors", + "model.layers.89.post_attention_layernorm.weight": "model-00098-of-00101.safetensors", + "model.layers.90.self_attn.q_proj.weight": "model-00098-of-00101.safetensors", + "model.layers.90.self_attn.q_proj.bias": "model-00098-of-00101.safetensors", + "model.layers.90.self_attn.k_proj.weight": "model-00098-of-00101.safetensors", + "model.layers.90.self_attn.k_proj.bias": "model-00098-of-00101.safetensors", + "model.layers.90.self_attn.v_proj.weight": "model-00098-of-00101.safetensors", + "model.layers.90.self_attn.v_proj.bias": "model-00098-of-00101.safetensors", + "model.layers.90.self_attn.o_proj.weight": "model-00098-of-00101.safetensors", + "model.layers.90.self_attn.q_norm.weight": "model-00098-of-00101.safetensors", + "model.layers.90.self_attn.k_norm.weight": "model-00098-of-00101.safetensors", + "model.layers.90.mlp.experts.0.gate_proj.weight": "model-00098-of-00101.safetensors", + "model.layers.90.mlp.experts.0.up_proj.weight": "model-00098-of-00101.safetensors", + "model.layers.90.mlp.experts.0.down_proj.weight": "model-00098-of-00101.safetensors", + "model.layers.90.mlp.experts.1.gate_proj.weight": "model-00098-of-00101.safetensors", + "model.layers.90.mlp.experts.1.up_proj.weight": "model-00098-of-00101.safetensors", + "model.layers.90.mlp.experts.1.down_proj.weight": "model-00098-of-00101.safetensors", + "model.layers.90.mlp.experts.2.gate_proj.weight": "model-00098-of-00101.safetensors", + "model.layers.90.mlp.experts.2.up_proj.weight": "model-00098-of-00101.safetensors", + "model.layers.90.mlp.experts.2.down_proj.weight": "model-00098-of-00101.safetensors", + "model.layers.90.mlp.experts.3.gate_proj.weight": "model-00098-of-00101.safetensors", + "model.layers.90.mlp.experts.3.up_proj.weight": "model-00098-of-00101.safetensors", + "model.layers.90.mlp.experts.3.down_proj.weight": "model-00098-of-00101.safetensors", + "model.layers.90.mlp.experts.4.gate_proj.weight": "model-00098-of-00101.safetensors", + "model.layers.90.mlp.experts.4.up_proj.weight": "model-00098-of-00101.safetensors", + "model.layers.90.mlp.experts.4.down_proj.weight": "model-00098-of-00101.safetensors", + "model.layers.90.mlp.experts.5.gate_proj.weight": "model-00098-of-00101.safetensors", + "model.layers.90.mlp.experts.5.up_proj.weight": "model-00098-of-00101.safetensors", + "model.layers.90.mlp.experts.5.down_proj.weight": "model-00098-of-00101.safetensors", + "model.layers.90.mlp.experts.6.gate_proj.weight": "model-00098-of-00101.safetensors", + "model.layers.90.mlp.experts.6.up_proj.weight": "model-00098-of-00101.safetensors", + "model.layers.90.mlp.experts.6.down_proj.weight": "model-00098-of-00101.safetensors", + "model.layers.90.mlp.experts.7.gate_proj.weight": "model-00098-of-00101.safetensors", + "model.layers.90.mlp.experts.7.up_proj.weight": "model-00098-of-00101.safetensors", + "model.layers.90.mlp.experts.7.down_proj.weight": "model-00098-of-00101.safetensors", + "model.layers.90.mlp.experts.8.gate_proj.weight": "model-00099-of-00101.safetensors", + "model.layers.90.mlp.experts.8.up_proj.weight": "model-00099-of-00101.safetensors", + "model.layers.90.mlp.experts.8.down_proj.weight": "model-00099-of-00101.safetensors", + "model.layers.90.mlp.experts.9.gate_proj.weight": "model-00099-of-00101.safetensors", + "model.layers.90.mlp.experts.9.up_proj.weight": "model-00099-of-00101.safetensors", + "model.layers.90.mlp.experts.9.down_proj.weight": "model-00099-of-00101.safetensors", + "model.layers.90.mlp.experts.10.gate_proj.weight": "model-00099-of-00101.safetensors", + "model.layers.90.mlp.experts.10.up_proj.weight": "model-00099-of-00101.safetensors", + "model.layers.90.mlp.experts.10.down_proj.weight": "model-00099-of-00101.safetensors", + "model.layers.90.mlp.experts.11.gate_proj.weight": "model-00099-of-00101.safetensors", + "model.layers.90.mlp.experts.11.up_proj.weight": "model-00099-of-00101.safetensors", + "model.layers.90.mlp.experts.11.down_proj.weight": "model-00099-of-00101.safetensors", + "model.layers.90.mlp.experts.12.gate_proj.weight": "model-00099-of-00101.safetensors", + "model.layers.90.mlp.experts.12.up_proj.weight": "model-00099-of-00101.safetensors", + "model.layers.90.mlp.experts.12.down_proj.weight": "model-00099-of-00101.safetensors", + "model.layers.90.mlp.experts.13.gate_proj.weight": "model-00099-of-00101.safetensors", + "model.layers.90.mlp.experts.13.up_proj.weight": "model-00099-of-00101.safetensors", + "model.layers.90.mlp.experts.13.down_proj.weight": "model-00099-of-00101.safetensors", + "model.layers.90.mlp.experts.14.gate_proj.weight": "model-00099-of-00101.safetensors", + "model.layers.90.mlp.experts.14.up_proj.weight": "model-00099-of-00101.safetensors", + "model.layers.90.mlp.experts.14.down_proj.weight": "model-00099-of-00101.safetensors", + "model.layers.90.mlp.experts.15.gate_proj.weight": "model-00099-of-00101.safetensors", + "model.layers.90.mlp.experts.15.up_proj.weight": "model-00099-of-00101.safetensors", + "model.layers.90.mlp.experts.15.down_proj.weight": "model-00099-of-00101.safetensors", + "model.layers.90.mlp.experts.16.gate_proj.weight": "model-00099-of-00101.safetensors", + "model.layers.90.mlp.experts.16.up_proj.weight": "model-00099-of-00101.safetensors", + "model.layers.90.mlp.experts.16.down_proj.weight": "model-00099-of-00101.safetensors", + "model.layers.90.mlp.experts.17.gate_proj.weight": "model-00099-of-00101.safetensors", + "model.layers.90.mlp.experts.17.up_proj.weight": "model-00099-of-00101.safetensors", + "model.layers.90.mlp.experts.17.down_proj.weight": "model-00099-of-00101.safetensors", + "model.layers.90.mlp.experts.18.gate_proj.weight": "model-00099-of-00101.safetensors", + "model.layers.90.mlp.experts.18.up_proj.weight": "model-00099-of-00101.safetensors", + "model.layers.90.mlp.experts.18.down_proj.weight": "model-00099-of-00101.safetensors", + "model.layers.90.mlp.experts.19.gate_proj.weight": "model-00099-of-00101.safetensors", + "model.layers.90.mlp.experts.19.up_proj.weight": "model-00099-of-00101.safetensors", + "model.layers.90.mlp.experts.19.down_proj.weight": "model-00099-of-00101.safetensors", + "model.layers.90.mlp.experts.20.gate_proj.weight": "model-00099-of-00101.safetensors", + "model.layers.90.mlp.experts.20.up_proj.weight": "model-00099-of-00101.safetensors", + "model.layers.90.mlp.experts.20.down_proj.weight": "model-00099-of-00101.safetensors", + "model.layers.90.mlp.experts.21.gate_proj.weight": "model-00099-of-00101.safetensors", + "model.layers.90.mlp.experts.21.up_proj.weight": "model-00099-of-00101.safetensors", + "model.layers.90.mlp.experts.21.down_proj.weight": "model-00099-of-00101.safetensors", + "model.layers.90.mlp.experts.22.gate_proj.weight": "model-00099-of-00101.safetensors", + "model.layers.90.mlp.experts.22.up_proj.weight": "model-00099-of-00101.safetensors", + "model.layers.90.mlp.experts.22.down_proj.weight": "model-00099-of-00101.safetensors", + "model.layers.90.mlp.experts.23.gate_proj.weight": "model-00099-of-00101.safetensors", + "model.layers.90.mlp.experts.23.up_proj.weight": "model-00099-of-00101.safetensors", + "model.layers.90.mlp.experts.23.down_proj.weight": "model-00099-of-00101.safetensors", + "model.layers.90.mlp.experts.24.gate_proj.weight": "model-00099-of-00101.safetensors", + "model.layers.90.mlp.experts.24.up_proj.weight": "model-00099-of-00101.safetensors", + "model.layers.90.mlp.experts.24.down_proj.weight": "model-00099-of-00101.safetensors", + "model.layers.90.mlp.experts.25.gate_proj.weight": "model-00099-of-00101.safetensors", + "model.layers.90.mlp.experts.25.up_proj.weight": "model-00099-of-00101.safetensors", + "model.layers.90.mlp.experts.25.down_proj.weight": "model-00099-of-00101.safetensors", + "model.layers.90.mlp.experts.26.gate_proj.weight": "model-00099-of-00101.safetensors", + "model.layers.90.mlp.experts.26.up_proj.weight": "model-00099-of-00101.safetensors", + "model.layers.90.mlp.experts.26.down_proj.weight": "model-00099-of-00101.safetensors", + "model.layers.90.mlp.experts.27.gate_proj.weight": "model-00099-of-00101.safetensors", + "model.layers.90.mlp.experts.27.up_proj.weight": "model-00099-of-00101.safetensors", + "model.layers.90.mlp.experts.27.down_proj.weight": "model-00099-of-00101.safetensors", + "model.layers.90.mlp.experts.28.gate_proj.weight": "model-00099-of-00101.safetensors", + "model.layers.90.mlp.experts.28.up_proj.weight": "model-00099-of-00101.safetensors", + "model.layers.90.mlp.experts.28.down_proj.weight": "model-00099-of-00101.safetensors", + "model.layers.90.mlp.experts.29.gate_proj.weight": "model-00099-of-00101.safetensors", + "model.layers.90.mlp.experts.29.up_proj.weight": "model-00099-of-00101.safetensors", + "model.layers.90.mlp.experts.29.down_proj.weight": "model-00099-of-00101.safetensors", + "model.layers.90.mlp.experts.30.gate_proj.weight": "model-00099-of-00101.safetensors", + "model.layers.90.mlp.experts.30.up_proj.weight": "model-00099-of-00101.safetensors", + "model.layers.90.mlp.experts.30.down_proj.weight": "model-00099-of-00101.safetensors", + "model.layers.90.mlp.experts.31.gate_proj.weight": "model-00099-of-00101.safetensors", + "model.layers.90.mlp.experts.31.up_proj.weight": "model-00099-of-00101.safetensors", + "model.layers.90.mlp.experts.31.down_proj.weight": "model-00099-of-00101.safetensors", + "model.layers.90.mlp.experts.32.gate_proj.weight": "model-00099-of-00101.safetensors", + "model.layers.90.mlp.experts.32.up_proj.weight": "model-00099-of-00101.safetensors", + "model.layers.90.mlp.experts.32.down_proj.weight": "model-00099-of-00101.safetensors", + "model.layers.90.mlp.experts.33.gate_proj.weight": "model-00099-of-00101.safetensors", + "model.layers.90.mlp.experts.33.up_proj.weight": "model-00099-of-00101.safetensors", + "model.layers.90.mlp.experts.33.down_proj.weight": "model-00099-of-00101.safetensors", + "model.layers.90.mlp.experts.34.gate_proj.weight": "model-00099-of-00101.safetensors", + "model.layers.90.mlp.experts.34.up_proj.weight": "model-00099-of-00101.safetensors", + "model.layers.90.mlp.experts.34.down_proj.weight": "model-00099-of-00101.safetensors", + "model.layers.90.mlp.experts.35.gate_proj.weight": "model-00099-of-00101.safetensors", + "model.layers.90.mlp.experts.35.up_proj.weight": "model-00099-of-00101.safetensors", + "model.layers.90.mlp.experts.35.down_proj.weight": "model-00099-of-00101.safetensors", + "model.layers.90.mlp.experts.36.gate_proj.weight": "model-00099-of-00101.safetensors", + "model.layers.90.mlp.experts.36.up_proj.weight": "model-00099-of-00101.safetensors", + "model.layers.90.mlp.experts.36.down_proj.weight": "model-00099-of-00101.safetensors", + "model.layers.90.mlp.experts.37.gate_proj.weight": "model-00099-of-00101.safetensors", + "model.layers.90.mlp.experts.37.up_proj.weight": "model-00099-of-00101.safetensors", + "model.layers.90.mlp.experts.37.down_proj.weight": "model-00099-of-00101.safetensors", + "model.layers.90.mlp.experts.38.gate_proj.weight": "model-00099-of-00101.safetensors", + "model.layers.90.mlp.experts.38.up_proj.weight": "model-00099-of-00101.safetensors", + "model.layers.90.mlp.experts.38.down_proj.weight": "model-00099-of-00101.safetensors", + "model.layers.90.mlp.experts.39.gate_proj.weight": "model-00099-of-00101.safetensors", + "model.layers.90.mlp.experts.39.up_proj.weight": "model-00099-of-00101.safetensors", + "model.layers.90.mlp.experts.39.down_proj.weight": "model-00099-of-00101.safetensors", + "model.layers.90.mlp.experts.40.gate_proj.weight": "model-00099-of-00101.safetensors", + "model.layers.90.mlp.experts.40.up_proj.weight": "model-00099-of-00101.safetensors", + "model.layers.90.mlp.experts.40.down_proj.weight": "model-00099-of-00101.safetensors", + "model.layers.90.mlp.experts.41.gate_proj.weight": "model-00099-of-00101.safetensors", + "model.layers.90.mlp.experts.41.up_proj.weight": "model-00099-of-00101.safetensors", + "model.layers.90.mlp.experts.41.down_proj.weight": "model-00099-of-00101.safetensors", + "model.layers.90.mlp.experts.42.gate_proj.weight": "model-00099-of-00101.safetensors", + "model.layers.90.mlp.experts.42.up_proj.weight": "model-00099-of-00101.safetensors", + "model.layers.90.mlp.experts.42.down_proj.weight": "model-00099-of-00101.safetensors", + "model.layers.90.mlp.experts.43.gate_proj.weight": "model-00099-of-00101.safetensors", + "model.layers.90.mlp.experts.43.up_proj.weight": "model-00099-of-00101.safetensors", + "model.layers.90.mlp.experts.43.down_proj.weight": "model-00099-of-00101.safetensors", + "model.layers.90.mlp.experts.44.gate_proj.weight": "model-00099-of-00101.safetensors", + "model.layers.90.mlp.experts.44.up_proj.weight": "model-00099-of-00101.safetensors", + "model.layers.90.mlp.experts.44.down_proj.weight": "model-00099-of-00101.safetensors", + "model.layers.90.mlp.experts.45.gate_proj.weight": "model-00099-of-00101.safetensors", + "model.layers.90.mlp.experts.45.up_proj.weight": "model-00099-of-00101.safetensors", + "model.layers.90.mlp.experts.45.down_proj.weight": "model-00099-of-00101.safetensors", + "model.layers.90.mlp.experts.46.gate_proj.weight": "model-00099-of-00101.safetensors", + "model.layers.90.mlp.experts.46.up_proj.weight": "model-00099-of-00101.safetensors", + "model.layers.90.mlp.experts.46.down_proj.weight": "model-00099-of-00101.safetensors", + "model.layers.90.mlp.experts.47.gate_proj.weight": "model-00099-of-00101.safetensors", + "model.layers.90.mlp.experts.47.up_proj.weight": "model-00099-of-00101.safetensors", + "model.layers.90.mlp.experts.47.down_proj.weight": "model-00099-of-00101.safetensors", + "model.layers.90.mlp.experts.48.gate_proj.weight": "model-00099-of-00101.safetensors", + "model.layers.90.mlp.experts.48.up_proj.weight": "model-00099-of-00101.safetensors", + "model.layers.90.mlp.experts.48.down_proj.weight": "model-00099-of-00101.safetensors", + "model.layers.90.mlp.experts.49.gate_proj.weight": "model-00099-of-00101.safetensors", + "model.layers.90.mlp.experts.49.up_proj.weight": "model-00099-of-00101.safetensors", + "model.layers.90.mlp.experts.49.down_proj.weight": "model-00099-of-00101.safetensors", + "model.layers.90.mlp.experts.50.gate_proj.weight": "model-00099-of-00101.safetensors", + "model.layers.90.mlp.experts.50.up_proj.weight": "model-00099-of-00101.safetensors", + "model.layers.90.mlp.experts.50.down_proj.weight": "model-00099-of-00101.safetensors", + "model.layers.90.mlp.experts.51.gate_proj.weight": "model-00099-of-00101.safetensors", + "model.layers.90.mlp.experts.51.up_proj.weight": "model-00099-of-00101.safetensors", + "model.layers.90.mlp.experts.51.down_proj.weight": "model-00099-of-00101.safetensors", + "model.layers.90.mlp.experts.52.gate_proj.weight": "model-00099-of-00101.safetensors", + "model.layers.90.mlp.experts.52.up_proj.weight": "model-00099-of-00101.safetensors", + "model.layers.90.mlp.experts.52.down_proj.weight": "model-00099-of-00101.safetensors", + "model.layers.90.mlp.experts.53.gate_proj.weight": "model-00099-of-00101.safetensors", + "model.layers.90.mlp.experts.53.up_proj.weight": "model-00099-of-00101.safetensors", + "model.layers.90.mlp.experts.53.down_proj.weight": "model-00099-of-00101.safetensors", + "model.layers.90.mlp.experts.54.gate_proj.weight": "model-00099-of-00101.safetensors", + "model.layers.90.mlp.experts.54.up_proj.weight": "model-00099-of-00101.safetensors", + "model.layers.90.mlp.experts.54.down_proj.weight": "model-00099-of-00101.safetensors", + "model.layers.90.mlp.experts.55.gate_proj.weight": "model-00099-of-00101.safetensors", + "model.layers.90.mlp.experts.55.up_proj.weight": "model-00099-of-00101.safetensors", + "model.layers.90.mlp.experts.55.down_proj.weight": "model-00099-of-00101.safetensors", + "model.layers.90.mlp.experts.56.gate_proj.weight": "model-00099-of-00101.safetensors", + "model.layers.90.mlp.experts.56.up_proj.weight": "model-00099-of-00101.safetensors", + "model.layers.90.mlp.experts.56.down_proj.weight": "model-00099-of-00101.safetensors", + "model.layers.90.mlp.experts.57.gate_proj.weight": "model-00099-of-00101.safetensors", + "model.layers.90.mlp.experts.57.up_proj.weight": "model-00099-of-00101.safetensors", + "model.layers.90.mlp.experts.57.down_proj.weight": "model-00099-of-00101.safetensors", + "model.layers.90.mlp.experts.58.gate_proj.weight": "model-00099-of-00101.safetensors", + "model.layers.90.mlp.experts.58.up_proj.weight": "model-00099-of-00101.safetensors", + "model.layers.90.mlp.experts.58.down_proj.weight": "model-00099-of-00101.safetensors", + "model.layers.90.mlp.experts.59.gate_proj.weight": "model-00099-of-00101.safetensors", + "model.layers.90.mlp.experts.59.up_proj.weight": "model-00099-of-00101.safetensors", + "model.layers.90.mlp.experts.59.down_proj.weight": "model-00099-of-00101.safetensors", + "model.layers.90.mlp.experts.60.gate_proj.weight": "model-00099-of-00101.safetensors", + "model.layers.90.mlp.experts.60.up_proj.weight": "model-00099-of-00101.safetensors", + "model.layers.90.mlp.experts.60.down_proj.weight": "model-00099-of-00101.safetensors", + "model.layers.90.mlp.experts.61.gate_proj.weight": "model-00099-of-00101.safetensors", + "model.layers.90.mlp.experts.61.up_proj.weight": "model-00099-of-00101.safetensors", + "model.layers.90.mlp.experts.61.down_proj.weight": "model-00099-of-00101.safetensors", + "model.layers.90.mlp.experts.62.gate_proj.weight": "model-00099-of-00101.safetensors", + "model.layers.90.mlp.experts.62.up_proj.weight": "model-00099-of-00101.safetensors", + "model.layers.90.mlp.experts.62.down_proj.weight": "model-00099-of-00101.safetensors", + "model.layers.90.mlp.experts.63.gate_proj.weight": "model-00099-of-00101.safetensors", + "model.layers.90.mlp.experts.63.up_proj.weight": "model-00099-of-00101.safetensors", + "model.layers.90.mlp.experts.63.down_proj.weight": "model-00099-of-00101.safetensors", + "model.layers.90.mlp.experts.64.gate_proj.weight": "model-00099-of-00101.safetensors", + "model.layers.90.mlp.experts.64.up_proj.weight": "model-00099-of-00101.safetensors", + "model.layers.90.mlp.experts.64.down_proj.weight": "model-00099-of-00101.safetensors", + "model.layers.90.mlp.experts.65.gate_proj.weight": "model-00099-of-00101.safetensors", + "model.layers.90.mlp.experts.65.up_proj.weight": "model-00099-of-00101.safetensors", + "model.layers.90.mlp.experts.65.down_proj.weight": "model-00099-of-00101.safetensors", + "model.layers.90.mlp.experts.66.gate_proj.weight": "model-00099-of-00101.safetensors", + "model.layers.90.mlp.experts.66.up_proj.weight": "model-00099-of-00101.safetensors", + "model.layers.90.mlp.experts.66.down_proj.weight": "model-00099-of-00101.safetensors", + "model.layers.90.mlp.experts.67.gate_proj.weight": "model-00099-of-00101.safetensors", + "model.layers.90.mlp.experts.67.up_proj.weight": "model-00099-of-00101.safetensors", + "model.layers.90.mlp.experts.67.down_proj.weight": "model-00099-of-00101.safetensors", + "model.layers.90.mlp.experts.68.gate_proj.weight": "model-00099-of-00101.safetensors", + "model.layers.90.mlp.experts.68.up_proj.weight": "model-00099-of-00101.safetensors", + "model.layers.90.mlp.experts.68.down_proj.weight": "model-00099-of-00101.safetensors", + "model.layers.90.mlp.experts.69.gate_proj.weight": "model-00099-of-00101.safetensors", + "model.layers.90.mlp.experts.69.up_proj.weight": "model-00099-of-00101.safetensors", + "model.layers.90.mlp.experts.69.down_proj.weight": "model-00099-of-00101.safetensors", + "model.layers.90.mlp.experts.70.gate_proj.weight": "model-00099-of-00101.safetensors", + "model.layers.90.mlp.experts.70.up_proj.weight": "model-00099-of-00101.safetensors", + "model.layers.90.mlp.experts.70.down_proj.weight": "model-00099-of-00101.safetensors", + "model.layers.90.mlp.experts.71.gate_proj.weight": "model-00099-of-00101.safetensors", + "model.layers.90.mlp.experts.71.up_proj.weight": "model-00099-of-00101.safetensors", + "model.layers.90.mlp.experts.71.down_proj.weight": "model-00099-of-00101.safetensors", + "model.layers.90.mlp.experts.72.gate_proj.weight": "model-00099-of-00101.safetensors", + "model.layers.90.mlp.experts.72.up_proj.weight": "model-00099-of-00101.safetensors", + "model.layers.90.mlp.experts.72.down_proj.weight": "model-00099-of-00101.safetensors", + "model.layers.90.mlp.experts.73.gate_proj.weight": "model-00099-of-00101.safetensors", + "model.layers.90.mlp.experts.73.up_proj.weight": "model-00099-of-00101.safetensors", + "model.layers.90.mlp.experts.73.down_proj.weight": "model-00099-of-00101.safetensors", + "model.layers.90.mlp.experts.74.gate_proj.weight": "model-00099-of-00101.safetensors", + "model.layers.90.mlp.experts.74.up_proj.weight": "model-00099-of-00101.safetensors", + "model.layers.90.mlp.experts.74.down_proj.weight": "model-00099-of-00101.safetensors", + "model.layers.90.mlp.experts.75.gate_proj.weight": "model-00099-of-00101.safetensors", + "model.layers.90.mlp.experts.75.up_proj.weight": "model-00099-of-00101.safetensors", + "model.layers.90.mlp.experts.75.down_proj.weight": "model-00099-of-00101.safetensors", + "model.layers.90.mlp.experts.76.gate_proj.weight": "model-00099-of-00101.safetensors", + "model.layers.90.mlp.experts.76.up_proj.weight": "model-00099-of-00101.safetensors", + "model.layers.90.mlp.experts.76.down_proj.weight": "model-00099-of-00101.safetensors", + "model.layers.90.mlp.experts.77.gate_proj.weight": "model-00099-of-00101.safetensors", + "model.layers.90.mlp.experts.77.up_proj.weight": "model-00099-of-00101.safetensors", + "model.layers.90.mlp.experts.77.down_proj.weight": "model-00099-of-00101.safetensors", + "model.layers.90.mlp.experts.78.gate_proj.weight": "model-00099-of-00101.safetensors", + "model.layers.90.mlp.experts.78.up_proj.weight": "model-00099-of-00101.safetensors", + "model.layers.90.mlp.experts.78.down_proj.weight": "model-00099-of-00101.safetensors", + "model.layers.90.mlp.experts.79.gate_proj.weight": "model-00099-of-00101.safetensors", + "model.layers.90.mlp.experts.79.up_proj.weight": "model-00099-of-00101.safetensors", + "model.layers.90.mlp.experts.79.down_proj.weight": "model-00099-of-00101.safetensors", + "model.layers.90.mlp.experts.80.gate_proj.weight": "model-00099-of-00101.safetensors", + "model.layers.90.mlp.experts.80.up_proj.weight": "model-00099-of-00101.safetensors", + "model.layers.90.mlp.experts.80.down_proj.weight": "model-00099-of-00101.safetensors", + "model.layers.90.mlp.experts.81.gate_proj.weight": "model-00099-of-00101.safetensors", + "model.layers.90.mlp.experts.81.up_proj.weight": "model-00099-of-00101.safetensors", + "model.layers.90.mlp.experts.81.down_proj.weight": "model-00099-of-00101.safetensors", + "model.layers.90.mlp.experts.82.gate_proj.weight": "model-00099-of-00101.safetensors", + "model.layers.90.mlp.experts.82.up_proj.weight": "model-00099-of-00101.safetensors", + "model.layers.90.mlp.experts.82.down_proj.weight": "model-00099-of-00101.safetensors", + "model.layers.90.mlp.experts.83.gate_proj.weight": "model-00099-of-00101.safetensors", + "model.layers.90.mlp.experts.83.up_proj.weight": "model-00099-of-00101.safetensors", + "model.layers.90.mlp.experts.83.down_proj.weight": "model-00099-of-00101.safetensors", + "model.layers.90.mlp.experts.84.gate_proj.weight": "model-00099-of-00101.safetensors", + "model.layers.90.mlp.experts.84.up_proj.weight": "model-00099-of-00101.safetensors", + "model.layers.90.mlp.experts.84.down_proj.weight": "model-00099-of-00101.safetensors", + "model.layers.90.mlp.experts.85.gate_proj.weight": "model-00099-of-00101.safetensors", + "model.layers.90.mlp.experts.85.up_proj.weight": "model-00099-of-00101.safetensors", + "model.layers.90.mlp.experts.85.down_proj.weight": "model-00099-of-00101.safetensors", + "model.layers.90.mlp.experts.86.gate_proj.weight": "model-00099-of-00101.safetensors", + "model.layers.90.mlp.experts.86.up_proj.weight": "model-00099-of-00101.safetensors", + "model.layers.90.mlp.experts.86.down_proj.weight": "model-00099-of-00101.safetensors", + "model.layers.90.mlp.experts.87.gate_proj.weight": "model-00099-of-00101.safetensors", + "model.layers.90.mlp.experts.87.up_proj.weight": "model-00099-of-00101.safetensors", + "model.layers.90.mlp.experts.87.down_proj.weight": "model-00099-of-00101.safetensors", + "model.layers.90.mlp.experts.88.gate_proj.weight": "model-00099-of-00101.safetensors", + "model.layers.90.mlp.experts.88.up_proj.weight": "model-00099-of-00101.safetensors", + "model.layers.90.mlp.experts.88.down_proj.weight": "model-00099-of-00101.safetensors", + "model.layers.90.mlp.experts.89.gate_proj.weight": "model-00099-of-00101.safetensors", + "model.layers.90.mlp.experts.89.up_proj.weight": "model-00099-of-00101.safetensors", + "model.layers.90.mlp.experts.89.down_proj.weight": "model-00099-of-00101.safetensors", + "model.layers.90.mlp.experts.90.gate_proj.weight": "model-00099-of-00101.safetensors", + "model.layers.90.mlp.experts.90.up_proj.weight": "model-00099-of-00101.safetensors", + "model.layers.90.mlp.experts.90.down_proj.weight": "model-00099-of-00101.safetensors", + "model.layers.90.mlp.experts.91.gate_proj.weight": "model-00099-of-00101.safetensors", + "model.layers.90.mlp.experts.91.up_proj.weight": "model-00099-of-00101.safetensors", + "model.layers.90.mlp.experts.91.down_proj.weight": "model-00099-of-00101.safetensors", + "model.layers.90.mlp.experts.92.gate_proj.weight": "model-00099-of-00101.safetensors", + "model.layers.90.mlp.experts.92.up_proj.weight": "model-00099-of-00101.safetensors", + "model.layers.90.mlp.experts.92.down_proj.weight": "model-00099-of-00101.safetensors", + "model.layers.90.mlp.experts.93.gate_proj.weight": "model-00099-of-00101.safetensors", + "model.layers.90.mlp.experts.93.up_proj.weight": "model-00099-of-00101.safetensors", + "model.layers.90.mlp.experts.93.down_proj.weight": "model-00099-of-00101.safetensors", + "model.layers.90.mlp.experts.94.gate_proj.weight": "model-00099-of-00101.safetensors", + "model.layers.90.mlp.experts.94.up_proj.weight": "model-00099-of-00101.safetensors", + "model.layers.90.mlp.experts.94.down_proj.weight": "model-00099-of-00101.safetensors", + "model.layers.90.mlp.experts.95.gate_proj.weight": "model-00099-of-00101.safetensors", + "model.layers.90.mlp.experts.95.up_proj.weight": "model-00099-of-00101.safetensors", + "model.layers.90.mlp.experts.95.down_proj.weight": "model-00099-of-00101.safetensors", + "model.layers.90.mlp.experts.96.gate_proj.weight": "model-00099-of-00101.safetensors", + "model.layers.90.mlp.experts.96.up_proj.weight": "model-00099-of-00101.safetensors", + "model.layers.90.mlp.experts.96.down_proj.weight": "model-00099-of-00101.safetensors", + "model.layers.90.mlp.experts.97.gate_proj.weight": "model-00099-of-00101.safetensors", + "model.layers.90.mlp.experts.97.up_proj.weight": "model-00099-of-00101.safetensors", + "model.layers.90.mlp.experts.97.down_proj.weight": "model-00099-of-00101.safetensors", + "model.layers.90.mlp.experts.98.gate_proj.weight": "model-00099-of-00101.safetensors", + "model.layers.90.mlp.experts.98.up_proj.weight": "model-00099-of-00101.safetensors", + "model.layers.90.mlp.experts.98.down_proj.weight": "model-00099-of-00101.safetensors", + "model.layers.90.mlp.experts.99.gate_proj.weight": "model-00099-of-00101.safetensors", + "model.layers.90.mlp.experts.99.up_proj.weight": "model-00099-of-00101.safetensors", + "model.layers.90.mlp.experts.99.down_proj.weight": "model-00099-of-00101.safetensors", + "model.layers.90.mlp.experts.100.gate_proj.weight": "model-00099-of-00101.safetensors", + "model.layers.90.mlp.experts.100.up_proj.weight": "model-00099-of-00101.safetensors", + "model.layers.90.mlp.experts.100.down_proj.weight": "model-00099-of-00101.safetensors", + "model.layers.90.mlp.experts.101.gate_proj.weight": "model-00099-of-00101.safetensors", + "model.layers.90.mlp.experts.101.up_proj.weight": "model-00099-of-00101.safetensors", + "model.layers.90.mlp.experts.101.down_proj.weight": "model-00099-of-00101.safetensors", + "model.layers.90.mlp.experts.102.gate_proj.weight": "model-00099-of-00101.safetensors", + "model.layers.90.mlp.experts.102.up_proj.weight": "model-00099-of-00101.safetensors", + "model.layers.90.mlp.experts.102.down_proj.weight": "model-00099-of-00101.safetensors", + "model.layers.90.mlp.experts.103.gate_proj.weight": "model-00099-of-00101.safetensors", + "model.layers.90.mlp.experts.103.up_proj.weight": "model-00099-of-00101.safetensors", + "model.layers.90.mlp.experts.103.down_proj.weight": "model-00099-of-00101.safetensors", + "model.layers.90.mlp.experts.104.gate_proj.weight": "model-00099-of-00101.safetensors", + "model.layers.90.mlp.experts.104.up_proj.weight": "model-00099-of-00101.safetensors", + "model.layers.90.mlp.experts.104.down_proj.weight": "model-00099-of-00101.safetensors", + "model.layers.90.mlp.experts.105.gate_proj.weight": "model-00099-of-00101.safetensors", + "model.layers.90.mlp.experts.105.up_proj.weight": "model-00099-of-00101.safetensors", + "model.layers.90.mlp.experts.105.down_proj.weight": "model-00099-of-00101.safetensors", + "model.layers.90.mlp.experts.106.gate_proj.weight": "model-00099-of-00101.safetensors", + "model.layers.90.mlp.experts.106.up_proj.weight": "model-00099-of-00101.safetensors", + "model.layers.90.mlp.experts.106.down_proj.weight": "model-00099-of-00101.safetensors", + "model.layers.90.mlp.experts.107.gate_proj.weight": "model-00099-of-00101.safetensors", + "model.layers.90.mlp.experts.107.up_proj.weight": "model-00099-of-00101.safetensors", + "model.layers.90.mlp.experts.107.down_proj.weight": "model-00099-of-00101.safetensors", + "model.layers.90.mlp.experts.108.gate_proj.weight": "model-00099-of-00101.safetensors", + "model.layers.90.mlp.experts.108.up_proj.weight": "model-00099-of-00101.safetensors", + "model.layers.90.mlp.experts.108.down_proj.weight": "model-00099-of-00101.safetensors", + "model.layers.90.mlp.experts.109.gate_proj.weight": "model-00099-of-00101.safetensors", + "model.layers.90.mlp.experts.109.up_proj.weight": "model-00099-of-00101.safetensors", + "model.layers.90.mlp.experts.109.down_proj.weight": "model-00099-of-00101.safetensors", + "model.layers.90.mlp.experts.110.gate_proj.weight": "model-00099-of-00101.safetensors", + "model.layers.90.mlp.experts.110.up_proj.weight": "model-00099-of-00101.safetensors", + "model.layers.90.mlp.experts.110.down_proj.weight": "model-00099-of-00101.safetensors", + "model.layers.90.mlp.experts.111.gate_proj.weight": "model-00099-of-00101.safetensors", + "model.layers.90.mlp.experts.111.up_proj.weight": "model-00099-of-00101.safetensors", + "model.layers.90.mlp.experts.111.down_proj.weight": "model-00099-of-00101.safetensors", + "model.layers.90.mlp.experts.112.gate_proj.weight": "model-00099-of-00101.safetensors", + "model.layers.90.mlp.experts.112.up_proj.weight": "model-00099-of-00101.safetensors", + "model.layers.90.mlp.experts.112.down_proj.weight": "model-00099-of-00101.safetensors", + "model.layers.90.mlp.experts.113.gate_proj.weight": "model-00099-of-00101.safetensors", + "model.layers.90.mlp.experts.113.up_proj.weight": "model-00099-of-00101.safetensors", + "model.layers.90.mlp.experts.113.down_proj.weight": "model-00099-of-00101.safetensors", + "model.layers.90.mlp.experts.114.gate_proj.weight": "model-00099-of-00101.safetensors", + "model.layers.90.mlp.experts.114.up_proj.weight": "model-00099-of-00101.safetensors", + "model.layers.90.mlp.experts.114.down_proj.weight": "model-00099-of-00101.safetensors", + "model.layers.90.mlp.experts.115.gate_proj.weight": "model-00099-of-00101.safetensors", + "model.layers.90.mlp.experts.115.up_proj.weight": "model-00099-of-00101.safetensors", + "model.layers.90.mlp.experts.115.down_proj.weight": "model-00099-of-00101.safetensors", + "model.layers.90.mlp.experts.116.gate_proj.weight": "model-00099-of-00101.safetensors", + "model.layers.90.mlp.experts.116.up_proj.weight": "model-00099-of-00101.safetensors", + "model.layers.90.mlp.experts.116.down_proj.weight": "model-00099-of-00101.safetensors", + "model.layers.90.mlp.experts.117.gate_proj.weight": "model-00099-of-00101.safetensors", + "model.layers.90.mlp.experts.117.up_proj.weight": "model-00099-of-00101.safetensors", + "model.layers.90.mlp.experts.117.down_proj.weight": "model-00099-of-00101.safetensors", + "model.layers.90.mlp.experts.118.gate_proj.weight": "model-00099-of-00101.safetensors", + "model.layers.90.mlp.experts.118.up_proj.weight": "model-00099-of-00101.safetensors", + "model.layers.90.mlp.experts.118.down_proj.weight": "model-00099-of-00101.safetensors", + "model.layers.90.mlp.experts.119.gate_proj.weight": "model-00099-of-00101.safetensors", + "model.layers.90.mlp.experts.119.up_proj.weight": "model-00099-of-00101.safetensors", + "model.layers.90.mlp.experts.119.down_proj.weight": "model-00099-of-00101.safetensors", + "model.layers.90.mlp.gate.weight": "model-00099-of-00101.safetensors", + "model.layers.90.mlp.gate.e_score_correction_bias": "model-00099-of-00101.safetensors", + "model.layers.90.mlp.shared_experts.gate_proj.weight": "model-00099-of-00101.safetensors", + "model.layers.90.mlp.shared_experts.up_proj.weight": "model-00099-of-00101.safetensors", + "model.layers.90.mlp.shared_experts.down_proj.weight": "model-00099-of-00101.safetensors", + "model.layers.90.input_layernorm.weight": "model-00099-of-00101.safetensors", + "model.layers.90.post_attention_layernorm.weight": "model-00099-of-00101.safetensors", + "model.layers.91.self_attn.q_proj.weight": "model-00100-of-00101.safetensors", + "model.layers.91.self_attn.q_proj.bias": "model-00100-of-00101.safetensors", + "model.layers.91.self_attn.k_proj.weight": "model-00100-of-00101.safetensors", + "model.layers.91.self_attn.k_proj.bias": "model-00100-of-00101.safetensors", + "model.layers.91.self_attn.v_proj.weight": "model-00100-of-00101.safetensors", + "model.layers.91.self_attn.v_proj.bias": "model-00100-of-00101.safetensors", + "model.layers.91.self_attn.o_proj.weight": "model-00100-of-00101.safetensors", + "model.layers.91.self_attn.q_norm.weight": "model-00100-of-00101.safetensors", + "model.layers.91.self_attn.k_norm.weight": "model-00100-of-00101.safetensors", + "model.layers.91.mlp.experts.0.gate_proj.weight": "model-00100-of-00101.safetensors", + "model.layers.91.mlp.experts.0.up_proj.weight": "model-00100-of-00101.safetensors", + "model.layers.91.mlp.experts.0.down_proj.weight": "model-00100-of-00101.safetensors", + "model.layers.91.mlp.experts.1.gate_proj.weight": "model-00100-of-00101.safetensors", + "model.layers.91.mlp.experts.1.up_proj.weight": "model-00100-of-00101.safetensors", + "model.layers.91.mlp.experts.1.down_proj.weight": "model-00100-of-00101.safetensors", + "model.layers.91.mlp.experts.2.gate_proj.weight": "model-00100-of-00101.safetensors", + "model.layers.91.mlp.experts.2.up_proj.weight": "model-00100-of-00101.safetensors", + "model.layers.91.mlp.experts.2.down_proj.weight": "model-00100-of-00101.safetensors", + "model.layers.91.mlp.experts.3.gate_proj.weight": "model-00100-of-00101.safetensors", + "model.layers.91.mlp.experts.3.up_proj.weight": "model-00100-of-00101.safetensors", + "model.layers.91.mlp.experts.3.down_proj.weight": "model-00100-of-00101.safetensors", + "model.layers.91.mlp.experts.4.gate_proj.weight": "model-00100-of-00101.safetensors", + "model.layers.91.mlp.experts.4.up_proj.weight": "model-00100-of-00101.safetensors", + "model.layers.91.mlp.experts.4.down_proj.weight": "model-00100-of-00101.safetensors", + "model.layers.91.mlp.experts.5.gate_proj.weight": "model-00100-of-00101.safetensors", + "model.layers.91.mlp.experts.5.up_proj.weight": "model-00100-of-00101.safetensors", + "model.layers.91.mlp.experts.5.down_proj.weight": "model-00100-of-00101.safetensors", + "model.layers.91.mlp.experts.6.gate_proj.weight": "model-00100-of-00101.safetensors", + "model.layers.91.mlp.experts.6.up_proj.weight": "model-00100-of-00101.safetensors", + "model.layers.91.mlp.experts.6.down_proj.weight": "model-00100-of-00101.safetensors", + "model.layers.91.mlp.experts.7.gate_proj.weight": "model-00100-of-00101.safetensors", + "model.layers.91.mlp.experts.7.up_proj.weight": "model-00100-of-00101.safetensors", + "model.layers.91.mlp.experts.7.down_proj.weight": "model-00100-of-00101.safetensors", + "model.layers.91.mlp.experts.8.gate_proj.weight": "model-00100-of-00101.safetensors", + "model.layers.91.mlp.experts.8.up_proj.weight": "model-00100-of-00101.safetensors", + "model.layers.91.mlp.experts.8.down_proj.weight": "model-00100-of-00101.safetensors", + "model.layers.91.mlp.experts.9.gate_proj.weight": "model-00100-of-00101.safetensors", + "model.layers.91.mlp.experts.9.up_proj.weight": "model-00100-of-00101.safetensors", + "model.layers.91.mlp.experts.9.down_proj.weight": "model-00100-of-00101.safetensors", + "model.layers.91.mlp.experts.10.gate_proj.weight": "model-00100-of-00101.safetensors", + "model.layers.91.mlp.experts.10.up_proj.weight": "model-00100-of-00101.safetensors", + "model.layers.91.mlp.experts.10.down_proj.weight": "model-00100-of-00101.safetensors", + "model.layers.91.mlp.experts.11.gate_proj.weight": "model-00100-of-00101.safetensors", + "model.layers.91.mlp.experts.11.up_proj.weight": "model-00100-of-00101.safetensors", + "model.layers.91.mlp.experts.11.down_proj.weight": "model-00100-of-00101.safetensors", + "model.layers.91.mlp.experts.12.gate_proj.weight": "model-00100-of-00101.safetensors", + "model.layers.91.mlp.experts.12.up_proj.weight": "model-00100-of-00101.safetensors", + "model.layers.91.mlp.experts.12.down_proj.weight": "model-00100-of-00101.safetensors", + "model.layers.91.mlp.experts.13.gate_proj.weight": "model-00100-of-00101.safetensors", + "model.layers.91.mlp.experts.13.up_proj.weight": "model-00100-of-00101.safetensors", + "model.layers.91.mlp.experts.13.down_proj.weight": "model-00100-of-00101.safetensors", + "model.layers.91.mlp.experts.14.gate_proj.weight": "model-00100-of-00101.safetensors", + "model.layers.91.mlp.experts.14.up_proj.weight": "model-00100-of-00101.safetensors", + "model.layers.91.mlp.experts.14.down_proj.weight": "model-00100-of-00101.safetensors", + "model.layers.91.mlp.experts.15.gate_proj.weight": "model-00100-of-00101.safetensors", + "model.layers.91.mlp.experts.15.up_proj.weight": "model-00100-of-00101.safetensors", + "model.layers.91.mlp.experts.15.down_proj.weight": "model-00100-of-00101.safetensors", + "model.layers.91.mlp.experts.16.gate_proj.weight": "model-00100-of-00101.safetensors", + "model.layers.91.mlp.experts.16.up_proj.weight": "model-00100-of-00101.safetensors", + "model.layers.91.mlp.experts.16.down_proj.weight": "model-00100-of-00101.safetensors", + "model.layers.91.mlp.experts.17.gate_proj.weight": "model-00100-of-00101.safetensors", + "model.layers.91.mlp.experts.17.up_proj.weight": "model-00100-of-00101.safetensors", + "model.layers.91.mlp.experts.17.down_proj.weight": "model-00100-of-00101.safetensors", + "model.layers.91.mlp.experts.18.gate_proj.weight": "model-00100-of-00101.safetensors", + "model.layers.91.mlp.experts.18.up_proj.weight": "model-00100-of-00101.safetensors", + "model.layers.91.mlp.experts.18.down_proj.weight": "model-00100-of-00101.safetensors", + "model.layers.91.mlp.experts.19.gate_proj.weight": "model-00100-of-00101.safetensors", + "model.layers.91.mlp.experts.19.up_proj.weight": "model-00100-of-00101.safetensors", + "model.layers.91.mlp.experts.19.down_proj.weight": "model-00100-of-00101.safetensors", + "model.layers.91.mlp.experts.20.gate_proj.weight": "model-00100-of-00101.safetensors", + "model.layers.91.mlp.experts.20.up_proj.weight": "model-00100-of-00101.safetensors", + "model.layers.91.mlp.experts.20.down_proj.weight": "model-00100-of-00101.safetensors", + "model.layers.91.mlp.experts.21.gate_proj.weight": "model-00100-of-00101.safetensors", + "model.layers.91.mlp.experts.21.up_proj.weight": "model-00100-of-00101.safetensors", + "model.layers.91.mlp.experts.21.down_proj.weight": "model-00100-of-00101.safetensors", + "model.layers.91.mlp.experts.22.gate_proj.weight": "model-00100-of-00101.safetensors", + "model.layers.91.mlp.experts.22.up_proj.weight": "model-00100-of-00101.safetensors", + "model.layers.91.mlp.experts.22.down_proj.weight": "model-00100-of-00101.safetensors", + "model.layers.91.mlp.experts.23.gate_proj.weight": "model-00100-of-00101.safetensors", + "model.layers.91.mlp.experts.23.up_proj.weight": "model-00100-of-00101.safetensors", + "model.layers.91.mlp.experts.23.down_proj.weight": "model-00100-of-00101.safetensors", + "model.layers.91.mlp.experts.24.gate_proj.weight": "model-00100-of-00101.safetensors", + "model.layers.91.mlp.experts.24.up_proj.weight": "model-00100-of-00101.safetensors", + "model.layers.91.mlp.experts.24.down_proj.weight": "model-00100-of-00101.safetensors", + "model.layers.91.mlp.experts.25.gate_proj.weight": "model-00100-of-00101.safetensors", + "model.layers.91.mlp.experts.25.up_proj.weight": "model-00100-of-00101.safetensors", + "model.layers.91.mlp.experts.25.down_proj.weight": "model-00100-of-00101.safetensors", + "model.layers.91.mlp.experts.26.gate_proj.weight": "model-00100-of-00101.safetensors", + "model.layers.91.mlp.experts.26.up_proj.weight": "model-00100-of-00101.safetensors", + "model.layers.91.mlp.experts.26.down_proj.weight": "model-00100-of-00101.safetensors", + "model.layers.91.mlp.experts.27.gate_proj.weight": "model-00100-of-00101.safetensors", + "model.layers.91.mlp.experts.27.up_proj.weight": "model-00100-of-00101.safetensors", + "model.layers.91.mlp.experts.27.down_proj.weight": "model-00100-of-00101.safetensors", + "model.layers.91.mlp.experts.28.gate_proj.weight": "model-00100-of-00101.safetensors", + "model.layers.91.mlp.experts.28.up_proj.weight": "model-00100-of-00101.safetensors", + "model.layers.91.mlp.experts.28.down_proj.weight": "model-00100-of-00101.safetensors", + "model.layers.91.mlp.experts.29.gate_proj.weight": "model-00100-of-00101.safetensors", + "model.layers.91.mlp.experts.29.up_proj.weight": "model-00100-of-00101.safetensors", + "model.layers.91.mlp.experts.29.down_proj.weight": "model-00100-of-00101.safetensors", + "model.layers.91.mlp.experts.30.gate_proj.weight": "model-00100-of-00101.safetensors", + "model.layers.91.mlp.experts.30.up_proj.weight": "model-00100-of-00101.safetensors", + "model.layers.91.mlp.experts.30.down_proj.weight": "model-00100-of-00101.safetensors", + "model.layers.91.mlp.experts.31.gate_proj.weight": "model-00100-of-00101.safetensors", + "model.layers.91.mlp.experts.31.up_proj.weight": "model-00100-of-00101.safetensors", + "model.layers.91.mlp.experts.31.down_proj.weight": "model-00100-of-00101.safetensors", + "model.layers.91.mlp.experts.32.gate_proj.weight": "model-00100-of-00101.safetensors", + "model.layers.91.mlp.experts.32.up_proj.weight": "model-00100-of-00101.safetensors", + "model.layers.91.mlp.experts.32.down_proj.weight": "model-00100-of-00101.safetensors", + "model.layers.91.mlp.experts.33.gate_proj.weight": "model-00100-of-00101.safetensors", + "model.layers.91.mlp.experts.33.up_proj.weight": "model-00100-of-00101.safetensors", + "model.layers.91.mlp.experts.33.down_proj.weight": "model-00100-of-00101.safetensors", + "model.layers.91.mlp.experts.34.gate_proj.weight": "model-00100-of-00101.safetensors", + "model.layers.91.mlp.experts.34.up_proj.weight": "model-00100-of-00101.safetensors", + "model.layers.91.mlp.experts.34.down_proj.weight": "model-00100-of-00101.safetensors", + "model.layers.91.mlp.experts.35.gate_proj.weight": "model-00100-of-00101.safetensors", + "model.layers.91.mlp.experts.35.up_proj.weight": "model-00100-of-00101.safetensors", + "model.layers.91.mlp.experts.35.down_proj.weight": "model-00100-of-00101.safetensors", + "model.layers.91.mlp.experts.36.gate_proj.weight": "model-00100-of-00101.safetensors", + "model.layers.91.mlp.experts.36.up_proj.weight": "model-00100-of-00101.safetensors", + "model.layers.91.mlp.experts.36.down_proj.weight": "model-00100-of-00101.safetensors", + "model.layers.91.mlp.experts.37.gate_proj.weight": "model-00100-of-00101.safetensors", + "model.layers.91.mlp.experts.37.up_proj.weight": "model-00100-of-00101.safetensors", + "model.layers.91.mlp.experts.37.down_proj.weight": "model-00100-of-00101.safetensors", + "model.layers.91.mlp.experts.38.gate_proj.weight": "model-00100-of-00101.safetensors", + "model.layers.91.mlp.experts.38.up_proj.weight": "model-00100-of-00101.safetensors", + "model.layers.91.mlp.experts.38.down_proj.weight": "model-00100-of-00101.safetensors", + "model.layers.91.mlp.experts.39.gate_proj.weight": "model-00100-of-00101.safetensors", + "model.layers.91.mlp.experts.39.up_proj.weight": "model-00100-of-00101.safetensors", + "model.layers.91.mlp.experts.39.down_proj.weight": "model-00100-of-00101.safetensors", + "model.layers.91.mlp.experts.40.gate_proj.weight": "model-00100-of-00101.safetensors", + "model.layers.91.mlp.experts.40.up_proj.weight": "model-00100-of-00101.safetensors", + "model.layers.91.mlp.experts.40.down_proj.weight": "model-00100-of-00101.safetensors", + "model.layers.91.mlp.experts.41.gate_proj.weight": "model-00100-of-00101.safetensors", + "model.layers.91.mlp.experts.41.up_proj.weight": "model-00100-of-00101.safetensors", + "model.layers.91.mlp.experts.41.down_proj.weight": "model-00100-of-00101.safetensors", + "model.layers.91.mlp.experts.42.gate_proj.weight": "model-00100-of-00101.safetensors", + "model.layers.91.mlp.experts.42.up_proj.weight": "model-00100-of-00101.safetensors", + "model.layers.91.mlp.experts.42.down_proj.weight": "model-00100-of-00101.safetensors", + "model.layers.91.mlp.experts.43.gate_proj.weight": "model-00100-of-00101.safetensors", + "model.layers.91.mlp.experts.43.up_proj.weight": "model-00100-of-00101.safetensors", + "model.layers.91.mlp.experts.43.down_proj.weight": "model-00100-of-00101.safetensors", + "model.layers.91.mlp.experts.44.gate_proj.weight": "model-00100-of-00101.safetensors", + "model.layers.91.mlp.experts.44.up_proj.weight": "model-00100-of-00101.safetensors", + "model.layers.91.mlp.experts.44.down_proj.weight": "model-00100-of-00101.safetensors", + "model.layers.91.mlp.experts.45.gate_proj.weight": "model-00100-of-00101.safetensors", + "model.layers.91.mlp.experts.45.up_proj.weight": "model-00100-of-00101.safetensors", + "model.layers.91.mlp.experts.45.down_proj.weight": "model-00100-of-00101.safetensors", + "model.layers.91.mlp.experts.46.gate_proj.weight": "model-00100-of-00101.safetensors", + "model.layers.91.mlp.experts.46.up_proj.weight": "model-00100-of-00101.safetensors", + "model.layers.91.mlp.experts.46.down_proj.weight": "model-00100-of-00101.safetensors", + "model.layers.91.mlp.experts.47.gate_proj.weight": "model-00100-of-00101.safetensors", + "model.layers.91.mlp.experts.47.up_proj.weight": "model-00100-of-00101.safetensors", + "model.layers.91.mlp.experts.47.down_proj.weight": "model-00100-of-00101.safetensors", + "model.layers.91.mlp.experts.48.gate_proj.weight": "model-00100-of-00101.safetensors", + "model.layers.91.mlp.experts.48.up_proj.weight": "model-00100-of-00101.safetensors", + "model.layers.91.mlp.experts.48.down_proj.weight": "model-00100-of-00101.safetensors", + "model.layers.91.mlp.experts.49.gate_proj.weight": "model-00100-of-00101.safetensors", + "model.layers.91.mlp.experts.49.up_proj.weight": "model-00100-of-00101.safetensors", + "model.layers.91.mlp.experts.49.down_proj.weight": "model-00100-of-00101.safetensors", + "model.layers.91.mlp.experts.50.gate_proj.weight": "model-00100-of-00101.safetensors", + "model.layers.91.mlp.experts.50.up_proj.weight": "model-00100-of-00101.safetensors", + "model.layers.91.mlp.experts.50.down_proj.weight": "model-00100-of-00101.safetensors", + "model.layers.91.mlp.experts.51.gate_proj.weight": "model-00100-of-00101.safetensors", + "model.layers.91.mlp.experts.51.up_proj.weight": "model-00100-of-00101.safetensors", + "model.layers.91.mlp.experts.51.down_proj.weight": "model-00100-of-00101.safetensors", + "model.layers.91.mlp.experts.52.gate_proj.weight": "model-00100-of-00101.safetensors", + "model.layers.91.mlp.experts.52.up_proj.weight": "model-00100-of-00101.safetensors", + "model.layers.91.mlp.experts.52.down_proj.weight": "model-00100-of-00101.safetensors", + "model.layers.91.mlp.experts.53.gate_proj.weight": "model-00100-of-00101.safetensors", + "model.layers.91.mlp.experts.53.up_proj.weight": "model-00100-of-00101.safetensors", + "model.layers.91.mlp.experts.53.down_proj.weight": "model-00100-of-00101.safetensors", + "model.layers.91.mlp.experts.54.gate_proj.weight": "model-00100-of-00101.safetensors", + "model.layers.91.mlp.experts.54.up_proj.weight": "model-00100-of-00101.safetensors", + "model.layers.91.mlp.experts.54.down_proj.weight": "model-00100-of-00101.safetensors", + "model.layers.91.mlp.experts.55.gate_proj.weight": "model-00100-of-00101.safetensors", + "model.layers.91.mlp.experts.55.up_proj.weight": "model-00100-of-00101.safetensors", + "model.layers.91.mlp.experts.55.down_proj.weight": "model-00100-of-00101.safetensors", + "model.layers.91.mlp.experts.56.gate_proj.weight": "model-00100-of-00101.safetensors", + "model.layers.91.mlp.experts.56.up_proj.weight": "model-00100-of-00101.safetensors", + "model.layers.91.mlp.experts.56.down_proj.weight": "model-00100-of-00101.safetensors", + "model.layers.91.mlp.experts.57.gate_proj.weight": "model-00100-of-00101.safetensors", + "model.layers.91.mlp.experts.57.up_proj.weight": "model-00100-of-00101.safetensors", + "model.layers.91.mlp.experts.57.down_proj.weight": "model-00100-of-00101.safetensors", + "model.layers.91.mlp.experts.58.gate_proj.weight": "model-00100-of-00101.safetensors", + "model.layers.91.mlp.experts.58.up_proj.weight": "model-00100-of-00101.safetensors", + "model.layers.91.mlp.experts.58.down_proj.weight": "model-00100-of-00101.safetensors", + "model.layers.91.mlp.experts.59.gate_proj.weight": "model-00100-of-00101.safetensors", + "model.layers.91.mlp.experts.59.up_proj.weight": "model-00100-of-00101.safetensors", + "model.layers.91.mlp.experts.59.down_proj.weight": "model-00100-of-00101.safetensors", + "model.layers.91.mlp.experts.60.gate_proj.weight": "model-00100-of-00101.safetensors", + "model.layers.91.mlp.experts.60.up_proj.weight": "model-00100-of-00101.safetensors", + "model.layers.91.mlp.experts.60.down_proj.weight": "model-00100-of-00101.safetensors", + "model.layers.91.mlp.experts.61.gate_proj.weight": "model-00100-of-00101.safetensors", + "model.layers.91.mlp.experts.61.up_proj.weight": "model-00100-of-00101.safetensors", + "model.layers.91.mlp.experts.61.down_proj.weight": "model-00100-of-00101.safetensors", + "model.layers.91.mlp.experts.62.gate_proj.weight": "model-00100-of-00101.safetensors", + "model.layers.91.mlp.experts.62.up_proj.weight": "model-00100-of-00101.safetensors", + "model.layers.91.mlp.experts.62.down_proj.weight": "model-00100-of-00101.safetensors", + "model.layers.91.mlp.experts.63.gate_proj.weight": "model-00100-of-00101.safetensors", + "model.layers.91.mlp.experts.63.up_proj.weight": "model-00100-of-00101.safetensors", + "model.layers.91.mlp.experts.63.down_proj.weight": "model-00100-of-00101.safetensors", + "model.layers.91.mlp.experts.64.gate_proj.weight": "model-00100-of-00101.safetensors", + "model.layers.91.mlp.experts.64.up_proj.weight": "model-00100-of-00101.safetensors", + "model.layers.91.mlp.experts.64.down_proj.weight": "model-00100-of-00101.safetensors", + "model.layers.91.mlp.experts.65.gate_proj.weight": "model-00100-of-00101.safetensors", + "model.layers.91.mlp.experts.65.up_proj.weight": "model-00100-of-00101.safetensors", + "model.layers.91.mlp.experts.65.down_proj.weight": "model-00100-of-00101.safetensors", + "model.layers.91.mlp.experts.66.gate_proj.weight": "model-00100-of-00101.safetensors", + "model.layers.91.mlp.experts.66.up_proj.weight": "model-00100-of-00101.safetensors", + "model.layers.91.mlp.experts.66.down_proj.weight": "model-00100-of-00101.safetensors", + "model.layers.91.mlp.experts.67.gate_proj.weight": "model-00100-of-00101.safetensors", + "model.layers.91.mlp.experts.67.up_proj.weight": "model-00100-of-00101.safetensors", + "model.layers.91.mlp.experts.67.down_proj.weight": "model-00100-of-00101.safetensors", + "model.layers.91.mlp.experts.68.gate_proj.weight": "model-00100-of-00101.safetensors", + "model.layers.91.mlp.experts.68.up_proj.weight": "model-00100-of-00101.safetensors", + "model.layers.91.mlp.experts.68.down_proj.weight": "model-00100-of-00101.safetensors", + "model.layers.91.mlp.experts.69.gate_proj.weight": "model-00100-of-00101.safetensors", + "model.layers.91.mlp.experts.69.up_proj.weight": "model-00100-of-00101.safetensors", + "model.layers.91.mlp.experts.69.down_proj.weight": "model-00100-of-00101.safetensors", + "model.layers.91.mlp.experts.70.gate_proj.weight": "model-00100-of-00101.safetensors", + "model.layers.91.mlp.experts.70.up_proj.weight": "model-00100-of-00101.safetensors", + "model.layers.91.mlp.experts.70.down_proj.weight": "model-00100-of-00101.safetensors", + "model.layers.91.mlp.experts.71.gate_proj.weight": "model-00100-of-00101.safetensors", + "model.layers.91.mlp.experts.71.up_proj.weight": "model-00100-of-00101.safetensors", + "model.layers.91.mlp.experts.71.down_proj.weight": "model-00100-of-00101.safetensors", + "model.layers.91.mlp.experts.72.gate_proj.weight": "model-00100-of-00101.safetensors", + "model.layers.91.mlp.experts.72.up_proj.weight": "model-00100-of-00101.safetensors", + "model.layers.91.mlp.experts.72.down_proj.weight": "model-00100-of-00101.safetensors", + "model.layers.91.mlp.experts.73.gate_proj.weight": "model-00100-of-00101.safetensors", + "model.layers.91.mlp.experts.73.up_proj.weight": "model-00100-of-00101.safetensors", + "model.layers.91.mlp.experts.73.down_proj.weight": "model-00100-of-00101.safetensors", + "model.layers.91.mlp.experts.74.gate_proj.weight": "model-00100-of-00101.safetensors", + "model.layers.91.mlp.experts.74.up_proj.weight": "model-00100-of-00101.safetensors", + "model.layers.91.mlp.experts.74.down_proj.weight": "model-00100-of-00101.safetensors", + "model.layers.91.mlp.experts.75.gate_proj.weight": "model-00100-of-00101.safetensors", + "model.layers.91.mlp.experts.75.up_proj.weight": "model-00100-of-00101.safetensors", + "model.layers.91.mlp.experts.75.down_proj.weight": "model-00100-of-00101.safetensors", + "model.layers.91.mlp.experts.76.gate_proj.weight": "model-00100-of-00101.safetensors", + "model.layers.91.mlp.experts.76.up_proj.weight": "model-00100-of-00101.safetensors", + "model.layers.91.mlp.experts.76.down_proj.weight": "model-00100-of-00101.safetensors", + "model.layers.91.mlp.experts.77.gate_proj.weight": "model-00100-of-00101.safetensors", + "model.layers.91.mlp.experts.77.up_proj.weight": "model-00100-of-00101.safetensors", + "model.layers.91.mlp.experts.77.down_proj.weight": "model-00100-of-00101.safetensors", + "model.layers.91.mlp.experts.78.gate_proj.weight": "model-00100-of-00101.safetensors", + "model.layers.91.mlp.experts.78.up_proj.weight": "model-00100-of-00101.safetensors", + "model.layers.91.mlp.experts.78.down_proj.weight": "model-00100-of-00101.safetensors", + "model.layers.91.mlp.experts.79.gate_proj.weight": "model-00100-of-00101.safetensors", + "model.layers.91.mlp.experts.79.up_proj.weight": "model-00100-of-00101.safetensors", + "model.layers.91.mlp.experts.79.down_proj.weight": "model-00100-of-00101.safetensors", + "model.layers.91.mlp.experts.80.gate_proj.weight": "model-00100-of-00101.safetensors", + "model.layers.91.mlp.experts.80.up_proj.weight": "model-00100-of-00101.safetensors", + "model.layers.91.mlp.experts.80.down_proj.weight": "model-00100-of-00101.safetensors", + "model.layers.91.mlp.experts.81.gate_proj.weight": "model-00100-of-00101.safetensors", + "model.layers.91.mlp.experts.81.up_proj.weight": "model-00100-of-00101.safetensors", + "model.layers.91.mlp.experts.81.down_proj.weight": "model-00100-of-00101.safetensors", + "model.layers.91.mlp.experts.82.gate_proj.weight": "model-00100-of-00101.safetensors", + "model.layers.91.mlp.experts.82.up_proj.weight": "model-00100-of-00101.safetensors", + "model.layers.91.mlp.experts.82.down_proj.weight": "model-00100-of-00101.safetensors", + "model.layers.91.mlp.experts.83.gate_proj.weight": "model-00100-of-00101.safetensors", + "model.layers.91.mlp.experts.83.up_proj.weight": "model-00100-of-00101.safetensors", + "model.layers.91.mlp.experts.83.down_proj.weight": "model-00100-of-00101.safetensors", + "model.layers.91.mlp.experts.84.gate_proj.weight": "model-00100-of-00101.safetensors", + "model.layers.91.mlp.experts.84.up_proj.weight": "model-00100-of-00101.safetensors", + "model.layers.91.mlp.experts.84.down_proj.weight": "model-00100-of-00101.safetensors", + "model.layers.91.mlp.experts.85.gate_proj.weight": "model-00100-of-00101.safetensors", + "model.layers.91.mlp.experts.85.up_proj.weight": "model-00100-of-00101.safetensors", + "model.layers.91.mlp.experts.85.down_proj.weight": "model-00100-of-00101.safetensors", + "model.layers.91.mlp.experts.86.gate_proj.weight": "model-00100-of-00101.safetensors", + "model.layers.91.mlp.experts.86.up_proj.weight": "model-00100-of-00101.safetensors", + "model.layers.91.mlp.experts.86.down_proj.weight": "model-00100-of-00101.safetensors", + "model.layers.91.mlp.experts.87.gate_proj.weight": "model-00100-of-00101.safetensors", + "model.layers.91.mlp.experts.87.up_proj.weight": "model-00100-of-00101.safetensors", + "model.layers.91.mlp.experts.87.down_proj.weight": "model-00100-of-00101.safetensors", + "model.layers.91.mlp.experts.88.gate_proj.weight": "model-00100-of-00101.safetensors", + "model.layers.91.mlp.experts.88.up_proj.weight": "model-00100-of-00101.safetensors", + "model.layers.91.mlp.experts.88.down_proj.weight": "model-00100-of-00101.safetensors", + "model.layers.91.mlp.experts.89.gate_proj.weight": "model-00100-of-00101.safetensors", + "model.layers.91.mlp.experts.89.up_proj.weight": "model-00100-of-00101.safetensors", + "model.layers.91.mlp.experts.89.down_proj.weight": "model-00100-of-00101.safetensors", + "model.layers.91.mlp.experts.90.gate_proj.weight": "model-00100-of-00101.safetensors", + "model.layers.91.mlp.experts.90.up_proj.weight": "model-00100-of-00101.safetensors", + "model.layers.91.mlp.experts.90.down_proj.weight": "model-00100-of-00101.safetensors", + "model.layers.91.mlp.experts.91.gate_proj.weight": "model-00100-of-00101.safetensors", + "model.layers.91.mlp.experts.91.up_proj.weight": "model-00100-of-00101.safetensors", + "model.layers.91.mlp.experts.91.down_proj.weight": "model-00100-of-00101.safetensors", + "model.layers.91.mlp.experts.92.gate_proj.weight": "model-00100-of-00101.safetensors", + "model.layers.91.mlp.experts.92.up_proj.weight": "model-00100-of-00101.safetensors", + "model.layers.91.mlp.experts.92.down_proj.weight": "model-00100-of-00101.safetensors", + "model.layers.91.mlp.experts.93.gate_proj.weight": "model-00100-of-00101.safetensors", + "model.layers.91.mlp.experts.93.up_proj.weight": "model-00100-of-00101.safetensors", + "model.layers.91.mlp.experts.93.down_proj.weight": "model-00100-of-00101.safetensors", + "model.layers.91.mlp.experts.94.gate_proj.weight": "model-00100-of-00101.safetensors", + "model.layers.91.mlp.experts.94.up_proj.weight": "model-00100-of-00101.safetensors", + "model.layers.91.mlp.experts.94.down_proj.weight": "model-00100-of-00101.safetensors", + "model.layers.91.mlp.experts.95.gate_proj.weight": "model-00100-of-00101.safetensors", + "model.layers.91.mlp.experts.95.up_proj.weight": "model-00100-of-00101.safetensors", + "model.layers.91.mlp.experts.95.down_proj.weight": "model-00100-of-00101.safetensors", + "model.layers.91.mlp.experts.96.gate_proj.weight": "model-00100-of-00101.safetensors", + "model.layers.91.mlp.experts.96.up_proj.weight": "model-00100-of-00101.safetensors", + "model.layers.91.mlp.experts.96.down_proj.weight": "model-00100-of-00101.safetensors", + "model.layers.91.mlp.experts.97.gate_proj.weight": "model-00100-of-00101.safetensors", + "model.layers.91.mlp.experts.97.up_proj.weight": "model-00100-of-00101.safetensors", + "model.layers.91.mlp.experts.97.down_proj.weight": "model-00100-of-00101.safetensors", + "model.layers.91.mlp.experts.98.gate_proj.weight": "model-00100-of-00101.safetensors", + "model.layers.91.mlp.experts.98.up_proj.weight": "model-00100-of-00101.safetensors", + "model.layers.91.mlp.experts.98.down_proj.weight": "model-00100-of-00101.safetensors", + "model.layers.91.mlp.experts.99.gate_proj.weight": "model-00100-of-00101.safetensors", + "model.layers.91.mlp.experts.99.up_proj.weight": "model-00100-of-00101.safetensors", + "model.layers.91.mlp.experts.99.down_proj.weight": "model-00100-of-00101.safetensors", + "model.layers.91.mlp.experts.100.gate_proj.weight": "model-00100-of-00101.safetensors", + "model.layers.91.mlp.experts.100.up_proj.weight": "model-00100-of-00101.safetensors", + "model.layers.91.mlp.experts.100.down_proj.weight": "model-00100-of-00101.safetensors", + "model.layers.91.mlp.experts.101.gate_proj.weight": "model-00100-of-00101.safetensors", + "model.layers.91.mlp.experts.101.up_proj.weight": "model-00100-of-00101.safetensors", + "model.layers.91.mlp.experts.101.down_proj.weight": "model-00100-of-00101.safetensors", + "model.layers.91.mlp.experts.102.gate_proj.weight": "model-00100-of-00101.safetensors", + "model.layers.91.mlp.experts.102.up_proj.weight": "model-00100-of-00101.safetensors", + "model.layers.91.mlp.experts.102.down_proj.weight": "model-00100-of-00101.safetensors", + "model.layers.91.mlp.experts.103.gate_proj.weight": "model-00100-of-00101.safetensors", + "model.layers.91.mlp.experts.103.up_proj.weight": "model-00100-of-00101.safetensors", + "model.layers.91.mlp.experts.103.down_proj.weight": "model-00100-of-00101.safetensors", + "model.layers.91.mlp.experts.104.gate_proj.weight": "model-00100-of-00101.safetensors", + "model.layers.91.mlp.experts.104.up_proj.weight": "model-00100-of-00101.safetensors", + "model.layers.91.mlp.experts.104.down_proj.weight": "model-00100-of-00101.safetensors", + "model.layers.91.mlp.experts.105.gate_proj.weight": "model-00100-of-00101.safetensors", + "model.layers.91.mlp.experts.105.up_proj.weight": "model-00100-of-00101.safetensors", + "model.layers.91.mlp.experts.105.down_proj.weight": "model-00100-of-00101.safetensors", + "model.layers.91.mlp.experts.106.gate_proj.weight": "model-00100-of-00101.safetensors", + "model.layers.91.mlp.experts.106.up_proj.weight": "model-00100-of-00101.safetensors", + "model.layers.91.mlp.experts.106.down_proj.weight": "model-00100-of-00101.safetensors", + "model.layers.91.mlp.experts.107.gate_proj.weight": "model-00100-of-00101.safetensors", + "model.layers.91.mlp.experts.107.up_proj.weight": "model-00100-of-00101.safetensors", + "model.layers.91.mlp.experts.107.down_proj.weight": "model-00101-of-00101.safetensors", + "model.layers.91.mlp.experts.108.gate_proj.weight": "model-00101-of-00101.safetensors", + "model.layers.91.mlp.experts.108.up_proj.weight": "model-00101-of-00101.safetensors", + "model.layers.91.mlp.experts.108.down_proj.weight": "model-00101-of-00101.safetensors", + "model.layers.91.mlp.experts.109.gate_proj.weight": "model-00101-of-00101.safetensors", + "model.layers.91.mlp.experts.109.up_proj.weight": "model-00101-of-00101.safetensors", + "model.layers.91.mlp.experts.109.down_proj.weight": "model-00101-of-00101.safetensors", + "model.layers.91.mlp.experts.110.gate_proj.weight": "model-00101-of-00101.safetensors", + "model.layers.91.mlp.experts.110.up_proj.weight": "model-00101-of-00101.safetensors", + "model.layers.91.mlp.experts.110.down_proj.weight": "model-00101-of-00101.safetensors", + "model.layers.91.mlp.experts.111.gate_proj.weight": "model-00101-of-00101.safetensors", + "model.layers.91.mlp.experts.111.up_proj.weight": "model-00101-of-00101.safetensors", + "model.layers.91.mlp.experts.111.down_proj.weight": "model-00101-of-00101.safetensors", + "model.layers.91.mlp.experts.112.gate_proj.weight": "model-00101-of-00101.safetensors", + "model.layers.91.mlp.experts.112.up_proj.weight": "model-00101-of-00101.safetensors", + "model.layers.91.mlp.experts.112.down_proj.weight": "model-00101-of-00101.safetensors", + "model.layers.91.mlp.experts.113.gate_proj.weight": "model-00101-of-00101.safetensors", + "model.layers.91.mlp.experts.113.up_proj.weight": "model-00101-of-00101.safetensors", + "model.layers.91.mlp.experts.113.down_proj.weight": "model-00101-of-00101.safetensors", + "model.layers.91.mlp.experts.114.gate_proj.weight": "model-00101-of-00101.safetensors", + "model.layers.91.mlp.experts.114.up_proj.weight": "model-00101-of-00101.safetensors", + "model.layers.91.mlp.experts.114.down_proj.weight": "model-00101-of-00101.safetensors", + "model.layers.91.mlp.experts.115.gate_proj.weight": "model-00101-of-00101.safetensors", + "model.layers.91.mlp.experts.115.up_proj.weight": "model-00101-of-00101.safetensors", + "model.layers.91.mlp.experts.115.down_proj.weight": "model-00101-of-00101.safetensors", + "model.layers.91.mlp.experts.116.gate_proj.weight": "model-00101-of-00101.safetensors", + "model.layers.91.mlp.experts.116.up_proj.weight": "model-00101-of-00101.safetensors", + "model.layers.91.mlp.experts.116.down_proj.weight": "model-00101-of-00101.safetensors", + "model.layers.91.mlp.experts.117.gate_proj.weight": "model-00101-of-00101.safetensors", + "model.layers.91.mlp.experts.117.up_proj.weight": "model-00101-of-00101.safetensors", + "model.layers.91.mlp.experts.117.down_proj.weight": "model-00101-of-00101.safetensors", + "model.layers.91.mlp.experts.118.gate_proj.weight": "model-00101-of-00101.safetensors", + "model.layers.91.mlp.experts.118.up_proj.weight": "model-00101-of-00101.safetensors", + "model.layers.91.mlp.experts.118.down_proj.weight": "model-00101-of-00101.safetensors", + "model.layers.91.mlp.experts.119.gate_proj.weight": "model-00101-of-00101.safetensors", + "model.layers.91.mlp.experts.119.up_proj.weight": "model-00101-of-00101.safetensors", + "model.layers.91.mlp.experts.119.down_proj.weight": "model-00101-of-00101.safetensors", + "model.layers.91.mlp.gate.weight": "model-00101-of-00101.safetensors", + "model.layers.91.mlp.gate.e_score_correction_bias": "model-00101-of-00101.safetensors", + "model.layers.91.mlp.shared_experts.gate_proj.weight": "model-00101-of-00101.safetensors", + "model.layers.91.mlp.shared_experts.up_proj.weight": "model-00101-of-00101.safetensors", + "model.layers.91.mlp.shared_experts.down_proj.weight": "model-00101-of-00101.safetensors", + "model.layers.91.input_layernorm.weight": "model-00101-of-00101.safetensors", + "model.layers.91.post_attention_layernorm.weight": "model-00101-of-00101.safetensors", + "model.norm.weight": "model-00101-of-00101.safetensors", + "lm_head.weight": "model-00101-of-00101.safetensors" + } +} \ No newline at end of file diff --git a/special_tokens_map.json b/special_tokens_map.json new file mode 100644 index 0000000000000000000000000000000000000000..9028cf84013844f17d7616bdec1d88e977924434 --- /dev/null +++ b/special_tokens_map.json @@ -0,0 +1,40 @@ +{ + "additional_special_tokens": [ + "<|endoftext|>", + "[MASK]", + "[gMASK]", + "[sMASK]", + "", + "", + "<|system|>", + "<|user|>", + "<|assistant|>", + "<|observation|>", + "<|begin_of_image|>", + "<|end_of_image|>", + "<|begin_of_video|>", + "<|end_of_video|>", + "<|begin_of_audio|>", + "<|end_of_audio|>", + "<|begin_of_transcription|>", + "<|end_of_transcription|>", + "<|code_prefix|>", + "<|code_middle|>", + "<|code_suffix|>", + "/nothink" + ], + "eos_token": { + "content": "<|endoftext|>", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false + }, + "pad_token": { + "content": "<|endoftext|>", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false + } +} diff --git a/tokenizer.json b/tokenizer.json new file mode 100644 index 0000000000000000000000000000000000000000..e3ed3c66baf1ec4de61840b0abf02142687bfed8 --- /dev/null +++ b/tokenizer.json @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:bda8e2146c3bb7b7e0fc96dcc4f0aeff041c6c27952e3ace0665663ebff346ba +size 19970700 diff --git a/tokenizer_config.json b/tokenizer_config.json new file mode 100644 index 0000000000000000000000000000000000000000..75e11cfb2e0cc09f19391ec2278b4825a4c3fae9 --- /dev/null +++ b/tokenizer_config.json @@ -0,0 +1,325 @@ +{ + "added_tokens_decoder": { + "151329": { + "content": "<|endoftext|>", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "151330": { + "content": "[MASK]", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "151331": { + "content": "[gMASK]", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "151332": { + "content": "[sMASK]", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "151333": { + "content": "", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "151334": { + "content": "", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "151335": { + "content": "<|system|>", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "151336": { + "content": "<|user|>", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "151337": { + "content": "<|assistant|>", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "151338": { + "content": "<|observation|>", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "151339": { + "content": "<|begin_of_image|>", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "151340": { + "content": "<|end_of_image|>", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "151341": { + "content": "<|begin_of_video|>", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "151342": { + "content": "<|end_of_video|>", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "151343": { + "content": "<|begin_of_audio|>", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "151344": { + "content": "<|end_of_audio|>", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "151345": { + "content": "<|begin_of_transcription|>", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "151346": { + "content": "<|end_of_transcription|>", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "151347": { + "content": "<|code_prefix|>", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "151348": { + "content": "<|code_middle|>", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "151349": { + "content": "<|code_suffix|>", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "151350": { + "content": "", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": false + }, + "151351": { + "content": "", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": false + }, + "151352": { + "content": "", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": false + }, + "151353": { + "content": "", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": false + }, + "151354": { + "content": "", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": false + }, + "151355": { + "content": "", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": false + }, + "151356": { + "content": "", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": false + }, + "151357": { + "content": "", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": false + }, + "151358": { + "content": "", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": false + }, + "151359": { + "content": "", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": false + }, + "151360": { + "content": "/nothink", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "151361": { + "content": "<|begin_of_box|>", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": false + }, + "151362": { + "content": "<|end_of_box|>", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": false + }, + "151363": { + "content": "<|image|>", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": false + }, + "151364": { + "content": "<|video|>", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": false + } + }, + "additional_special_tokens": [ + "<|endoftext|>", + "[MASK]", + "[gMASK]", + "[sMASK]", + "", + "", + "<|system|>", + "<|user|>", + "<|assistant|>", + "<|observation|>", + "<|begin_of_image|>", + "<|end_of_image|>", + "<|begin_of_video|>", + "<|end_of_video|>", + "<|begin_of_audio|>", + "<|end_of_audio|>", + "<|begin_of_transcription|>", + "<|end_of_transcription|>", + "<|code_prefix|>", + "<|code_middle|>", + "<|code_suffix|>", + "/nothink" + ], + "clean_up_tokenization_spaces": false, + "do_lower_case": false, + "eos_token": "<|endoftext|>", + "extra_special_tokens": {}, + "model_max_length": 128000, + "pad_token": "<|endoftext|>", + "padding_side": "left", + "remove_space": false, + "tokenizer_class": "PreTrainedTokenizerFast" +}