Hugging Face's logo

amd
/
Qwen3.5-397B-A17B-MXFP4

Image-Text-to-Text
Transformers
Safetensors
English
qwen3_5_moe
conversational
8-bit precision
quark
Model card Files
xet
 Community
6

update model

#2
by linzhao-amd - opened
base: refs/heads/main
←
from: refs/pr/2
Discussion Files changed
+115
-1675
This view is limited to 50 files because it contains too many changes. See the raw diff here.
Files changed (50) hide show
  1. README.md +14 -37
  2. config.json +1 -1538
  3. model.safetensors-00001-of-00094.safetensors +1 -1
  4. model.safetensors-00002-of-00094.safetensors +1 -1
  5. model.safetensors-00003-of-00094.safetensors +1 -1
  6. model.safetensors-00004-of-00094.safetensors +1 -1
  7. model.safetensors-00005-of-00094.safetensors +1 -1
  8. model.safetensors-00006-of-00094.safetensors +1 -1
  9. model.safetensors-00007-of-00094.safetensors +1 -1
  10. model.safetensors-00008-of-00094.safetensors +1 -1
  11. model.safetensors-00009-of-00094.safetensors +1 -1
  12. model.safetensors-00010-of-00094.safetensors +1 -1
  13. model.safetensors-00011-of-00094.safetensors +1 -1
  14. model.safetensors-00012-of-00094.safetensors +1 -1
  15. model.safetensors-00013-of-00094.safetensors +1 -1
  16. model.safetensors-00014-of-00094.safetensors +1 -1
  17. model.safetensors-00015-of-00094.safetensors +1 -1
  18. model.safetensors-00016-of-00094.safetensors +1 -1
  19. model.safetensors-00017-of-00094.safetensors +1 -1
  20. model.safetensors-00018-of-00094.safetensors +1 -1
  21. model.safetensors-00019-of-00094.safetensors +1 -1
  22. model.safetensors-00020-of-00094.safetensors +1 -1
  23. model.safetensors-00021-of-00094.safetensors +1 -1
  24. model.safetensors-00022-of-00094.safetensors +1 -1
  25. model.safetensors-00023-of-00094.safetensors +1 -1
  26. model.safetensors-00024-of-00094.safetensors +1 -1
  27. model.safetensors-00025-of-00094.safetensors +1 -1
  28. model.safetensors-00026-of-00094.safetensors +1 -1
  29. model.safetensors-00027-of-00094.safetensors +1 -1
  30. model.safetensors-00028-of-00094.safetensors +1 -1
  31. model.safetensors-00029-of-00094.safetensors +1 -1
  32. model.safetensors-00030-of-00094.safetensors +1 -1
  33. model.safetensors-00031-of-00094.safetensors +1 -1
  34. model.safetensors-00032-of-00094.safetensors +1 -1
  35. model.safetensors-00033-of-00094.safetensors +1 -1
  36. model.safetensors-00034-of-00094.safetensors +1 -1
  37. model.safetensors-00035-of-00094.safetensors +1 -1
  38. model.safetensors-00036-of-00094.safetensors +1 -1
  39. model.safetensors-00037-of-00094.safetensors +1 -1
  40. model.safetensors-00038-of-00094.safetensors +1 -1
  41. model.safetensors-00039-of-00094.safetensors +1 -1
  42. model.safetensors-00040-of-00094.safetensors +1 -1
  43. model.safetensors-00041-of-00094.safetensors +1 -1
  44. model.safetensors-00042-of-00094.safetensors +1 -1
  45. model.safetensors-00043-of-00094.safetensors +1 -1
  46. model.safetensors-00044-of-00094.safetensors +1 -1
  47. model.safetensors-00045-of-00094.safetensors +1 -1
  48. model.safetensors-00046-of-00094.safetensors +1 -1
  49. model.safetensors-00047-of-00094.safetensors +1 -1
  50. model.safetensors-00048-of-00094.safetensors +1 -1
README.md CHANGED
@@ -1,6 +1,6 @@
1
  ---
2
  base_model:
3
- - Qwen/Qwen3.5-397B-A17B-FP8
4
  language:
5
  - en
6
  library_name: transformers
@@ -14,13 +14,12 @@ license_link: https://huggingface.co/Qwen/Qwen3.5-397B-A17B/blob/main/LICENSE
14
  - **Input:** Text
15
  - **Output:** Text
16
  - **Supported Hardware Microarchitecture:** AMD MI300 MI350/MI355
17
- - **ROCm**: 7.0.0
18
- - **PyTorch**: 2.9.1
19
- - **Transformers**: 5.3.0
20
  - **Operating System(s):** Linux
21
  - **Inference Engine:** [SGLang](https://docs.sglang.ai/)/[vLLM](https://docs.vllm.ai/en/latest/)
22
- - **Model Optimizer:** [AMD-Quark](https://quark.docs.amd.com/latest/index.html) (v0.11.1)
23
- - **Quantized layers:** Experts in language model only
24
  - **Weight quantization:** OCP MXFP4, Static
25
  - **Activation quantization:** OCP MXFP4, Dynamic
26
 
@@ -32,35 +31,13 @@ The model was quantized from [Qwen/Qwen3.5-397B-A17B-FP8](https://huggingface.co
32
 
33
  **Quantization scripts:**
34
  ```
35
- import os
36
- from quark.torch import LLMTemplate, ModelQuantizer
37
- from quark.common.profiler import GlobalProfiler
38
-
39
- # Register qwen3_5_moe template
40
- qwen3_5_moe_template = LLMTemplate(
41
- model_type="qwen3_5_moe",
42
- kv_layers_name=["*k_proj", "*v_proj"],
43
- q_layer_name="*q_proj"
44
- )
45
- LLMTemplate.register_template(qwen3_5_moe_template)
46
-
47
- # Configuration
48
- ckpt_path = "Qwen/Qwen3.5-397B-A17B-FP8"
49
- output_dir = "amd/Qwen3.5-397B-A17B-MXFP4"
50
- quant_scheme = "mxfp4"
51
- exclude_layers = ["lm_head", "model.visual.*", "mtp.*", "*mlp.gate", "*shared_expert_gate*", "*.linear_attn.*", "*.self_attn.*", "*.shared_expert.*"]
52
-
53
- # Get quant config from template
54
- template = LLMTemplate.get("qwen3_5_moe")
55
- quant_config = template.get_config(scheme=quant_scheme, exclude_layers=exclude_layers)
56
-
57
- # Quantize with File-to-file mode
58
- profiler = GlobalProfiler(output_path=os.path.join(output_dir, "quark_profile.yaml"))
59
- quantizer = ModelQuantizer(quant_config)
60
- quantizer.direct_quantize_checkpoint(
61
- pretrained_model_path=ckpt_path,
62
- save_path=output_dir,
63
- )
64
  ```
65
  For further details or issues, please refer to the AMD-Quark documentation or contact the respective developers.
66
 
@@ -95,13 +72,13 @@ The model was evaluated on gsm8k benchmarks using the [vllm](https://github.com/
95
 
96
  ### Reproduction
97
 
98
- The GSM8K results were obtained using the vLLM framework, based on the Docker image `rocm/vllm-dev:nightly_main_20260211`, and vLLM is installed inside the container.
99
 
100
  #### Evaluating model in a new terminal
101
  ```
102
  lm_eval \
103
  --model vllm \
104
- --model_args pretrained=$MODEL,tensor_parallel_size=4,max_model_len=262144,gpu_memory_utilization=0.90,max_gen_toks=2048,trust_remote_code=True,reasoning_parser=qwen3 \
105
  --tasks gsm8k --num_fewshot 5 \
106
  --batch_size auto
107
  ```
 
1
  ---
2
  base_model:
3
+ - Qwen/Qwen3.5-397B-A17B
4
  language:
5
  - en
6
  library_name: transformers
 
14
  - **Input:** Text
15
  - **Output:** Text
16
  - **Supported Hardware Microarchitecture:** AMD MI300 MI350/MI355
17
+ - **ROCm**: 7.0
18
+ - **PyTorch**: 2.8.0
19
+ - **Transformers**: 5.2.0
20
  - **Operating System(s):** Linux
21
  - **Inference Engine:** [SGLang](https://docs.sglang.ai/)/[vLLM](https://docs.vllm.ai/en/latest/)
22
+ - **Model Optimizer:** [AMD-Quark](https://quark.docs.amd.com/latest/index.html) (v0.11)
 
23
  - **Weight quantization:** OCP MXFP4, Static
24
  - **Activation quantization:** OCP MXFP4, Dynamic
25
 
 
31
 
32
  **Quantization scripts:**
33
  ```
34
+ cd Quark/examples/torch/language_modeling/llm_ptq/
35
+ export exclude_layers="lm_head model.visual.* mtp.* *mlp.gate *shared_expert_gate* *.linear_attn.* *.self_attn.* *.shared_expert.*"
36
+ python3 quantize_quark.py --model_dir $MODEL_DIR \
37
+ --quant_scheme mxfp4 \
38
+ --file2file_quantization \
39
+ --exclude_layers $exclude_layers \
40
+ --output_dir $output_dir \
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
41
  ```
42
  For further details or issues, please refer to the AMD-Quark documentation or contact the respective developers.
43
 
 
72
 
73
  ### Reproduction
74
 
75
+ The GSM8K results were obtained using the vLLM framework, based on the Docker image `rocm/vllm-dev:nightly_main_20260211`, and vLLM is installed inside the container with fixes applied for model support.
76
 
77
  #### Evaluating model in a new terminal
78
  ```
79
  lm_eval \
80
  --model vllm \
81
+ --model_args pretrained=$MODEL,tensor_parallel_size=1,max_model_len=262144,gpu_memory_utilization=0.90,max_gen_toks=2048,trust_remote_code=True,reasoning_parser=qwen3 \
82
  --tasks gsm8k --num_fewshot 5 \
83
  --batch_size auto
84
  ```
config.json CHANGED
@@ -964,1543 +964,6 @@
964
  "model.visual.blocks.9.norm2",
965
  "model.visual.merger.linear_fc1",
966
  "model.visual.merger.linear_fc2",
967
- "mtp.fc",
968
- "mtp.layers.0.mlp.experts.0.down_proj",
969
- "mtp.layers.0.mlp.experts.0.gate_proj",
970
- "mtp.layers.0.mlp.experts.0.up_proj",
971
- "mtp.layers.0.mlp.experts.1.down_proj",
972
- "mtp.layers.0.mlp.experts.1.gate_proj",
973
- "mtp.layers.0.mlp.experts.1.up_proj",
974
- "mtp.layers.0.mlp.experts.10.down_proj",
975
- "mtp.layers.0.mlp.experts.10.gate_proj",
976
- "mtp.layers.0.mlp.experts.10.up_proj",
977
- "mtp.layers.0.mlp.experts.100.down_proj",
978
- "mtp.layers.0.mlp.experts.100.gate_proj",
979
- "mtp.layers.0.mlp.experts.100.up_proj",
980
- "mtp.layers.0.mlp.experts.101.down_proj",
981
- "mtp.layers.0.mlp.experts.101.gate_proj",
982
- "mtp.layers.0.mlp.experts.101.up_proj",
983
- "mtp.layers.0.mlp.experts.102.down_proj",
984
- "mtp.layers.0.mlp.experts.102.gate_proj",
985
- "mtp.layers.0.mlp.experts.102.up_proj",
986
- "mtp.layers.0.mlp.experts.103.down_proj",
987
- "mtp.layers.0.mlp.experts.103.gate_proj",
988
- "mtp.layers.0.mlp.experts.103.up_proj",
989
- "mtp.layers.0.mlp.experts.104.down_proj",
990
- "mtp.layers.0.mlp.experts.104.gate_proj",
991
- "mtp.layers.0.mlp.experts.104.up_proj",
992
- "mtp.layers.0.mlp.experts.105.down_proj",
993
- "mtp.layers.0.mlp.experts.105.gate_proj",
994
- "mtp.layers.0.mlp.experts.105.up_proj",
995
- "mtp.layers.0.mlp.experts.106.down_proj",
996
- "mtp.layers.0.mlp.experts.106.gate_proj",
997
- "mtp.layers.0.mlp.experts.106.up_proj",
998
- "mtp.layers.0.mlp.experts.107.down_proj",
999
- "mtp.layers.0.mlp.experts.107.gate_proj",
1000
- "mtp.layers.0.mlp.experts.107.up_proj",
1001
- "mtp.layers.0.mlp.experts.108.down_proj",
1002
- "mtp.layers.0.mlp.experts.108.gate_proj",
1003
- "mtp.layers.0.mlp.experts.108.up_proj",
1004
- "mtp.layers.0.mlp.experts.109.down_proj",
1005
- "mtp.layers.0.mlp.experts.109.gate_proj",
1006
- "mtp.layers.0.mlp.experts.109.up_proj",
1007
- "mtp.layers.0.mlp.experts.11.down_proj",
1008
- "mtp.layers.0.mlp.experts.11.gate_proj",
1009
- "mtp.layers.0.mlp.experts.11.up_proj",
1010
- "mtp.layers.0.mlp.experts.110.down_proj",
1011
- "mtp.layers.0.mlp.experts.110.gate_proj",
1012
- "mtp.layers.0.mlp.experts.110.up_proj",
1013
- "mtp.layers.0.mlp.experts.111.down_proj",
1014
- "mtp.layers.0.mlp.experts.111.gate_proj",
1015
- "mtp.layers.0.mlp.experts.111.up_proj",
1016
- "mtp.layers.0.mlp.experts.112.down_proj",
1017
- "mtp.layers.0.mlp.experts.112.gate_proj",
1018
- "mtp.layers.0.mlp.experts.112.up_proj",
1019
- "mtp.layers.0.mlp.experts.113.down_proj",
1020
- "mtp.layers.0.mlp.experts.113.gate_proj",
1021
- "mtp.layers.0.mlp.experts.113.up_proj",
1022
- "mtp.layers.0.mlp.experts.114.down_proj",
1023
- "mtp.layers.0.mlp.experts.114.gate_proj",
1024
- "mtp.layers.0.mlp.experts.114.up_proj",
1025
- "mtp.layers.0.mlp.experts.115.down_proj",
1026
- "mtp.layers.0.mlp.experts.115.gate_proj",
1027
- "mtp.layers.0.mlp.experts.115.up_proj",
1028
- "mtp.layers.0.mlp.experts.116.down_proj",
1029
- "mtp.layers.0.mlp.experts.116.gate_proj",
1030
- "mtp.layers.0.mlp.experts.116.up_proj",
1031
- "mtp.layers.0.mlp.experts.117.down_proj",
1032
- "mtp.layers.0.mlp.experts.117.gate_proj",
1033
- "mtp.layers.0.mlp.experts.117.up_proj",
1034
- "mtp.layers.0.mlp.experts.118.down_proj",
1035
- "mtp.layers.0.mlp.experts.118.gate_proj",
1036
- "mtp.layers.0.mlp.experts.118.up_proj",
1037
- "mtp.layers.0.mlp.experts.119.down_proj",
1038
- "mtp.layers.0.mlp.experts.119.gate_proj",
1039
- "mtp.layers.0.mlp.experts.119.up_proj",
1040
- "mtp.layers.0.mlp.experts.12.down_proj",
1041
- "mtp.layers.0.mlp.experts.12.gate_proj",
1042
- "mtp.layers.0.mlp.experts.12.up_proj",
1043
- "mtp.layers.0.mlp.experts.120.down_proj",
1044
- "mtp.layers.0.mlp.experts.120.gate_proj",
1045
- "mtp.layers.0.mlp.experts.120.up_proj",
1046
- "mtp.layers.0.mlp.experts.121.down_proj",
1047
- "mtp.layers.0.mlp.experts.121.gate_proj",
1048
- "mtp.layers.0.mlp.experts.121.up_proj",
1049
- "mtp.layers.0.mlp.experts.122.down_proj",
1050
- "mtp.layers.0.mlp.experts.122.gate_proj",
1051
- "mtp.layers.0.mlp.experts.122.up_proj",
1052
- "mtp.layers.0.mlp.experts.123.down_proj",
1053
- "mtp.layers.0.mlp.experts.123.gate_proj",
1054
- "mtp.layers.0.mlp.experts.123.up_proj",
1055
- "mtp.layers.0.mlp.experts.124.down_proj",
1056
- "mtp.layers.0.mlp.experts.124.gate_proj",
1057
- "mtp.layers.0.mlp.experts.124.up_proj",
1058
- "mtp.layers.0.mlp.experts.125.down_proj",
1059
- "mtp.layers.0.mlp.experts.125.gate_proj",
1060
- "mtp.layers.0.mlp.experts.125.up_proj",
1061
- "mtp.layers.0.mlp.experts.126.down_proj",
1062
- "mtp.layers.0.mlp.experts.126.gate_proj",
1063
- "mtp.layers.0.mlp.experts.126.up_proj",
1064
- "mtp.layers.0.mlp.experts.127.down_proj",
1065
- "mtp.layers.0.mlp.experts.127.gate_proj",
1066
- "mtp.layers.0.mlp.experts.127.up_proj",
1067
- "mtp.layers.0.mlp.experts.128.down_proj",
1068
- "mtp.layers.0.mlp.experts.128.gate_proj",
1069
- "mtp.layers.0.mlp.experts.128.up_proj",
1070
- "mtp.layers.0.mlp.experts.129.down_proj",
1071
- "mtp.layers.0.mlp.experts.129.gate_proj",
1072
- "mtp.layers.0.mlp.experts.129.up_proj",
1073
- "mtp.layers.0.mlp.experts.13.down_proj",
1074
- "mtp.layers.0.mlp.experts.13.gate_proj",
1075
- "mtp.layers.0.mlp.experts.13.up_proj",
1076
- "mtp.layers.0.mlp.experts.130.down_proj",
1077
- "mtp.layers.0.mlp.experts.130.gate_proj",
1078
- "mtp.layers.0.mlp.experts.130.up_proj",
1079
- "mtp.layers.0.mlp.experts.131.down_proj",
1080
- "mtp.layers.0.mlp.experts.131.gate_proj",
1081
- "mtp.layers.0.mlp.experts.131.up_proj",
1082
- "mtp.layers.0.mlp.experts.132.down_proj",
1083
- "mtp.layers.0.mlp.experts.132.gate_proj",
1084
- "mtp.layers.0.mlp.experts.132.up_proj",
1085
- "mtp.layers.0.mlp.experts.133.down_proj",
1086
- "mtp.layers.0.mlp.experts.133.gate_proj",
1087
- "mtp.layers.0.mlp.experts.133.up_proj",
1088
- "mtp.layers.0.mlp.experts.134.down_proj",
1089
- "mtp.layers.0.mlp.experts.134.gate_proj",
1090
- "mtp.layers.0.mlp.experts.134.up_proj",
1091
- "mtp.layers.0.mlp.experts.135.down_proj",
1092
- "mtp.layers.0.mlp.experts.135.gate_proj",
1093
- "mtp.layers.0.mlp.experts.135.up_proj",
1094
- "mtp.layers.0.mlp.experts.136.down_proj",
1095
- "mtp.layers.0.mlp.experts.136.gate_proj",
1096
- "mtp.layers.0.mlp.experts.136.up_proj",
1097
- "mtp.layers.0.mlp.experts.137.down_proj",
1098
- "mtp.layers.0.mlp.experts.137.gate_proj",
1099
- "mtp.layers.0.mlp.experts.137.up_proj",
1100
- "mtp.layers.0.mlp.experts.138.down_proj",
1101
- "mtp.layers.0.mlp.experts.138.gate_proj",
1102
- "mtp.layers.0.mlp.experts.138.up_proj",
1103
- "mtp.layers.0.mlp.experts.139.down_proj",
1104
- "mtp.layers.0.mlp.experts.139.gate_proj",
1105
- "mtp.layers.0.mlp.experts.139.up_proj",
1106
- "mtp.layers.0.mlp.experts.14.down_proj",
1107
- "mtp.layers.0.mlp.experts.14.gate_proj",
1108
- "mtp.layers.0.mlp.experts.14.up_proj",
1109
- "mtp.layers.0.mlp.experts.140.down_proj",
1110
- "mtp.layers.0.mlp.experts.140.gate_proj",
1111
- "mtp.layers.0.mlp.experts.140.up_proj",
1112
- "mtp.layers.0.mlp.experts.141.down_proj",
1113
- "mtp.layers.0.mlp.experts.141.gate_proj",
1114
- "mtp.layers.0.mlp.experts.141.up_proj",
1115
- "mtp.layers.0.mlp.experts.142.down_proj",
1116
- "mtp.layers.0.mlp.experts.142.gate_proj",
1117
- "mtp.layers.0.mlp.experts.142.up_proj",
1118
- "mtp.layers.0.mlp.experts.143.down_proj",
1119
- "mtp.layers.0.mlp.experts.143.gate_proj",
1120
- "mtp.layers.0.mlp.experts.143.up_proj",
1121
- "mtp.layers.0.mlp.experts.144.down_proj",
1122
- "mtp.layers.0.mlp.experts.144.gate_proj",
1123
- "mtp.layers.0.mlp.experts.144.up_proj",
1124
- "mtp.layers.0.mlp.experts.145.down_proj",
1125
- "mtp.layers.0.mlp.experts.145.gate_proj",
1126
- "mtp.layers.0.mlp.experts.145.up_proj",
1127
- "mtp.layers.0.mlp.experts.146.down_proj",
1128
- "mtp.layers.0.mlp.experts.146.gate_proj",
1129
- "mtp.layers.0.mlp.experts.146.up_proj",
1130
- "mtp.layers.0.mlp.experts.147.down_proj",
1131
- "mtp.layers.0.mlp.experts.147.gate_proj",
1132
- "mtp.layers.0.mlp.experts.147.up_proj",
1133
- "mtp.layers.0.mlp.experts.148.down_proj",
1134
- "mtp.layers.0.mlp.experts.148.gate_proj",
1135
- "mtp.layers.0.mlp.experts.148.up_proj",
1136
- "mtp.layers.0.mlp.experts.149.down_proj",
1137
- "mtp.layers.0.mlp.experts.149.gate_proj",
1138
- "mtp.layers.0.mlp.experts.149.up_proj",
1139
- "mtp.layers.0.mlp.experts.15.down_proj",
1140
- "mtp.layers.0.mlp.experts.15.gate_proj",
1141
- "mtp.layers.0.mlp.experts.15.up_proj",
1142
- "mtp.layers.0.mlp.experts.150.down_proj",
1143
- "mtp.layers.0.mlp.experts.150.gate_proj",
1144
- "mtp.layers.0.mlp.experts.150.up_proj",
1145
- "mtp.layers.0.mlp.experts.151.down_proj",
1146
- "mtp.layers.0.mlp.experts.151.gate_proj",
1147
- "mtp.layers.0.mlp.experts.151.up_proj",
1148
- "mtp.layers.0.mlp.experts.152.down_proj",
1149
- "mtp.layers.0.mlp.experts.152.gate_proj",
1150
- "mtp.layers.0.mlp.experts.152.up_proj",
1151
- "mtp.layers.0.mlp.experts.153.down_proj",
1152
- "mtp.layers.0.mlp.experts.153.gate_proj",
1153
- "mtp.layers.0.mlp.experts.153.up_proj",
1154
- "mtp.layers.0.mlp.experts.154.down_proj",
1155
- "mtp.layers.0.mlp.experts.154.gate_proj",
1156
- "mtp.layers.0.mlp.experts.154.up_proj",
1157
- "mtp.layers.0.mlp.experts.155.down_proj",
1158
- "mtp.layers.0.mlp.experts.155.gate_proj",
1159
- "mtp.layers.0.mlp.experts.155.up_proj",
1160
- "mtp.layers.0.mlp.experts.156.down_proj",
1161
- "mtp.layers.0.mlp.experts.156.gate_proj",
1162
- "mtp.layers.0.mlp.experts.156.up_proj",
1163
- "mtp.layers.0.mlp.experts.157.down_proj",
1164
- "mtp.layers.0.mlp.experts.157.gate_proj",
1165
- "mtp.layers.0.mlp.experts.157.up_proj",
1166
- "mtp.layers.0.mlp.experts.158.down_proj",
1167
- "mtp.layers.0.mlp.experts.158.gate_proj",
1168
- "mtp.layers.0.mlp.experts.158.up_proj",
1169
- "mtp.layers.0.mlp.experts.159.down_proj",
1170
- "mtp.layers.0.mlp.experts.159.gate_proj",
1171
- "mtp.layers.0.mlp.experts.159.up_proj",
1172
- "mtp.layers.0.mlp.experts.16.down_proj",
1173
- "mtp.layers.0.mlp.experts.16.gate_proj",
1174
- "mtp.layers.0.mlp.experts.16.up_proj",
1175
- "mtp.layers.0.mlp.experts.160.down_proj",
1176
- "mtp.layers.0.mlp.experts.160.gate_proj",
1177
- "mtp.layers.0.mlp.experts.160.up_proj",
1178
- "mtp.layers.0.mlp.experts.161.down_proj",
1179
- "mtp.layers.0.mlp.experts.161.gate_proj",
1180
- "mtp.layers.0.mlp.experts.161.up_proj",
1181
- "mtp.layers.0.mlp.experts.162.down_proj",
1182
- "mtp.layers.0.mlp.experts.162.gate_proj",
1183
- "mtp.layers.0.mlp.experts.162.up_proj",
1184
- "mtp.layers.0.mlp.experts.163.down_proj",
1185
- "mtp.layers.0.mlp.experts.163.gate_proj",
1186
- "mtp.layers.0.mlp.experts.163.up_proj",
1187
- "mtp.layers.0.mlp.experts.164.down_proj",
1188
- "mtp.layers.0.mlp.experts.164.gate_proj",
1189
- "mtp.layers.0.mlp.experts.164.up_proj",
1190
- "mtp.layers.0.mlp.experts.165.down_proj",
1191
- "mtp.layers.0.mlp.experts.165.gate_proj",
1192
- "mtp.layers.0.mlp.experts.165.up_proj",
1193
- "mtp.layers.0.mlp.experts.166.down_proj",
1194
- "mtp.layers.0.mlp.experts.166.gate_proj",
1195
- "mtp.layers.0.mlp.experts.166.up_proj",
1196
- "mtp.layers.0.mlp.experts.167.down_proj",
1197
- "mtp.layers.0.mlp.experts.167.gate_proj",
1198
- "mtp.layers.0.mlp.experts.167.up_proj",
1199
- "mtp.layers.0.mlp.experts.168.down_proj",
1200
- "mtp.layers.0.mlp.experts.168.gate_proj",
1201
- "mtp.layers.0.mlp.experts.168.up_proj",
1202
- "mtp.layers.0.mlp.experts.169.down_proj",
1203
- "mtp.layers.0.mlp.experts.169.gate_proj",
1204
- "mtp.layers.0.mlp.experts.169.up_proj",
1205
- "mtp.layers.0.mlp.experts.17.down_proj",
1206
- "mtp.layers.0.mlp.experts.17.gate_proj",
1207
- "mtp.layers.0.mlp.experts.17.up_proj",
1208
- "mtp.layers.0.mlp.experts.170.down_proj",
1209
- "mtp.layers.0.mlp.experts.170.gate_proj",
1210
- "mtp.layers.0.mlp.experts.170.up_proj",
1211
- "mtp.layers.0.mlp.experts.171.down_proj",
1212
- "mtp.layers.0.mlp.experts.171.gate_proj",
1213
- "mtp.layers.0.mlp.experts.171.up_proj",
1214
- "mtp.layers.0.mlp.experts.172.down_proj",
1215
- "mtp.layers.0.mlp.experts.172.gate_proj",
1216
- "mtp.layers.0.mlp.experts.172.up_proj",
1217
- "mtp.layers.0.mlp.experts.173.down_proj",
1218
- "mtp.layers.0.mlp.experts.173.gate_proj",
1219
- "mtp.layers.0.mlp.experts.173.up_proj",
1220
- "mtp.layers.0.mlp.experts.174.down_proj",
1221
- "mtp.layers.0.mlp.experts.174.gate_proj",
1222
- "mtp.layers.0.mlp.experts.174.up_proj",
1223
- "mtp.layers.0.mlp.experts.175.down_proj",
1224
- "mtp.layers.0.mlp.experts.175.gate_proj",
1225
- "mtp.layers.0.mlp.experts.175.up_proj",
1226
- "mtp.layers.0.mlp.experts.176.down_proj",
1227
- "mtp.layers.0.mlp.experts.176.gate_proj",
1228
- "mtp.layers.0.mlp.experts.176.up_proj",
1229
- "mtp.layers.0.mlp.experts.177.down_proj",
1230
- "mtp.layers.0.mlp.experts.177.gate_proj",
1231
- "mtp.layers.0.mlp.experts.177.up_proj",
1232
- "mtp.layers.0.mlp.experts.178.down_proj",
1233
- "mtp.layers.0.mlp.experts.178.gate_proj",
1234
- "mtp.layers.0.mlp.experts.178.up_proj",
1235
- "mtp.layers.0.mlp.experts.179.down_proj",
1236
- "mtp.layers.0.mlp.experts.179.gate_proj",
1237
- "mtp.layers.0.mlp.experts.179.up_proj",
1238
- "mtp.layers.0.mlp.experts.18.down_proj",
1239
- "mtp.layers.0.mlp.experts.18.gate_proj",
1240
- "mtp.layers.0.mlp.experts.18.up_proj",
1241
- "mtp.layers.0.mlp.experts.180.down_proj",
1242
- "mtp.layers.0.mlp.experts.180.gate_proj",
1243
- "mtp.layers.0.mlp.experts.180.up_proj",
1244
- "mtp.layers.0.mlp.experts.181.down_proj",
1245
- "mtp.layers.0.mlp.experts.181.gate_proj",
1246
- "mtp.layers.0.mlp.experts.181.up_proj",
1247
- "mtp.layers.0.mlp.experts.182.down_proj",
1248
- "mtp.layers.0.mlp.experts.182.gate_proj",
1249
- "mtp.layers.0.mlp.experts.182.up_proj",
1250
- "mtp.layers.0.mlp.experts.183.down_proj",
1251
- "mtp.layers.0.mlp.experts.183.gate_proj",
1252
- "mtp.layers.0.mlp.experts.183.up_proj",
1253
- "mtp.layers.0.mlp.experts.184.down_proj",
1254
- "mtp.layers.0.mlp.experts.184.gate_proj",
1255
- "mtp.layers.0.mlp.experts.184.up_proj",
1256
- "mtp.layers.0.mlp.experts.185.down_proj",
1257
- "mtp.layers.0.mlp.experts.185.gate_proj",
1258
- "mtp.layers.0.mlp.experts.185.up_proj",
1259
- "mtp.layers.0.mlp.experts.186.down_proj",
1260
- "mtp.layers.0.mlp.experts.186.gate_proj",
1261
- "mtp.layers.0.mlp.experts.186.up_proj",
1262
- "mtp.layers.0.mlp.experts.187.down_proj",
1263
- "mtp.layers.0.mlp.experts.187.gate_proj",
1264
- "mtp.layers.0.mlp.experts.187.up_proj",
1265
- "mtp.layers.0.mlp.experts.188.down_proj",
1266
- "mtp.layers.0.mlp.experts.188.gate_proj",
1267
- "mtp.layers.0.mlp.experts.188.up_proj",
1268
- "mtp.layers.0.mlp.experts.189.down_proj",
1269
- "mtp.layers.0.mlp.experts.189.gate_proj",
1270
- "mtp.layers.0.mlp.experts.189.up_proj",
1271
- "mtp.layers.0.mlp.experts.19.down_proj",
1272
- "mtp.layers.0.mlp.experts.19.gate_proj",
1273
- "mtp.layers.0.mlp.experts.19.up_proj",
1274
- "mtp.layers.0.mlp.experts.190.down_proj",
1275
- "mtp.layers.0.mlp.experts.190.gate_proj",
1276
- "mtp.layers.0.mlp.experts.190.up_proj",
1277
- "mtp.layers.0.mlp.experts.191.down_proj",
1278
- "mtp.layers.0.mlp.experts.191.gate_proj",
1279
- "mtp.layers.0.mlp.experts.191.up_proj",
1280
- "mtp.layers.0.mlp.experts.192.down_proj",
1281
- "mtp.layers.0.mlp.experts.192.gate_proj",
1282
- "mtp.layers.0.mlp.experts.192.up_proj",
1283
- "mtp.layers.0.mlp.experts.193.down_proj",
1284
- "mtp.layers.0.mlp.experts.193.gate_proj",
1285
- "mtp.layers.0.mlp.experts.193.up_proj",
1286
- "mtp.layers.0.mlp.experts.194.down_proj",
1287
- "mtp.layers.0.mlp.experts.194.gate_proj",
1288
- "mtp.layers.0.mlp.experts.194.up_proj",
1289
- "mtp.layers.0.mlp.experts.195.down_proj",
1290
- "mtp.layers.0.mlp.experts.195.gate_proj",
1291
- "mtp.layers.0.mlp.experts.195.up_proj",
1292
- "mtp.layers.0.mlp.experts.196.down_proj",
1293
- "mtp.layers.0.mlp.experts.196.gate_proj",
1294
- "mtp.layers.0.mlp.experts.196.up_proj",
1295
- "mtp.layers.0.mlp.experts.197.down_proj",
1296
- "mtp.layers.0.mlp.experts.197.gate_proj",
1297
- "mtp.layers.0.mlp.experts.197.up_proj",
1298
- "mtp.layers.0.mlp.experts.198.down_proj",
1299
- "mtp.layers.0.mlp.experts.198.gate_proj",
1300
- "mtp.layers.0.mlp.experts.198.up_proj",
1301
- "mtp.layers.0.mlp.experts.199.down_proj",
1302
- "mtp.layers.0.mlp.experts.199.gate_proj",
1303
- "mtp.layers.0.mlp.experts.199.up_proj",
1304
- "mtp.layers.0.mlp.experts.2.down_proj",
1305
- "mtp.layers.0.mlp.experts.2.gate_proj",
1306
- "mtp.layers.0.mlp.experts.2.up_proj",
1307
- "mtp.layers.0.mlp.experts.20.down_proj",
1308
- "mtp.layers.0.mlp.experts.20.gate_proj",
1309
- "mtp.layers.0.mlp.experts.20.up_proj",
1310
- "mtp.layers.0.mlp.experts.200.down_proj",
1311
- "mtp.layers.0.mlp.experts.200.gate_proj",
1312
- "mtp.layers.0.mlp.experts.200.up_proj",
1313
- "mtp.layers.0.mlp.experts.201.down_proj",
1314
- "mtp.layers.0.mlp.experts.201.gate_proj",
1315
- "mtp.layers.0.mlp.experts.201.up_proj",
1316
- "mtp.layers.0.mlp.experts.202.down_proj",
1317
- "mtp.layers.0.mlp.experts.202.gate_proj",
1318
- "mtp.layers.0.mlp.experts.202.up_proj",
1319
- "mtp.layers.0.mlp.experts.203.down_proj",
1320
- "mtp.layers.0.mlp.experts.203.gate_proj",
1321
- "mtp.layers.0.mlp.experts.203.up_proj",
1322
- "mtp.layers.0.mlp.experts.204.down_proj",
1323
- "mtp.layers.0.mlp.experts.204.gate_proj",
1324
- "mtp.layers.0.mlp.experts.204.up_proj",
1325
- "mtp.layers.0.mlp.experts.205.down_proj",
1326
- "mtp.layers.0.mlp.experts.205.gate_proj",
1327
- "mtp.layers.0.mlp.experts.205.up_proj",
1328
- "mtp.layers.0.mlp.experts.206.down_proj",
1329
- "mtp.layers.0.mlp.experts.206.gate_proj",
1330
- "mtp.layers.0.mlp.experts.206.up_proj",
1331
- "mtp.layers.0.mlp.experts.207.down_proj",
1332
- "mtp.layers.0.mlp.experts.207.gate_proj",
1333
- "mtp.layers.0.mlp.experts.207.up_proj",
1334
- "mtp.layers.0.mlp.experts.208.down_proj",
1335
- "mtp.layers.0.mlp.experts.208.gate_proj",
1336
- "mtp.layers.0.mlp.experts.208.up_proj",
1337
- "mtp.layers.0.mlp.experts.209.down_proj",
1338
- "mtp.layers.0.mlp.experts.209.gate_proj",
1339
- "mtp.layers.0.mlp.experts.209.up_proj",
1340
- "mtp.layers.0.mlp.experts.21.down_proj",
1341
- "mtp.layers.0.mlp.experts.21.gate_proj",
1342
- "mtp.layers.0.mlp.experts.21.up_proj",
1343
- "mtp.layers.0.mlp.experts.210.down_proj",
1344
- "mtp.layers.0.mlp.experts.210.gate_proj",
1345
- "mtp.layers.0.mlp.experts.210.up_proj",
1346
- "mtp.layers.0.mlp.experts.211.down_proj",
1347
- "mtp.layers.0.mlp.experts.211.gate_proj",
1348
- "mtp.layers.0.mlp.experts.211.up_proj",
1349
- "mtp.layers.0.mlp.experts.212.down_proj",
1350
- "mtp.layers.0.mlp.experts.212.gate_proj",
1351
- "mtp.layers.0.mlp.experts.212.up_proj",
1352
- "mtp.layers.0.mlp.experts.213.down_proj",
1353
- "mtp.layers.0.mlp.experts.213.gate_proj",
1354
- "mtp.layers.0.mlp.experts.213.up_proj",
1355
- "mtp.layers.0.mlp.experts.214.down_proj",
1356
- "mtp.layers.0.mlp.experts.214.gate_proj",
1357
- "mtp.layers.0.mlp.experts.214.up_proj",
1358
- "mtp.layers.0.mlp.experts.215.down_proj",
1359
- "mtp.layers.0.mlp.experts.215.gate_proj",
1360
- "mtp.layers.0.mlp.experts.215.up_proj",
1361
- "mtp.layers.0.mlp.experts.216.down_proj",
1362
- "mtp.layers.0.mlp.experts.216.gate_proj",
1363
- "mtp.layers.0.mlp.experts.216.up_proj",
1364
- "mtp.layers.0.mlp.experts.217.down_proj",
1365
- "mtp.layers.0.mlp.experts.217.gate_proj",
1366
- "mtp.layers.0.mlp.experts.217.up_proj",
1367
- "mtp.layers.0.mlp.experts.218.down_proj",
1368
- "mtp.layers.0.mlp.experts.218.gate_proj",
1369
- "mtp.layers.0.mlp.experts.218.up_proj",
1370
- "mtp.layers.0.mlp.experts.219.down_proj",
1371
- "mtp.layers.0.mlp.experts.219.gate_proj",
1372
- "mtp.layers.0.mlp.experts.219.up_proj",
1373
- "mtp.layers.0.mlp.experts.22.down_proj",
1374
- "mtp.layers.0.mlp.experts.22.gate_proj",
1375
- "mtp.layers.0.mlp.experts.22.up_proj",
1376
- "mtp.layers.0.mlp.experts.220.down_proj",
1377
- "mtp.layers.0.mlp.experts.220.gate_proj",
1378
- "mtp.layers.0.mlp.experts.220.up_proj",
1379
- "mtp.layers.0.mlp.experts.221.down_proj",
1380
- "mtp.layers.0.mlp.experts.221.gate_proj",
1381
- "mtp.layers.0.mlp.experts.221.up_proj",
1382
- "mtp.layers.0.mlp.experts.222.down_proj",
1383
- "mtp.layers.0.mlp.experts.222.gate_proj",
1384
- "mtp.layers.0.mlp.experts.222.up_proj",
1385
- "mtp.layers.0.mlp.experts.223.down_proj",
1386
- "mtp.layers.0.mlp.experts.223.gate_proj",
1387
- "mtp.layers.0.mlp.experts.223.up_proj",
1388
- "mtp.layers.0.mlp.experts.224.down_proj",
1389
- "mtp.layers.0.mlp.experts.224.gate_proj",
1390
- "mtp.layers.0.mlp.experts.224.up_proj",
1391
- "mtp.layers.0.mlp.experts.225.down_proj",
1392
- "mtp.layers.0.mlp.experts.225.gate_proj",
1393
- "mtp.layers.0.mlp.experts.225.up_proj",
1394
- "mtp.layers.0.mlp.experts.226.down_proj",
1395
- "mtp.layers.0.mlp.experts.226.gate_proj",
1396
- "mtp.layers.0.mlp.experts.226.up_proj",
1397
- "mtp.layers.0.mlp.experts.227.down_proj",
1398
- "mtp.layers.0.mlp.experts.227.gate_proj",
1399
- "mtp.layers.0.mlp.experts.227.up_proj",
1400
- "mtp.layers.0.mlp.experts.228.down_proj",
1401
- "mtp.layers.0.mlp.experts.228.gate_proj",
1402
- "mtp.layers.0.mlp.experts.228.up_proj",
1403
- "mtp.layers.0.mlp.experts.229.down_proj",
1404
- "mtp.layers.0.mlp.experts.229.gate_proj",
1405
- "mtp.layers.0.mlp.experts.229.up_proj",
1406
- "mtp.layers.0.mlp.experts.23.down_proj",
1407
- "mtp.layers.0.mlp.experts.23.gate_proj",
1408
- "mtp.layers.0.mlp.experts.23.up_proj",
1409
- "mtp.layers.0.mlp.experts.230.down_proj",
1410
- "mtp.layers.0.mlp.experts.230.gate_proj",
1411
- "mtp.layers.0.mlp.experts.230.up_proj",
1412
- "mtp.layers.0.mlp.experts.231.down_proj",
1413
- "mtp.layers.0.mlp.experts.231.gate_proj",
1414
- "mtp.layers.0.mlp.experts.231.up_proj",
1415
- "mtp.layers.0.mlp.experts.232.down_proj",
1416
- "mtp.layers.0.mlp.experts.232.gate_proj",
1417
- "mtp.layers.0.mlp.experts.232.up_proj",
1418
- "mtp.layers.0.mlp.experts.233.down_proj",
1419
- "mtp.layers.0.mlp.experts.233.gate_proj",
1420
- "mtp.layers.0.mlp.experts.233.up_proj",
1421
- "mtp.layers.0.mlp.experts.234.down_proj",
1422
- "mtp.layers.0.mlp.experts.234.gate_proj",
1423
- "mtp.layers.0.mlp.experts.234.up_proj",
1424
- "mtp.layers.0.mlp.experts.235.down_proj",
1425
- "mtp.layers.0.mlp.experts.235.gate_proj",
1426
- "mtp.layers.0.mlp.experts.235.up_proj",
1427
- "mtp.layers.0.mlp.experts.236.down_proj",
1428
- "mtp.layers.0.mlp.experts.236.gate_proj",
1429
- "mtp.layers.0.mlp.experts.236.up_proj",
1430
- "mtp.layers.0.mlp.experts.237.down_proj",
1431
- "mtp.layers.0.mlp.experts.237.gate_proj",
1432
- "mtp.layers.0.mlp.experts.237.up_proj",
1433
- "mtp.layers.0.mlp.experts.238.down_proj",
1434
- "mtp.layers.0.mlp.experts.238.gate_proj",
1435
- "mtp.layers.0.mlp.experts.238.up_proj",
1436
- "mtp.layers.0.mlp.experts.239.down_proj",
1437
- "mtp.layers.0.mlp.experts.239.gate_proj",
1438
- "mtp.layers.0.mlp.experts.239.up_proj",
1439
- "mtp.layers.0.mlp.experts.24.down_proj",
1440
- "mtp.layers.0.mlp.experts.24.gate_proj",
1441
- "mtp.layers.0.mlp.experts.24.up_proj",
1442
- "mtp.layers.0.mlp.experts.240.down_proj",
1443
- "mtp.layers.0.mlp.experts.240.gate_proj",
1444
- "mtp.layers.0.mlp.experts.240.up_proj",
1445
- "mtp.layers.0.mlp.experts.241.down_proj",
1446
- "mtp.layers.0.mlp.experts.241.gate_proj",
1447
- "mtp.layers.0.mlp.experts.241.up_proj",
1448
- "mtp.layers.0.mlp.experts.242.down_proj",
1449
- "mtp.layers.0.mlp.experts.242.gate_proj",
1450
- "mtp.layers.0.mlp.experts.242.up_proj",
1451
- "mtp.layers.0.mlp.experts.243.down_proj",
1452
- "mtp.layers.0.mlp.experts.243.gate_proj",
1453
- "mtp.layers.0.mlp.experts.243.up_proj",
1454
- "mtp.layers.0.mlp.experts.244.down_proj",
1455
- "mtp.layers.0.mlp.experts.244.gate_proj",
1456
- "mtp.layers.0.mlp.experts.244.up_proj",
1457
- "mtp.layers.0.mlp.experts.245.down_proj",
1458
- "mtp.layers.0.mlp.experts.245.gate_proj",
1459
- "mtp.layers.0.mlp.experts.245.up_proj",
1460
- "mtp.layers.0.mlp.experts.246.down_proj",
1461
- "mtp.layers.0.mlp.experts.246.gate_proj",
1462
- "mtp.layers.0.mlp.experts.246.up_proj",
1463
- "mtp.layers.0.mlp.experts.247.down_proj",
1464
- "mtp.layers.0.mlp.experts.247.gate_proj",
1465
- "mtp.layers.0.mlp.experts.247.up_proj",
1466
- "mtp.layers.0.mlp.experts.248.down_proj",
1467
- "mtp.layers.0.mlp.experts.248.gate_proj",
1468
- "mtp.layers.0.mlp.experts.248.up_proj",
1469
- "mtp.layers.0.mlp.experts.249.down_proj",
1470
- "mtp.layers.0.mlp.experts.249.gate_proj",
1471
- "mtp.layers.0.mlp.experts.249.up_proj",
1472
- "mtp.layers.0.mlp.experts.25.down_proj",
1473
- "mtp.layers.0.mlp.experts.25.gate_proj",
1474
- "mtp.layers.0.mlp.experts.25.up_proj",
1475
- "mtp.layers.0.mlp.experts.250.down_proj",
1476
- "mtp.layers.0.mlp.experts.250.gate_proj",
1477
- "mtp.layers.0.mlp.experts.250.up_proj",
1478
- "mtp.layers.0.mlp.experts.251.down_proj",
1479
- "mtp.layers.0.mlp.experts.251.gate_proj",
1480
- "mtp.layers.0.mlp.experts.251.up_proj",
1481
- "mtp.layers.0.mlp.experts.252.down_proj",
1482
- "mtp.layers.0.mlp.experts.252.gate_proj",
1483
- "mtp.layers.0.mlp.experts.252.up_proj",
1484
- "mtp.layers.0.mlp.experts.253.down_proj",
1485
- "mtp.layers.0.mlp.experts.253.gate_proj",
1486
- "mtp.layers.0.mlp.experts.253.up_proj",
1487
- "mtp.layers.0.mlp.experts.254.down_proj",
1488
- "mtp.layers.0.mlp.experts.254.gate_proj",
1489
- "mtp.layers.0.mlp.experts.254.up_proj",
1490
- "mtp.layers.0.mlp.experts.255.down_proj",
1491
- "mtp.layers.0.mlp.experts.255.gate_proj",
1492
- "mtp.layers.0.mlp.experts.255.up_proj",
1493
- "mtp.layers.0.mlp.experts.256.down_proj",
1494
- "mtp.layers.0.mlp.experts.256.gate_proj",
1495
- "mtp.layers.0.mlp.experts.256.up_proj",
1496
- "mtp.layers.0.mlp.experts.257.down_proj",
1497
- "mtp.layers.0.mlp.experts.257.gate_proj",
1498
- "mtp.layers.0.mlp.experts.257.up_proj",
1499
- "mtp.layers.0.mlp.experts.258.down_proj",
1500
- "mtp.layers.0.mlp.experts.258.gate_proj",
1501
- "mtp.layers.0.mlp.experts.258.up_proj",
1502
- "mtp.layers.0.mlp.experts.259.down_proj",
1503
- "mtp.layers.0.mlp.experts.259.gate_proj",
1504
- "mtp.layers.0.mlp.experts.259.up_proj",
1505
- "mtp.layers.0.mlp.experts.26.down_proj",
1506
- "mtp.layers.0.mlp.experts.26.gate_proj",
1507
- "mtp.layers.0.mlp.experts.26.up_proj",
1508
- "mtp.layers.0.mlp.experts.260.down_proj",
1509
- "mtp.layers.0.mlp.experts.260.gate_proj",
1510
- "mtp.layers.0.mlp.experts.260.up_proj",
1511
- "mtp.layers.0.mlp.experts.261.down_proj",
1512
- "mtp.layers.0.mlp.experts.261.gate_proj",
1513
- "mtp.layers.0.mlp.experts.261.up_proj",
1514
- "mtp.layers.0.mlp.experts.262.down_proj",
1515
- "mtp.layers.0.mlp.experts.262.gate_proj",
1516
- "mtp.layers.0.mlp.experts.262.up_proj",
1517
- "mtp.layers.0.mlp.experts.263.down_proj",
1518
- "mtp.layers.0.mlp.experts.263.gate_proj",
1519
- "mtp.layers.0.mlp.experts.263.up_proj",
1520
- "mtp.layers.0.mlp.experts.264.down_proj",
1521
- "mtp.layers.0.mlp.experts.264.gate_proj",
1522
- "mtp.layers.0.mlp.experts.264.up_proj",
1523
- "mtp.layers.0.mlp.experts.265.down_proj",
1524
- "mtp.layers.0.mlp.experts.265.gate_proj",
1525
- "mtp.layers.0.mlp.experts.265.up_proj",
1526
- "mtp.layers.0.mlp.experts.266.down_proj",
1527
- "mtp.layers.0.mlp.experts.266.gate_proj",
1528
- "mtp.layers.0.mlp.experts.266.up_proj",
1529
- "mtp.layers.0.mlp.experts.267.down_proj",
1530
- "mtp.layers.0.mlp.experts.267.gate_proj",
1531
- "mtp.layers.0.mlp.experts.267.up_proj",
1532
- "mtp.layers.0.mlp.experts.268.down_proj",
1533
- "mtp.layers.0.mlp.experts.268.gate_proj",
1534
- "mtp.layers.0.mlp.experts.268.up_proj",
1535
- "mtp.layers.0.mlp.experts.269.down_proj",
1536
- "mtp.layers.0.mlp.experts.269.gate_proj",
1537
- "mtp.layers.0.mlp.experts.269.up_proj",
1538
- "mtp.layers.0.mlp.experts.27.down_proj",
1539
- "mtp.layers.0.mlp.experts.27.gate_proj",
1540
- "mtp.layers.0.mlp.experts.27.up_proj",
1541
- "mtp.layers.0.mlp.experts.270.down_proj",
1542
- "mtp.layers.0.mlp.experts.270.gate_proj",
1543
- "mtp.layers.0.mlp.experts.270.up_proj",
1544
- "mtp.layers.0.mlp.experts.271.down_proj",
1545
- "mtp.layers.0.mlp.experts.271.gate_proj",
1546
- "mtp.layers.0.mlp.experts.271.up_proj",
1547
- "mtp.layers.0.mlp.experts.272.down_proj",
1548
- "mtp.layers.0.mlp.experts.272.gate_proj",
1549
- "mtp.layers.0.mlp.experts.272.up_proj",
1550
- "mtp.layers.0.mlp.experts.273.down_proj",
1551
- "mtp.layers.0.mlp.experts.273.gate_proj",
1552
- "mtp.layers.0.mlp.experts.273.up_proj",
1553
- "mtp.layers.0.mlp.experts.274.down_proj",
1554
- "mtp.layers.0.mlp.experts.274.gate_proj",
1555
- "mtp.layers.0.mlp.experts.274.up_proj",
1556
- "mtp.layers.0.mlp.experts.275.down_proj",
1557
- "mtp.layers.0.mlp.experts.275.gate_proj",
1558
- "mtp.layers.0.mlp.experts.275.up_proj",
1559
- "mtp.layers.0.mlp.experts.276.down_proj",
1560
- "mtp.layers.0.mlp.experts.276.gate_proj",
1561
- "mtp.layers.0.mlp.experts.276.up_proj",
1562
- "mtp.layers.0.mlp.experts.277.down_proj",
1563
- "mtp.layers.0.mlp.experts.277.gate_proj",
1564
- "mtp.layers.0.mlp.experts.277.up_proj",
1565
- "mtp.layers.0.mlp.experts.278.down_proj",
1566
- "mtp.layers.0.mlp.experts.278.gate_proj",
1567
- "mtp.layers.0.mlp.experts.278.up_proj",
1568
- "mtp.layers.0.mlp.experts.279.down_proj",
1569
- "mtp.layers.0.mlp.experts.279.gate_proj",
1570
- "mtp.layers.0.mlp.experts.279.up_proj",
1571
- "mtp.layers.0.mlp.experts.28.down_proj",
1572
- "mtp.layers.0.mlp.experts.28.gate_proj",
1573
- "mtp.layers.0.mlp.experts.28.up_proj",
1574
- "mtp.layers.0.mlp.experts.280.down_proj",
1575
- "mtp.layers.0.mlp.experts.280.gate_proj",
1576
- "mtp.layers.0.mlp.experts.280.up_proj",
1577
- "mtp.layers.0.mlp.experts.281.down_proj",
1578
- "mtp.layers.0.mlp.experts.281.gate_proj",
1579
- "mtp.layers.0.mlp.experts.281.up_proj",
1580
- "mtp.layers.0.mlp.experts.282.down_proj",
1581
- "mtp.layers.0.mlp.experts.282.gate_proj",
1582
- "mtp.layers.0.mlp.experts.282.up_proj",
1583
- "mtp.layers.0.mlp.experts.283.down_proj",
1584
- "mtp.layers.0.mlp.experts.283.gate_proj",
1585
- "mtp.layers.0.mlp.experts.283.up_proj",
1586
- "mtp.layers.0.mlp.experts.284.down_proj",
1587
- "mtp.layers.0.mlp.experts.284.gate_proj",
1588
- "mtp.layers.0.mlp.experts.284.up_proj",
1589
- "mtp.layers.0.mlp.experts.285.down_proj",
1590
- "mtp.layers.0.mlp.experts.285.gate_proj",
1591
- "mtp.layers.0.mlp.experts.285.up_proj",
1592
- "mtp.layers.0.mlp.experts.286.down_proj",
1593
- "mtp.layers.0.mlp.experts.286.gate_proj",
1594
- "mtp.layers.0.mlp.experts.286.up_proj",
1595
- "mtp.layers.0.mlp.experts.287.down_proj",
1596
- "mtp.layers.0.mlp.experts.287.gate_proj",
1597
- "mtp.layers.0.mlp.experts.287.up_proj",
1598
- "mtp.layers.0.mlp.experts.288.down_proj",
1599
- "mtp.layers.0.mlp.experts.288.gate_proj",
1600
- "mtp.layers.0.mlp.experts.288.up_proj",
1601
- "mtp.layers.0.mlp.experts.289.down_proj",
1602
- "mtp.layers.0.mlp.experts.289.gate_proj",
1603
- "mtp.layers.0.mlp.experts.289.up_proj",
1604
- "mtp.layers.0.mlp.experts.29.down_proj",
1605
- "mtp.layers.0.mlp.experts.29.gate_proj",
1606
- "mtp.layers.0.mlp.experts.29.up_proj",
1607
- "mtp.layers.0.mlp.experts.290.down_proj",
1608
- "mtp.layers.0.mlp.experts.290.gate_proj",
1609
- "mtp.layers.0.mlp.experts.290.up_proj",
1610
- "mtp.layers.0.mlp.experts.291.down_proj",
1611
- "mtp.layers.0.mlp.experts.291.gate_proj",
1612
- "mtp.layers.0.mlp.experts.291.up_proj",
1613
- "mtp.layers.0.mlp.experts.292.down_proj",
1614
- "mtp.layers.0.mlp.experts.292.gate_proj",
1615
- "mtp.layers.0.mlp.experts.292.up_proj",
1616
- "mtp.layers.0.mlp.experts.293.down_proj",
1617
- "mtp.layers.0.mlp.experts.293.gate_proj",
1618
- "mtp.layers.0.mlp.experts.293.up_proj",
1619
- "mtp.layers.0.mlp.experts.294.down_proj",
1620
- "mtp.layers.0.mlp.experts.294.gate_proj",
1621
- "mtp.layers.0.mlp.experts.294.up_proj",
1622
- "mtp.layers.0.mlp.experts.295.down_proj",
1623
- "mtp.layers.0.mlp.experts.295.gate_proj",
1624
- "mtp.layers.0.mlp.experts.295.up_proj",
1625
- "mtp.layers.0.mlp.experts.296.down_proj",
1626
- "mtp.layers.0.mlp.experts.296.gate_proj",
1627
- "mtp.layers.0.mlp.experts.296.up_proj",
1628
- "mtp.layers.0.mlp.experts.297.down_proj",
1629
- "mtp.layers.0.mlp.experts.297.gate_proj",
1630
- "mtp.layers.0.mlp.experts.297.up_proj",
1631
- "mtp.layers.0.mlp.experts.298.down_proj",
1632
- "mtp.layers.0.mlp.experts.298.gate_proj",
1633
- "mtp.layers.0.mlp.experts.298.up_proj",
1634
- "mtp.layers.0.mlp.experts.299.down_proj",
1635
- "mtp.layers.0.mlp.experts.299.gate_proj",
1636
- "mtp.layers.0.mlp.experts.299.up_proj",
1637
- "mtp.layers.0.mlp.experts.3.down_proj",
1638
- "mtp.layers.0.mlp.experts.3.gate_proj",
1639
- "mtp.layers.0.mlp.experts.3.up_proj",
1640
- "mtp.layers.0.mlp.experts.30.down_proj",
1641
- "mtp.layers.0.mlp.experts.30.gate_proj",
1642
- "mtp.layers.0.mlp.experts.30.up_proj",
1643
- "mtp.layers.0.mlp.experts.300.down_proj",
1644
- "mtp.layers.0.mlp.experts.300.gate_proj",
1645
- "mtp.layers.0.mlp.experts.300.up_proj",
1646
- "mtp.layers.0.mlp.experts.301.down_proj",
1647
- "mtp.layers.0.mlp.experts.301.gate_proj",
1648
- "mtp.layers.0.mlp.experts.301.up_proj",
1649
- "mtp.layers.0.mlp.experts.302.down_proj",
1650
- "mtp.layers.0.mlp.experts.302.gate_proj",
1651
- "mtp.layers.0.mlp.experts.302.up_proj",
1652
- "mtp.layers.0.mlp.experts.303.down_proj",
1653
- "mtp.layers.0.mlp.experts.303.gate_proj",
1654
- "mtp.layers.0.mlp.experts.303.up_proj",
1655
- "mtp.layers.0.mlp.experts.304.down_proj",
1656
- "mtp.layers.0.mlp.experts.304.gate_proj",
1657
- "mtp.layers.0.mlp.experts.304.up_proj",
1658
- "mtp.layers.0.mlp.experts.305.down_proj",
1659
- "mtp.layers.0.mlp.experts.305.gate_proj",
1660
- "mtp.layers.0.mlp.experts.305.up_proj",
1661
- "mtp.layers.0.mlp.experts.306.down_proj",
1662
- "mtp.layers.0.mlp.experts.306.gate_proj",
1663
- "mtp.layers.0.mlp.experts.306.up_proj",
1664
- "mtp.layers.0.mlp.experts.307.down_proj",
1665
- "mtp.layers.0.mlp.experts.307.gate_proj",
1666
- "mtp.layers.0.mlp.experts.307.up_proj",
1667
- "mtp.layers.0.mlp.experts.308.down_proj",
1668
- "mtp.layers.0.mlp.experts.308.gate_proj",
1669
- "mtp.layers.0.mlp.experts.308.up_proj",
1670
- "mtp.layers.0.mlp.experts.309.down_proj",
1671
- "mtp.layers.0.mlp.experts.309.gate_proj",
1672
- "mtp.layers.0.mlp.experts.309.up_proj",
1673
- "mtp.layers.0.mlp.experts.31.down_proj",
1674
- "mtp.layers.0.mlp.experts.31.gate_proj",
1675
- "mtp.layers.0.mlp.experts.31.up_proj",
1676
- "mtp.layers.0.mlp.experts.310.down_proj",
1677
- "mtp.layers.0.mlp.experts.310.gate_proj",
1678
- "mtp.layers.0.mlp.experts.310.up_proj",
1679
- "mtp.layers.0.mlp.experts.311.down_proj",
1680
- "mtp.layers.0.mlp.experts.311.gate_proj",
1681
- "mtp.layers.0.mlp.experts.311.up_proj",
1682
- "mtp.layers.0.mlp.experts.312.down_proj",
1683
- "mtp.layers.0.mlp.experts.312.gate_proj",
1684
- "mtp.layers.0.mlp.experts.312.up_proj",
1685
- "mtp.layers.0.mlp.experts.313.down_proj",
1686
- "mtp.layers.0.mlp.experts.313.gate_proj",
1687
- "mtp.layers.0.mlp.experts.313.up_proj",
1688
- "mtp.layers.0.mlp.experts.314.down_proj",
1689
- "mtp.layers.0.mlp.experts.314.gate_proj",
1690
- "mtp.layers.0.mlp.experts.314.up_proj",
1691
- "mtp.layers.0.mlp.experts.315.down_proj",
1692
- "mtp.layers.0.mlp.experts.315.gate_proj",
1693
- "mtp.layers.0.mlp.experts.315.up_proj",
1694
- "mtp.layers.0.mlp.experts.316.down_proj",
1695
- "mtp.layers.0.mlp.experts.316.gate_proj",
1696
- "mtp.layers.0.mlp.experts.316.up_proj",
1697
- "mtp.layers.0.mlp.experts.317.down_proj",
1698
- "mtp.layers.0.mlp.experts.317.gate_proj",
1699
- "mtp.layers.0.mlp.experts.317.up_proj",
1700
- "mtp.layers.0.mlp.experts.318.down_proj",
1701
- "mtp.layers.0.mlp.experts.318.gate_proj",
1702
- "mtp.layers.0.mlp.experts.318.up_proj",
1703
- "mtp.layers.0.mlp.experts.319.down_proj",
1704
- "mtp.layers.0.mlp.experts.319.gate_proj",
1705
- "mtp.layers.0.mlp.experts.319.up_proj",
1706
- "mtp.layers.0.mlp.experts.32.down_proj",
1707
- "mtp.layers.0.mlp.experts.32.gate_proj",
1708
- "mtp.layers.0.mlp.experts.32.up_proj",
1709
- "mtp.layers.0.mlp.experts.320.down_proj",
1710
- "mtp.layers.0.mlp.experts.320.gate_proj",
1711
- "mtp.layers.0.mlp.experts.320.up_proj",
1712
- "mtp.layers.0.mlp.experts.321.down_proj",
1713
- "mtp.layers.0.mlp.experts.321.gate_proj",
1714
- "mtp.layers.0.mlp.experts.321.up_proj",
1715
- "mtp.layers.0.mlp.experts.322.down_proj",
1716
- "mtp.layers.0.mlp.experts.322.gate_proj",
1717
- "mtp.layers.0.mlp.experts.322.up_proj",
1718
- "mtp.layers.0.mlp.experts.323.down_proj",
1719
- "mtp.layers.0.mlp.experts.323.gate_proj",
1720
- "mtp.layers.0.mlp.experts.323.up_proj",
1721
- "mtp.layers.0.mlp.experts.324.down_proj",
1722
- "mtp.layers.0.mlp.experts.324.gate_proj",
1723
- "mtp.layers.0.mlp.experts.324.up_proj",
1724
- "mtp.layers.0.mlp.experts.325.down_proj",
1725
- "mtp.layers.0.mlp.experts.325.gate_proj",
1726
- "mtp.layers.0.mlp.experts.325.up_proj",
1727
- "mtp.layers.0.mlp.experts.326.down_proj",
1728
- "mtp.layers.0.mlp.experts.326.gate_proj",
1729
- "mtp.layers.0.mlp.experts.326.up_proj",
1730
- "mtp.layers.0.mlp.experts.327.down_proj",
1731
- "mtp.layers.0.mlp.experts.327.gate_proj",
1732
- "mtp.layers.0.mlp.experts.327.up_proj",
1733
- "mtp.layers.0.mlp.experts.328.down_proj",
1734
- "mtp.layers.0.mlp.experts.328.gate_proj",
1735
- "mtp.layers.0.mlp.experts.328.up_proj",
1736
- "mtp.layers.0.mlp.experts.329.down_proj",
1737
- "mtp.layers.0.mlp.experts.329.gate_proj",
1738
- "mtp.layers.0.mlp.experts.329.up_proj",
1739
- "mtp.layers.0.mlp.experts.33.down_proj",
1740
- "mtp.layers.0.mlp.experts.33.gate_proj",
1741
- "mtp.layers.0.mlp.experts.33.up_proj",
1742
- "mtp.layers.0.mlp.experts.330.down_proj",
1743
- "mtp.layers.0.mlp.experts.330.gate_proj",
1744
- "mtp.layers.0.mlp.experts.330.up_proj",
1745
- "mtp.layers.0.mlp.experts.331.down_proj",
1746
- "mtp.layers.0.mlp.experts.331.gate_proj",
1747
- "mtp.layers.0.mlp.experts.331.up_proj",
1748
- "mtp.layers.0.mlp.experts.332.down_proj",
1749
- "mtp.layers.0.mlp.experts.332.gate_proj",
1750
- "mtp.layers.0.mlp.experts.332.up_proj",
1751
- "mtp.layers.0.mlp.experts.333.down_proj",
1752
- "mtp.layers.0.mlp.experts.333.gate_proj",
1753
- "mtp.layers.0.mlp.experts.333.up_proj",
1754
- "mtp.layers.0.mlp.experts.334.down_proj",
1755
- "mtp.layers.0.mlp.experts.334.gate_proj",
1756
- "mtp.layers.0.mlp.experts.334.up_proj",
1757
- "mtp.layers.0.mlp.experts.335.down_proj",
1758
- "mtp.layers.0.mlp.experts.335.gate_proj",
1759
- "mtp.layers.0.mlp.experts.335.up_proj",
1760
- "mtp.layers.0.mlp.experts.336.down_proj",
1761
- "mtp.layers.0.mlp.experts.336.gate_proj",
1762
- "mtp.layers.0.mlp.experts.336.up_proj",
1763
- "mtp.layers.0.mlp.experts.337.down_proj",
1764
- "mtp.layers.0.mlp.experts.337.gate_proj",
1765
- "mtp.layers.0.mlp.experts.337.up_proj",
1766
- "mtp.layers.0.mlp.experts.338.down_proj",
1767
- "mtp.layers.0.mlp.experts.338.gate_proj",
1768
- "mtp.layers.0.mlp.experts.338.up_proj",
1769
- "mtp.layers.0.mlp.experts.339.down_proj",
1770
- "mtp.layers.0.mlp.experts.339.gate_proj",
1771
- "mtp.layers.0.mlp.experts.339.up_proj",
1772
- "mtp.layers.0.mlp.experts.34.down_proj",
1773
- "mtp.layers.0.mlp.experts.34.gate_proj",
1774
- "mtp.layers.0.mlp.experts.34.up_proj",
1775
- "mtp.layers.0.mlp.experts.340.down_proj",
1776
- "mtp.layers.0.mlp.experts.340.gate_proj",
1777
- "mtp.layers.0.mlp.experts.340.up_proj",
1778
- "mtp.layers.0.mlp.experts.341.down_proj",
1779
- "mtp.layers.0.mlp.experts.341.gate_proj",
1780
- "mtp.layers.0.mlp.experts.341.up_proj",
1781
- "mtp.layers.0.mlp.experts.342.down_proj",
1782
- "mtp.layers.0.mlp.experts.342.gate_proj",
1783
- "mtp.layers.0.mlp.experts.342.up_proj",
1784
- "mtp.layers.0.mlp.experts.343.down_proj",
1785
- "mtp.layers.0.mlp.experts.343.gate_proj",
1786
- "mtp.layers.0.mlp.experts.343.up_proj",
1787
- "mtp.layers.0.mlp.experts.344.down_proj",
1788
- "mtp.layers.0.mlp.experts.344.gate_proj",
1789
- "mtp.layers.0.mlp.experts.344.up_proj",
1790
- "mtp.layers.0.mlp.experts.345.down_proj",
1791
- "mtp.layers.0.mlp.experts.345.gate_proj",
1792
- "mtp.layers.0.mlp.experts.345.up_proj",
1793
- "mtp.layers.0.mlp.experts.346.down_proj",
1794
- "mtp.layers.0.mlp.experts.346.gate_proj",
1795
- "mtp.layers.0.mlp.experts.346.up_proj",
1796
- "mtp.layers.0.mlp.experts.347.down_proj",
1797
- "mtp.layers.0.mlp.experts.347.gate_proj",
1798
- "mtp.layers.0.mlp.experts.347.up_proj",
1799
- "mtp.layers.0.mlp.experts.348.down_proj",
1800
- "mtp.layers.0.mlp.experts.348.gate_proj",
1801
- "mtp.layers.0.mlp.experts.348.up_proj",
1802
- "mtp.layers.0.mlp.experts.349.down_proj",
1803
- "mtp.layers.0.mlp.experts.349.gate_proj",
1804
- "mtp.layers.0.mlp.experts.349.up_proj",
1805
- "mtp.layers.0.mlp.experts.35.down_proj",
1806
- "mtp.layers.0.mlp.experts.35.gate_proj",
1807
- "mtp.layers.0.mlp.experts.35.up_proj",
1808
- "mtp.layers.0.mlp.experts.350.down_proj",
1809
- "mtp.layers.0.mlp.experts.350.gate_proj",
1810
- "mtp.layers.0.mlp.experts.350.up_proj",
1811
- "mtp.layers.0.mlp.experts.351.down_proj",
1812
- "mtp.layers.0.mlp.experts.351.gate_proj",
1813
- "mtp.layers.0.mlp.experts.351.up_proj",
1814
- "mtp.layers.0.mlp.experts.352.down_proj",
1815
- "mtp.layers.0.mlp.experts.352.gate_proj",
1816
- "mtp.layers.0.mlp.experts.352.up_proj",
1817
- "mtp.layers.0.mlp.experts.353.down_proj",
1818
- "mtp.layers.0.mlp.experts.353.gate_proj",
1819
- "mtp.layers.0.mlp.experts.353.up_proj",
1820
- "mtp.layers.0.mlp.experts.354.down_proj",
1821
- "mtp.layers.0.mlp.experts.354.gate_proj",
1822
- "mtp.layers.0.mlp.experts.354.up_proj",
1823
- "mtp.layers.0.mlp.experts.355.down_proj",
1824
- "mtp.layers.0.mlp.experts.355.gate_proj",
1825
- "mtp.layers.0.mlp.experts.355.up_proj",
1826
- "mtp.layers.0.mlp.experts.356.down_proj",
1827
- "mtp.layers.0.mlp.experts.356.gate_proj",
1828
- "mtp.layers.0.mlp.experts.356.up_proj",
1829
- "mtp.layers.0.mlp.experts.357.down_proj",
1830
- "mtp.layers.0.mlp.experts.357.gate_proj",
1831
- "mtp.layers.0.mlp.experts.357.up_proj",
1832
- "mtp.layers.0.mlp.experts.358.down_proj",
1833
- "mtp.layers.0.mlp.experts.358.gate_proj",
1834
- "mtp.layers.0.mlp.experts.358.up_proj",
1835
- "mtp.layers.0.mlp.experts.359.down_proj",
1836
- "mtp.layers.0.mlp.experts.359.gate_proj",
1837
- "mtp.layers.0.mlp.experts.359.up_proj",
1838
- "mtp.layers.0.mlp.experts.36.down_proj",
1839
- "mtp.layers.0.mlp.experts.36.gate_proj",
1840
- "mtp.layers.0.mlp.experts.36.up_proj",
1841
- "mtp.layers.0.mlp.experts.360.down_proj",
1842
- "mtp.layers.0.mlp.experts.360.gate_proj",
1843
- "mtp.layers.0.mlp.experts.360.up_proj",
1844
- "mtp.layers.0.mlp.experts.361.down_proj",
1845
- "mtp.layers.0.mlp.experts.361.gate_proj",
1846
- "mtp.layers.0.mlp.experts.361.up_proj",
1847
- "mtp.layers.0.mlp.experts.362.down_proj",
1848
- "mtp.layers.0.mlp.experts.362.gate_proj",
1849
- "mtp.layers.0.mlp.experts.362.up_proj",
1850
- "mtp.layers.0.mlp.experts.363.down_proj",
1851
- "mtp.layers.0.mlp.experts.363.gate_proj",
1852
- "mtp.layers.0.mlp.experts.363.up_proj",
1853
- "mtp.layers.0.mlp.experts.364.down_proj",
1854
- "mtp.layers.0.mlp.experts.364.gate_proj",
1855
- "mtp.layers.0.mlp.experts.364.up_proj",
1856
- "mtp.layers.0.mlp.experts.365.down_proj",
1857
- "mtp.layers.0.mlp.experts.365.gate_proj",
1858
- "mtp.layers.0.mlp.experts.365.up_proj",
1859
- "mtp.layers.0.mlp.experts.366.down_proj",
1860
- "mtp.layers.0.mlp.experts.366.gate_proj",
1861
- "mtp.layers.0.mlp.experts.366.up_proj",
1862
- "mtp.layers.0.mlp.experts.367.down_proj",
1863
- "mtp.layers.0.mlp.experts.367.gate_proj",
1864
- "mtp.layers.0.mlp.experts.367.up_proj",
1865
- "mtp.layers.0.mlp.experts.368.down_proj",
1866
- "mtp.layers.0.mlp.experts.368.gate_proj",
1867
- "mtp.layers.0.mlp.experts.368.up_proj",
1868
- "mtp.layers.0.mlp.experts.369.down_proj",
1869
- "mtp.layers.0.mlp.experts.369.gate_proj",
1870
- "mtp.layers.0.mlp.experts.369.up_proj",
1871
- "mtp.layers.0.mlp.experts.37.down_proj",
1872
- "mtp.layers.0.mlp.experts.37.gate_proj",
1873
- "mtp.layers.0.mlp.experts.37.up_proj",
1874
- "mtp.layers.0.mlp.experts.370.down_proj",
1875
- "mtp.layers.0.mlp.experts.370.gate_proj",
1876
- "mtp.layers.0.mlp.experts.370.up_proj",
1877
- "mtp.layers.0.mlp.experts.371.down_proj",
1878
- "mtp.layers.0.mlp.experts.371.gate_proj",
1879
- "mtp.layers.0.mlp.experts.371.up_proj",
1880
- "mtp.layers.0.mlp.experts.372.down_proj",
1881
- "mtp.layers.0.mlp.experts.372.gate_proj",
1882
- "mtp.layers.0.mlp.experts.372.up_proj",
1883
- "mtp.layers.0.mlp.experts.373.down_proj",
1884
- "mtp.layers.0.mlp.experts.373.gate_proj",
1885
- "mtp.layers.0.mlp.experts.373.up_proj",
1886
- "mtp.layers.0.mlp.experts.374.down_proj",
1887
- "mtp.layers.0.mlp.experts.374.gate_proj",
1888
- "mtp.layers.0.mlp.experts.374.up_proj",
1889
- "mtp.layers.0.mlp.experts.375.down_proj",
1890
- "mtp.layers.0.mlp.experts.375.gate_proj",
1891
- "mtp.layers.0.mlp.experts.375.up_proj",
1892
- "mtp.layers.0.mlp.experts.376.down_proj",
1893
- "mtp.layers.0.mlp.experts.376.gate_proj",
1894
- "mtp.layers.0.mlp.experts.376.up_proj",
1895
- "mtp.layers.0.mlp.experts.377.down_proj",
1896
- "mtp.layers.0.mlp.experts.377.gate_proj",
1897
- "mtp.layers.0.mlp.experts.377.up_proj",
1898
- "mtp.layers.0.mlp.experts.378.down_proj",
1899
- "mtp.layers.0.mlp.experts.378.gate_proj",
1900
- "mtp.layers.0.mlp.experts.378.up_proj",
1901
- "mtp.layers.0.mlp.experts.379.down_proj",
1902
- "mtp.layers.0.mlp.experts.379.gate_proj",
1903
- "mtp.layers.0.mlp.experts.379.up_proj",
1904
- "mtp.layers.0.mlp.experts.38.down_proj",
1905
- "mtp.layers.0.mlp.experts.38.gate_proj",
1906
- "mtp.layers.0.mlp.experts.38.up_proj",
1907
- "mtp.layers.0.mlp.experts.380.down_proj",
1908
- "mtp.layers.0.mlp.experts.380.gate_proj",
1909
- "mtp.layers.0.mlp.experts.380.up_proj",
1910
- "mtp.layers.0.mlp.experts.381.down_proj",
1911
- "mtp.layers.0.mlp.experts.381.gate_proj",
1912
- "mtp.layers.0.mlp.experts.381.up_proj",
1913
- "mtp.layers.0.mlp.experts.382.down_proj",
1914
- "mtp.layers.0.mlp.experts.382.gate_proj",
1915
- "mtp.layers.0.mlp.experts.382.up_proj",
1916
- "mtp.layers.0.mlp.experts.383.down_proj",
1917
- "mtp.layers.0.mlp.experts.383.gate_proj",
1918
- "mtp.layers.0.mlp.experts.383.up_proj",
1919
- "mtp.layers.0.mlp.experts.384.down_proj",
1920
- "mtp.layers.0.mlp.experts.384.gate_proj",
1921
- "mtp.layers.0.mlp.experts.384.up_proj",
1922
- "mtp.layers.0.mlp.experts.385.down_proj",
1923
- "mtp.layers.0.mlp.experts.385.gate_proj",
1924
- "mtp.layers.0.mlp.experts.385.up_proj",
1925
- "mtp.layers.0.mlp.experts.386.down_proj",
1926
- "mtp.layers.0.mlp.experts.386.gate_proj",
1927
- "mtp.layers.0.mlp.experts.386.up_proj",
1928
- "mtp.layers.0.mlp.experts.387.down_proj",
1929
- "mtp.layers.0.mlp.experts.387.gate_proj",
1930
- "mtp.layers.0.mlp.experts.387.up_proj",
1931
- "mtp.layers.0.mlp.experts.388.down_proj",
1932
- "mtp.layers.0.mlp.experts.388.gate_proj",
1933
- "mtp.layers.0.mlp.experts.388.up_proj",
1934
- "mtp.layers.0.mlp.experts.389.down_proj",
1935
- "mtp.layers.0.mlp.experts.389.gate_proj",
1936
- "mtp.layers.0.mlp.experts.389.up_proj",
1937
- "mtp.layers.0.mlp.experts.39.down_proj",
1938
- "mtp.layers.0.mlp.experts.39.gate_proj",
1939
- "mtp.layers.0.mlp.experts.39.up_proj",
1940
- "mtp.layers.0.mlp.experts.390.down_proj",
1941
- "mtp.layers.0.mlp.experts.390.gate_proj",
1942
- "mtp.layers.0.mlp.experts.390.up_proj",
1943
- "mtp.layers.0.mlp.experts.391.down_proj",
1944
- "mtp.layers.0.mlp.experts.391.gate_proj",
1945
- "mtp.layers.0.mlp.experts.391.up_proj",
1946
- "mtp.layers.0.mlp.experts.392.down_proj",
1947
- "mtp.layers.0.mlp.experts.392.gate_proj",
1948
- "mtp.layers.0.mlp.experts.392.up_proj",
1949
- "mtp.layers.0.mlp.experts.393.down_proj",
1950
- "mtp.layers.0.mlp.experts.393.gate_proj",
1951
- "mtp.layers.0.mlp.experts.393.up_proj",
1952
- "mtp.layers.0.mlp.experts.394.down_proj",
1953
- "mtp.layers.0.mlp.experts.394.gate_proj",
1954
- "mtp.layers.0.mlp.experts.394.up_proj",
1955
- "mtp.layers.0.mlp.experts.395.down_proj",
1956
- "mtp.layers.0.mlp.experts.395.gate_proj",
1957
- "mtp.layers.0.mlp.experts.395.up_proj",
1958
- "mtp.layers.0.mlp.experts.396.down_proj",
1959
- "mtp.layers.0.mlp.experts.396.gate_proj",
1960
- "mtp.layers.0.mlp.experts.396.up_proj",
1961
- "mtp.layers.0.mlp.experts.397.down_proj",
1962
- "mtp.layers.0.mlp.experts.397.gate_proj",
1963
- "mtp.layers.0.mlp.experts.397.up_proj",
1964
- "mtp.layers.0.mlp.experts.398.down_proj",
1965
- "mtp.layers.0.mlp.experts.398.gate_proj",
1966
- "mtp.layers.0.mlp.experts.398.up_proj",
1967
- "mtp.layers.0.mlp.experts.399.down_proj",
1968
- "mtp.layers.0.mlp.experts.399.gate_proj",
1969
- "mtp.layers.0.mlp.experts.399.up_proj",
1970
- "mtp.layers.0.mlp.experts.4.down_proj",
1971
- "mtp.layers.0.mlp.experts.4.gate_proj",
1972
- "mtp.layers.0.mlp.experts.4.up_proj",
1973
- "mtp.layers.0.mlp.experts.40.down_proj",
1974
- "mtp.layers.0.mlp.experts.40.gate_proj",
1975
- "mtp.layers.0.mlp.experts.40.up_proj",
1976
- "mtp.layers.0.mlp.experts.400.down_proj",
1977
- "mtp.layers.0.mlp.experts.400.gate_proj",
1978
- "mtp.layers.0.mlp.experts.400.up_proj",
1979
- "mtp.layers.0.mlp.experts.401.down_proj",
1980
- "mtp.layers.0.mlp.experts.401.gate_proj",
1981
- "mtp.layers.0.mlp.experts.401.up_proj",
1982
- "mtp.layers.0.mlp.experts.402.down_proj",
1983
- "mtp.layers.0.mlp.experts.402.gate_proj",
1984
- "mtp.layers.0.mlp.experts.402.up_proj",
1985
- "mtp.layers.0.mlp.experts.403.down_proj",
1986
- "mtp.layers.0.mlp.experts.403.gate_proj",
1987
- "mtp.layers.0.mlp.experts.403.up_proj",
1988
- "mtp.layers.0.mlp.experts.404.down_proj",
1989
- "mtp.layers.0.mlp.experts.404.gate_proj",
1990
- "mtp.layers.0.mlp.experts.404.up_proj",
1991
- "mtp.layers.0.mlp.experts.405.down_proj",
1992
- "mtp.layers.0.mlp.experts.405.gate_proj",
1993
- "mtp.layers.0.mlp.experts.405.up_proj",
1994
- "mtp.layers.0.mlp.experts.406.down_proj",
1995
- "mtp.layers.0.mlp.experts.406.gate_proj",
1996
- "mtp.layers.0.mlp.experts.406.up_proj",
1997
- "mtp.layers.0.mlp.experts.407.down_proj",
1998
- "mtp.layers.0.mlp.experts.407.gate_proj",
1999
- "mtp.layers.0.mlp.experts.407.up_proj",
2000
- "mtp.layers.0.mlp.experts.408.down_proj",
2001
- "mtp.layers.0.mlp.experts.408.gate_proj",
2002
- "mtp.layers.0.mlp.experts.408.up_proj",
2003
- "mtp.layers.0.mlp.experts.409.down_proj",
2004
- "mtp.layers.0.mlp.experts.409.gate_proj",
2005
- "mtp.layers.0.mlp.experts.409.up_proj",
2006
- "mtp.layers.0.mlp.experts.41.down_proj",
2007
- "mtp.layers.0.mlp.experts.41.gate_proj",
2008
- "mtp.layers.0.mlp.experts.41.up_proj",
2009
- "mtp.layers.0.mlp.experts.410.down_proj",
2010
- "mtp.layers.0.mlp.experts.410.gate_proj",
2011
- "mtp.layers.0.mlp.experts.410.up_proj",
2012
- "mtp.layers.0.mlp.experts.411.down_proj",
2013
- "mtp.layers.0.mlp.experts.411.gate_proj",
2014
- "mtp.layers.0.mlp.experts.411.up_proj",
2015
- "mtp.layers.0.mlp.experts.412.down_proj",
2016
- "mtp.layers.0.mlp.experts.412.gate_proj",
2017
- "mtp.layers.0.mlp.experts.412.up_proj",
2018
- "mtp.layers.0.mlp.experts.413.down_proj",
2019
- "mtp.layers.0.mlp.experts.413.gate_proj",
2020
- "mtp.layers.0.mlp.experts.413.up_proj",
2021
- "mtp.layers.0.mlp.experts.414.down_proj",
2022
- "mtp.layers.0.mlp.experts.414.gate_proj",
2023
- "mtp.layers.0.mlp.experts.414.up_proj",
2024
- "mtp.layers.0.mlp.experts.415.down_proj",
2025
- "mtp.layers.0.mlp.experts.415.gate_proj",
2026
- "mtp.layers.0.mlp.experts.415.up_proj",
2027
- "mtp.layers.0.mlp.experts.416.down_proj",
2028
- "mtp.layers.0.mlp.experts.416.gate_proj",
2029
- "mtp.layers.0.mlp.experts.416.up_proj",
2030
- "mtp.layers.0.mlp.experts.417.down_proj",
2031
- "mtp.layers.0.mlp.experts.417.gate_proj",
2032
- "mtp.layers.0.mlp.experts.417.up_proj",
2033
- "mtp.layers.0.mlp.experts.418.down_proj",
2034
- "mtp.layers.0.mlp.experts.418.gate_proj",
2035
- "mtp.layers.0.mlp.experts.418.up_proj",
2036
- "mtp.layers.0.mlp.experts.419.down_proj",
2037
- "mtp.layers.0.mlp.experts.419.gate_proj",
2038
- "mtp.layers.0.mlp.experts.419.up_proj",
2039
- "mtp.layers.0.mlp.experts.42.down_proj",
2040
- "mtp.layers.0.mlp.experts.42.gate_proj",
2041
- "mtp.layers.0.mlp.experts.42.up_proj",
2042
- "mtp.layers.0.mlp.experts.420.down_proj",
2043
- "mtp.layers.0.mlp.experts.420.gate_proj",
2044
- "mtp.layers.0.mlp.experts.420.up_proj",
2045
- "mtp.layers.0.mlp.experts.421.down_proj",
2046
- "mtp.layers.0.mlp.experts.421.gate_proj",
2047
- "mtp.layers.0.mlp.experts.421.up_proj",
2048
- "mtp.layers.0.mlp.experts.422.down_proj",
2049
- "mtp.layers.0.mlp.experts.422.gate_proj",
2050
- "mtp.layers.0.mlp.experts.422.up_proj",
2051
- "mtp.layers.0.mlp.experts.423.down_proj",
2052
- "mtp.layers.0.mlp.experts.423.gate_proj",
2053
- "mtp.layers.0.mlp.experts.423.up_proj",
2054
- "mtp.layers.0.mlp.experts.424.down_proj",
2055
- "mtp.layers.0.mlp.experts.424.gate_proj",
2056
- "mtp.layers.0.mlp.experts.424.up_proj",
2057
- "mtp.layers.0.mlp.experts.425.down_proj",
2058
- "mtp.layers.0.mlp.experts.425.gate_proj",
2059
- "mtp.layers.0.mlp.experts.425.up_proj",
2060
- "mtp.layers.0.mlp.experts.426.down_proj",
2061
- "mtp.layers.0.mlp.experts.426.gate_proj",
2062
- "mtp.layers.0.mlp.experts.426.up_proj",
2063
- "mtp.layers.0.mlp.experts.427.down_proj",
2064
- "mtp.layers.0.mlp.experts.427.gate_proj",
2065
- "mtp.layers.0.mlp.experts.427.up_proj",
2066
- "mtp.layers.0.mlp.experts.428.down_proj",
2067
- "mtp.layers.0.mlp.experts.428.gate_proj",
2068
- "mtp.layers.0.mlp.experts.428.up_proj",
2069
- "mtp.layers.0.mlp.experts.429.down_proj",
2070
- "mtp.layers.0.mlp.experts.429.gate_proj",
2071
- "mtp.layers.0.mlp.experts.429.up_proj",
2072
- "mtp.layers.0.mlp.experts.43.down_proj",
2073
- "mtp.layers.0.mlp.experts.43.gate_proj",
2074
- "mtp.layers.0.mlp.experts.43.up_proj",
2075
- "mtp.layers.0.mlp.experts.430.down_proj",
2076
- "mtp.layers.0.mlp.experts.430.gate_proj",
2077
- "mtp.layers.0.mlp.experts.430.up_proj",
2078
- "mtp.layers.0.mlp.experts.431.down_proj",
2079
- "mtp.layers.0.mlp.experts.431.gate_proj",
2080
- "mtp.layers.0.mlp.experts.431.up_proj",
2081
- "mtp.layers.0.mlp.experts.432.down_proj",
2082
- "mtp.layers.0.mlp.experts.432.gate_proj",
2083
- "mtp.layers.0.mlp.experts.432.up_proj",
2084
- "mtp.layers.0.mlp.experts.433.down_proj",
2085
- "mtp.layers.0.mlp.experts.433.gate_proj",
2086
- "mtp.layers.0.mlp.experts.433.up_proj",
2087
- "mtp.layers.0.mlp.experts.434.down_proj",
2088
- "mtp.layers.0.mlp.experts.434.gate_proj",
2089
- "mtp.layers.0.mlp.experts.434.up_proj",
2090
- "mtp.layers.0.mlp.experts.435.down_proj",
2091
- "mtp.layers.0.mlp.experts.435.gate_proj",
2092
- "mtp.layers.0.mlp.experts.435.up_proj",
2093
- "mtp.layers.0.mlp.experts.436.down_proj",
2094
- "mtp.layers.0.mlp.experts.436.gate_proj",
2095
- "mtp.layers.0.mlp.experts.436.up_proj",
2096
- "mtp.layers.0.mlp.experts.437.down_proj",
2097
- "mtp.layers.0.mlp.experts.437.gate_proj",
2098
- "mtp.layers.0.mlp.experts.437.up_proj",
2099
- "mtp.layers.0.mlp.experts.438.down_proj",
2100
- "mtp.layers.0.mlp.experts.438.gate_proj",
2101
- "mtp.layers.0.mlp.experts.438.up_proj",
2102
- "mtp.layers.0.mlp.experts.439.down_proj",
2103
- "mtp.layers.0.mlp.experts.439.gate_proj",
2104
- "mtp.layers.0.mlp.experts.439.up_proj",
2105
- "mtp.layers.0.mlp.experts.44.down_proj",
2106
- "mtp.layers.0.mlp.experts.44.gate_proj",
2107
- "mtp.layers.0.mlp.experts.44.up_proj",
2108
- "mtp.layers.0.mlp.experts.440.down_proj",
2109
- "mtp.layers.0.mlp.experts.440.gate_proj",
2110
- "mtp.layers.0.mlp.experts.440.up_proj",
2111
- "mtp.layers.0.mlp.experts.441.down_proj",
2112
- "mtp.layers.0.mlp.experts.441.gate_proj",
2113
- "mtp.layers.0.mlp.experts.441.up_proj",
2114
- "mtp.layers.0.mlp.experts.442.down_proj",
2115
- "mtp.layers.0.mlp.experts.442.gate_proj",
2116
- "mtp.layers.0.mlp.experts.442.up_proj",
2117
- "mtp.layers.0.mlp.experts.443.down_proj",
2118
- "mtp.layers.0.mlp.experts.443.gate_proj",
2119
- "mtp.layers.0.mlp.experts.443.up_proj",
2120
- "mtp.layers.0.mlp.experts.444.down_proj",
2121
- "mtp.layers.0.mlp.experts.444.gate_proj",
2122
- "mtp.layers.0.mlp.experts.444.up_proj",
2123
- "mtp.layers.0.mlp.experts.445.down_proj",
2124
- "mtp.layers.0.mlp.experts.445.gate_proj",
2125
- "mtp.layers.0.mlp.experts.445.up_proj",
2126
- "mtp.layers.0.mlp.experts.446.down_proj",
2127
- "mtp.layers.0.mlp.experts.446.gate_proj",
2128
- "mtp.layers.0.mlp.experts.446.up_proj",
2129
- "mtp.layers.0.mlp.experts.447.down_proj",
2130
- "mtp.layers.0.mlp.experts.447.gate_proj",
2131
- "mtp.layers.0.mlp.experts.447.up_proj",
2132
- "mtp.layers.0.mlp.experts.448.down_proj",
2133
- "mtp.layers.0.mlp.experts.448.gate_proj",
2134
- "mtp.layers.0.mlp.experts.448.up_proj",
2135
- "mtp.layers.0.mlp.experts.449.down_proj",
2136
- "mtp.layers.0.mlp.experts.449.gate_proj",
2137
- "mtp.layers.0.mlp.experts.449.up_proj",
2138
- "mtp.layers.0.mlp.experts.45.down_proj",
2139
- "mtp.layers.0.mlp.experts.45.gate_proj",
2140
- "mtp.layers.0.mlp.experts.45.up_proj",
2141
- "mtp.layers.0.mlp.experts.450.down_proj",
2142
- "mtp.layers.0.mlp.experts.450.gate_proj",
2143
- "mtp.layers.0.mlp.experts.450.up_proj",
2144
- "mtp.layers.0.mlp.experts.451.down_proj",
2145
- "mtp.layers.0.mlp.experts.451.gate_proj",
2146
- "mtp.layers.0.mlp.experts.451.up_proj",
2147
- "mtp.layers.0.mlp.experts.452.down_proj",
2148
- "mtp.layers.0.mlp.experts.452.gate_proj",
2149
- "mtp.layers.0.mlp.experts.452.up_proj",
2150
- "mtp.layers.0.mlp.experts.453.down_proj",
2151
- "mtp.layers.0.mlp.experts.453.gate_proj",
2152
- "mtp.layers.0.mlp.experts.453.up_proj",
2153
- "mtp.layers.0.mlp.experts.454.down_proj",
2154
- "mtp.layers.0.mlp.experts.454.gate_proj",
2155
- "mtp.layers.0.mlp.experts.454.up_proj",
2156
- "mtp.layers.0.mlp.experts.455.down_proj",
2157
- "mtp.layers.0.mlp.experts.455.gate_proj",
2158
- "mtp.layers.0.mlp.experts.455.up_proj",
2159
- "mtp.layers.0.mlp.experts.456.down_proj",
2160
- "mtp.layers.0.mlp.experts.456.gate_proj",
2161
- "mtp.layers.0.mlp.experts.456.up_proj",
2162
- "mtp.layers.0.mlp.experts.457.down_proj",
2163
- "mtp.layers.0.mlp.experts.457.gate_proj",
2164
- "mtp.layers.0.mlp.experts.457.up_proj",
2165
- "mtp.layers.0.mlp.experts.458.down_proj",
2166
- "mtp.layers.0.mlp.experts.458.gate_proj",
2167
- "mtp.layers.0.mlp.experts.458.up_proj",
2168
- "mtp.layers.0.mlp.experts.459.down_proj",
2169
- "mtp.layers.0.mlp.experts.459.gate_proj",
2170
- "mtp.layers.0.mlp.experts.459.up_proj",
2171
- "mtp.layers.0.mlp.experts.46.down_proj",
2172
- "mtp.layers.0.mlp.experts.46.gate_proj",
2173
- "mtp.layers.0.mlp.experts.46.up_proj",
2174
- "mtp.layers.0.mlp.experts.460.down_proj",
2175
- "mtp.layers.0.mlp.experts.460.gate_proj",
2176
- "mtp.layers.0.mlp.experts.460.up_proj",
2177
- "mtp.layers.0.mlp.experts.461.down_proj",
2178
- "mtp.layers.0.mlp.experts.461.gate_proj",
2179
- "mtp.layers.0.mlp.experts.461.up_proj",
2180
- "mtp.layers.0.mlp.experts.462.down_proj",
2181
- "mtp.layers.0.mlp.experts.462.gate_proj",
2182
- "mtp.layers.0.mlp.experts.462.up_proj",
2183
- "mtp.layers.0.mlp.experts.463.down_proj",
2184
- "mtp.layers.0.mlp.experts.463.gate_proj",
2185
- "mtp.layers.0.mlp.experts.463.up_proj",
2186
- "mtp.layers.0.mlp.experts.464.down_proj",
2187
- "mtp.layers.0.mlp.experts.464.gate_proj",
2188
- "mtp.layers.0.mlp.experts.464.up_proj",
2189
- "mtp.layers.0.mlp.experts.465.down_proj",
2190
- "mtp.layers.0.mlp.experts.465.gate_proj",
2191
- "mtp.layers.0.mlp.experts.465.up_proj",
2192
- "mtp.layers.0.mlp.experts.466.down_proj",
2193
- "mtp.layers.0.mlp.experts.466.gate_proj",
2194
- "mtp.layers.0.mlp.experts.466.up_proj",
2195
- "mtp.layers.0.mlp.experts.467.down_proj",
2196
- "mtp.layers.0.mlp.experts.467.gate_proj",
2197
- "mtp.layers.0.mlp.experts.467.up_proj",
2198
- "mtp.layers.0.mlp.experts.468.down_proj",
2199
- "mtp.layers.0.mlp.experts.468.gate_proj",
2200
- "mtp.layers.0.mlp.experts.468.up_proj",
2201
- "mtp.layers.0.mlp.experts.469.down_proj",
2202
- "mtp.layers.0.mlp.experts.469.gate_proj",
2203
- "mtp.layers.0.mlp.experts.469.up_proj",
2204
- "mtp.layers.0.mlp.experts.47.down_proj",
2205
- "mtp.layers.0.mlp.experts.47.gate_proj",
2206
- "mtp.layers.0.mlp.experts.47.up_proj",
2207
- "mtp.layers.0.mlp.experts.470.down_proj",
2208
- "mtp.layers.0.mlp.experts.470.gate_proj",
2209
- "mtp.layers.0.mlp.experts.470.up_proj",
2210
- "mtp.layers.0.mlp.experts.471.down_proj",
2211
- "mtp.layers.0.mlp.experts.471.gate_proj",
2212
- "mtp.layers.0.mlp.experts.471.up_proj",
2213
- "mtp.layers.0.mlp.experts.472.down_proj",
2214
- "mtp.layers.0.mlp.experts.472.gate_proj",
2215
- "mtp.layers.0.mlp.experts.472.up_proj",
2216
- "mtp.layers.0.mlp.experts.473.down_proj",
2217
- "mtp.layers.0.mlp.experts.473.gate_proj",
2218
- "mtp.layers.0.mlp.experts.473.up_proj",
2219
- "mtp.layers.0.mlp.experts.474.down_proj",
2220
- "mtp.layers.0.mlp.experts.474.gate_proj",
2221
- "mtp.layers.0.mlp.experts.474.up_proj",
2222
- "mtp.layers.0.mlp.experts.475.down_proj",
2223
- "mtp.layers.0.mlp.experts.475.gate_proj",
2224
- "mtp.layers.0.mlp.experts.475.up_proj",
2225
- "mtp.layers.0.mlp.experts.476.down_proj",
2226
- "mtp.layers.0.mlp.experts.476.gate_proj",
2227
- "mtp.layers.0.mlp.experts.476.up_proj",
2228
- "mtp.layers.0.mlp.experts.477.down_proj",
2229
- "mtp.layers.0.mlp.experts.477.gate_proj",
2230
- "mtp.layers.0.mlp.experts.477.up_proj",
2231
- "mtp.layers.0.mlp.experts.478.down_proj",
2232
- "mtp.layers.0.mlp.experts.478.gate_proj",
2233
- "mtp.layers.0.mlp.experts.478.up_proj",
2234
- "mtp.layers.0.mlp.experts.479.down_proj",
2235
- "mtp.layers.0.mlp.experts.479.gate_proj",
2236
- "mtp.layers.0.mlp.experts.479.up_proj",
2237
- "mtp.layers.0.mlp.experts.48.down_proj",
2238
- "mtp.layers.0.mlp.experts.48.gate_proj",
2239
- "mtp.layers.0.mlp.experts.48.up_proj",
2240
- "mtp.layers.0.mlp.experts.480.down_proj",
2241
- "mtp.layers.0.mlp.experts.480.gate_proj",
2242
- "mtp.layers.0.mlp.experts.480.up_proj",
2243
- "mtp.layers.0.mlp.experts.481.down_proj",
2244
- "mtp.layers.0.mlp.experts.481.gate_proj",
2245
- "mtp.layers.0.mlp.experts.481.up_proj",
2246
- "mtp.layers.0.mlp.experts.482.down_proj",
2247
- "mtp.layers.0.mlp.experts.482.gate_proj",
2248
- "mtp.layers.0.mlp.experts.482.up_proj",
2249
- "mtp.layers.0.mlp.experts.483.down_proj",
2250
- "mtp.layers.0.mlp.experts.483.gate_proj",
2251
- "mtp.layers.0.mlp.experts.483.up_proj",
2252
- "mtp.layers.0.mlp.experts.484.down_proj",
2253
- "mtp.layers.0.mlp.experts.484.gate_proj",
2254
- "mtp.layers.0.mlp.experts.484.up_proj",
2255
- "mtp.layers.0.mlp.experts.485.down_proj",
2256
- "mtp.layers.0.mlp.experts.485.gate_proj",
2257
- "mtp.layers.0.mlp.experts.485.up_proj",
2258
- "mtp.layers.0.mlp.experts.486.down_proj",
2259
- "mtp.layers.0.mlp.experts.486.gate_proj",
2260
- "mtp.layers.0.mlp.experts.486.up_proj",
2261
- "mtp.layers.0.mlp.experts.487.down_proj",
2262
- "mtp.layers.0.mlp.experts.487.gate_proj",
2263
- "mtp.layers.0.mlp.experts.487.up_proj",
2264
- "mtp.layers.0.mlp.experts.488.down_proj",
2265
- "mtp.layers.0.mlp.experts.488.gate_proj",
2266
- "mtp.layers.0.mlp.experts.488.up_proj",
2267
- "mtp.layers.0.mlp.experts.489.down_proj",
2268
- "mtp.layers.0.mlp.experts.489.gate_proj",
2269
- "mtp.layers.0.mlp.experts.489.up_proj",
2270
- "mtp.layers.0.mlp.experts.49.down_proj",
2271
- "mtp.layers.0.mlp.experts.49.gate_proj",
2272
- "mtp.layers.0.mlp.experts.49.up_proj",
2273
- "mtp.layers.0.mlp.experts.490.down_proj",
2274
- "mtp.layers.0.mlp.experts.490.gate_proj",
2275
- "mtp.layers.0.mlp.experts.490.up_proj",
2276
- "mtp.layers.0.mlp.experts.491.down_proj",
2277
- "mtp.layers.0.mlp.experts.491.gate_proj",
2278
- "mtp.layers.0.mlp.experts.491.up_proj",
2279
- "mtp.layers.0.mlp.experts.492.down_proj",
2280
- "mtp.layers.0.mlp.experts.492.gate_proj",
2281
- "mtp.layers.0.mlp.experts.492.up_proj",
2282
- "mtp.layers.0.mlp.experts.493.down_proj",
2283
- "mtp.layers.0.mlp.experts.493.gate_proj",
2284
- "mtp.layers.0.mlp.experts.493.up_proj",
2285
- "mtp.layers.0.mlp.experts.494.down_proj",
2286
- "mtp.layers.0.mlp.experts.494.gate_proj",
2287
- "mtp.layers.0.mlp.experts.494.up_proj",
2288
- "mtp.layers.0.mlp.experts.495.down_proj",
2289
- "mtp.layers.0.mlp.experts.495.gate_proj",
2290
- "mtp.layers.0.mlp.experts.495.up_proj",
2291
- "mtp.layers.0.mlp.experts.496.down_proj",
2292
- "mtp.layers.0.mlp.experts.496.gate_proj",
2293
- "mtp.layers.0.mlp.experts.496.up_proj",
2294
- "mtp.layers.0.mlp.experts.497.down_proj",
2295
- "mtp.layers.0.mlp.experts.497.gate_proj",
2296
- "mtp.layers.0.mlp.experts.497.up_proj",
2297
- "mtp.layers.0.mlp.experts.498.down_proj",
2298
- "mtp.layers.0.mlp.experts.498.gate_proj",
2299
- "mtp.layers.0.mlp.experts.498.up_proj",
2300
- "mtp.layers.0.mlp.experts.499.down_proj",
2301
- "mtp.layers.0.mlp.experts.499.gate_proj",
2302
- "mtp.layers.0.mlp.experts.499.up_proj",
2303
- "mtp.layers.0.mlp.experts.5.down_proj",
2304
- "mtp.layers.0.mlp.experts.5.gate_proj",
2305
- "mtp.layers.0.mlp.experts.5.up_proj",
2306
- "mtp.layers.0.mlp.experts.50.down_proj",
2307
- "mtp.layers.0.mlp.experts.50.gate_proj",
2308
- "mtp.layers.0.mlp.experts.50.up_proj",
2309
- "mtp.layers.0.mlp.experts.500.down_proj",
2310
- "mtp.layers.0.mlp.experts.500.gate_proj",
2311
- "mtp.layers.0.mlp.experts.500.up_proj",
2312
- "mtp.layers.0.mlp.experts.501.down_proj",
2313
- "mtp.layers.0.mlp.experts.501.gate_proj",
2314
- "mtp.layers.0.mlp.experts.501.up_proj",
2315
- "mtp.layers.0.mlp.experts.502.down_proj",
2316
- "mtp.layers.0.mlp.experts.502.gate_proj",
2317
- "mtp.layers.0.mlp.experts.502.up_proj",
2318
- "mtp.layers.0.mlp.experts.503.down_proj",
2319
- "mtp.layers.0.mlp.experts.503.gate_proj",
2320
- "mtp.layers.0.mlp.experts.503.up_proj",
2321
- "mtp.layers.0.mlp.experts.504.down_proj",
2322
- "mtp.layers.0.mlp.experts.504.gate_proj",
2323
- "mtp.layers.0.mlp.experts.504.up_proj",
2324
- "mtp.layers.0.mlp.experts.505.down_proj",
2325
- "mtp.layers.0.mlp.experts.505.gate_proj",
2326
- "mtp.layers.0.mlp.experts.505.up_proj",
2327
- "mtp.layers.0.mlp.experts.506.down_proj",
2328
- "mtp.layers.0.mlp.experts.506.gate_proj",
2329
- "mtp.layers.0.mlp.experts.506.up_proj",
2330
- "mtp.layers.0.mlp.experts.507.down_proj",
2331
- "mtp.layers.0.mlp.experts.507.gate_proj",
2332
- "mtp.layers.0.mlp.experts.507.up_proj",
2333
- "mtp.layers.0.mlp.experts.508.down_proj",
2334
- "mtp.layers.0.mlp.experts.508.gate_proj",
2335
- "mtp.layers.0.mlp.experts.508.up_proj",
2336
- "mtp.layers.0.mlp.experts.509.down_proj",
2337
- "mtp.layers.0.mlp.experts.509.gate_proj",
2338
- "mtp.layers.0.mlp.experts.509.up_proj",
2339
- "mtp.layers.0.mlp.experts.51.down_proj",
2340
- "mtp.layers.0.mlp.experts.51.gate_proj",
2341
- "mtp.layers.0.mlp.experts.51.up_proj",
2342
- "mtp.layers.0.mlp.experts.510.down_proj",
2343
- "mtp.layers.0.mlp.experts.510.gate_proj",
2344
- "mtp.layers.0.mlp.experts.510.up_proj",
2345
- "mtp.layers.0.mlp.experts.511.down_proj",
2346
- "mtp.layers.0.mlp.experts.511.gate_proj",
2347
- "mtp.layers.0.mlp.experts.511.up_proj",
2348
- "mtp.layers.0.mlp.experts.52.down_proj",
2349
- "mtp.layers.0.mlp.experts.52.gate_proj",
2350
- "mtp.layers.0.mlp.experts.52.up_proj",
2351
- "mtp.layers.0.mlp.experts.53.down_proj",
2352
- "mtp.layers.0.mlp.experts.53.gate_proj",
2353
- "mtp.layers.0.mlp.experts.53.up_proj",
2354
- "mtp.layers.0.mlp.experts.54.down_proj",
2355
- "mtp.layers.0.mlp.experts.54.gate_proj",
2356
- "mtp.layers.0.mlp.experts.54.up_proj",
2357
- "mtp.layers.0.mlp.experts.55.down_proj",
2358
- "mtp.layers.0.mlp.experts.55.gate_proj",
2359
- "mtp.layers.0.mlp.experts.55.up_proj",
2360
- "mtp.layers.0.mlp.experts.56.down_proj",
2361
- "mtp.layers.0.mlp.experts.56.gate_proj",
2362
- "mtp.layers.0.mlp.experts.56.up_proj",
2363
- "mtp.layers.0.mlp.experts.57.down_proj",
2364
- "mtp.layers.0.mlp.experts.57.gate_proj",
2365
- "mtp.layers.0.mlp.experts.57.up_proj",
2366
- "mtp.layers.0.mlp.experts.58.down_proj",
2367
- "mtp.layers.0.mlp.experts.58.gate_proj",
2368
- "mtp.layers.0.mlp.experts.58.up_proj",
2369
- "mtp.layers.0.mlp.experts.59.down_proj",
2370
- "mtp.layers.0.mlp.experts.59.gate_proj",
2371
- "mtp.layers.0.mlp.experts.59.up_proj",
2372
- "mtp.layers.0.mlp.experts.6.down_proj",
2373
- "mtp.layers.0.mlp.experts.6.gate_proj",
2374
- "mtp.layers.0.mlp.experts.6.up_proj",
2375
- "mtp.layers.0.mlp.experts.60.down_proj",
2376
- "mtp.layers.0.mlp.experts.60.gate_proj",
2377
- "mtp.layers.0.mlp.experts.60.up_proj",
2378
- "mtp.layers.0.mlp.experts.61.down_proj",
2379
- "mtp.layers.0.mlp.experts.61.gate_proj",
2380
- "mtp.layers.0.mlp.experts.61.up_proj",
2381
- "mtp.layers.0.mlp.experts.62.down_proj",
2382
- "mtp.layers.0.mlp.experts.62.gate_proj",
2383
- "mtp.layers.0.mlp.experts.62.up_proj",
2384
- "mtp.layers.0.mlp.experts.63.down_proj",
2385
- "mtp.layers.0.mlp.experts.63.gate_proj",
2386
- "mtp.layers.0.mlp.experts.63.up_proj",
2387
- "mtp.layers.0.mlp.experts.64.down_proj",
2388
- "mtp.layers.0.mlp.experts.64.gate_proj",
2389
- "mtp.layers.0.mlp.experts.64.up_proj",
2390
- "mtp.layers.0.mlp.experts.65.down_proj",
2391
- "mtp.layers.0.mlp.experts.65.gate_proj",
2392
- "mtp.layers.0.mlp.experts.65.up_proj",
2393
- "mtp.layers.0.mlp.experts.66.down_proj",
2394
- "mtp.layers.0.mlp.experts.66.gate_proj",
2395
- "mtp.layers.0.mlp.experts.66.up_proj",
2396
- "mtp.layers.0.mlp.experts.67.down_proj",
2397
- "mtp.layers.0.mlp.experts.67.gate_proj",
2398
- "mtp.layers.0.mlp.experts.67.up_proj",
2399
- "mtp.layers.0.mlp.experts.68.down_proj",
2400
- "mtp.layers.0.mlp.experts.68.gate_proj",
2401
- "mtp.layers.0.mlp.experts.68.up_proj",
2402
- "mtp.layers.0.mlp.experts.69.down_proj",
2403
- "mtp.layers.0.mlp.experts.69.gate_proj",
2404
- "mtp.layers.0.mlp.experts.69.up_proj",
2405
- "mtp.layers.0.mlp.experts.7.down_proj",
2406
- "mtp.layers.0.mlp.experts.7.gate_proj",
2407
- "mtp.layers.0.mlp.experts.7.up_proj",
2408
- "mtp.layers.0.mlp.experts.70.down_proj",
2409
- "mtp.layers.0.mlp.experts.70.gate_proj",
2410
- "mtp.layers.0.mlp.experts.70.up_proj",
2411
- "mtp.layers.0.mlp.experts.71.down_proj",
2412
- "mtp.layers.0.mlp.experts.71.gate_proj",
2413
- "mtp.layers.0.mlp.experts.71.up_proj",
2414
- "mtp.layers.0.mlp.experts.72.down_proj",
2415
- "mtp.layers.0.mlp.experts.72.gate_proj",
2416
- "mtp.layers.0.mlp.experts.72.up_proj",
2417
- "mtp.layers.0.mlp.experts.73.down_proj",
2418
- "mtp.layers.0.mlp.experts.73.gate_proj",
2419
- "mtp.layers.0.mlp.experts.73.up_proj",
2420
- "mtp.layers.0.mlp.experts.74.down_proj",
2421
- "mtp.layers.0.mlp.experts.74.gate_proj",
2422
- "mtp.layers.0.mlp.experts.74.up_proj",
2423
- "mtp.layers.0.mlp.experts.75.down_proj",
2424
- "mtp.layers.0.mlp.experts.75.gate_proj",
2425
- "mtp.layers.0.mlp.experts.75.up_proj",
2426
- "mtp.layers.0.mlp.experts.76.down_proj",
2427
- "mtp.layers.0.mlp.experts.76.gate_proj",
2428
- "mtp.layers.0.mlp.experts.76.up_proj",
2429
- "mtp.layers.0.mlp.experts.77.down_proj",
2430
- "mtp.layers.0.mlp.experts.77.gate_proj",
2431
- "mtp.layers.0.mlp.experts.77.up_proj",
2432
- "mtp.layers.0.mlp.experts.78.down_proj",
2433
- "mtp.layers.0.mlp.experts.78.gate_proj",
2434
- "mtp.layers.0.mlp.experts.78.up_proj",
2435
- "mtp.layers.0.mlp.experts.79.down_proj",
2436
- "mtp.layers.0.mlp.experts.79.gate_proj",
2437
- "mtp.layers.0.mlp.experts.79.up_proj",
2438
- "mtp.layers.0.mlp.experts.8.down_proj",
2439
- "mtp.layers.0.mlp.experts.8.gate_proj",
2440
- "mtp.layers.0.mlp.experts.8.up_proj",
2441
- "mtp.layers.0.mlp.experts.80.down_proj",
2442
- "mtp.layers.0.mlp.experts.80.gate_proj",
2443
- "mtp.layers.0.mlp.experts.80.up_proj",
2444
- "mtp.layers.0.mlp.experts.81.down_proj",
2445
- "mtp.layers.0.mlp.experts.81.gate_proj",
2446
- "mtp.layers.0.mlp.experts.81.up_proj",
2447
- "mtp.layers.0.mlp.experts.82.down_proj",
2448
- "mtp.layers.0.mlp.experts.82.gate_proj",
2449
- "mtp.layers.0.mlp.experts.82.up_proj",
2450
- "mtp.layers.0.mlp.experts.83.down_proj",
2451
- "mtp.layers.0.mlp.experts.83.gate_proj",
2452
- "mtp.layers.0.mlp.experts.83.up_proj",
2453
- "mtp.layers.0.mlp.experts.84.down_proj",
2454
- "mtp.layers.0.mlp.experts.84.gate_proj",
2455
- "mtp.layers.0.mlp.experts.84.up_proj",
2456
- "mtp.layers.0.mlp.experts.85.down_proj",
2457
- "mtp.layers.0.mlp.experts.85.gate_proj",
2458
- "mtp.layers.0.mlp.experts.85.up_proj",
2459
- "mtp.layers.0.mlp.experts.86.down_proj",
2460
- "mtp.layers.0.mlp.experts.86.gate_proj",
2461
- "mtp.layers.0.mlp.experts.86.up_proj",
2462
- "mtp.layers.0.mlp.experts.87.down_proj",
2463
- "mtp.layers.0.mlp.experts.87.gate_proj",
2464
- "mtp.layers.0.mlp.experts.87.up_proj",
2465
- "mtp.layers.0.mlp.experts.88.down_proj",
2466
- "mtp.layers.0.mlp.experts.88.gate_proj",
2467
- "mtp.layers.0.mlp.experts.88.up_proj",
2468
- "mtp.layers.0.mlp.experts.89.down_proj",
2469
- "mtp.layers.0.mlp.experts.89.gate_proj",
2470
- "mtp.layers.0.mlp.experts.89.up_proj",
2471
- "mtp.layers.0.mlp.experts.9.down_proj",
2472
- "mtp.layers.0.mlp.experts.9.gate_proj",
2473
- "mtp.layers.0.mlp.experts.9.up_proj",
2474
- "mtp.layers.0.mlp.experts.90.down_proj",
2475
- "mtp.layers.0.mlp.experts.90.gate_proj",
2476
- "mtp.layers.0.mlp.experts.90.up_proj",
2477
- "mtp.layers.0.mlp.experts.91.down_proj",
2478
- "mtp.layers.0.mlp.experts.91.gate_proj",
2479
- "mtp.layers.0.mlp.experts.91.up_proj",
2480
- "mtp.layers.0.mlp.experts.92.down_proj",
2481
- "mtp.layers.0.mlp.experts.92.gate_proj",
2482
- "mtp.layers.0.mlp.experts.92.up_proj",
2483
- "mtp.layers.0.mlp.experts.93.down_proj",
2484
- "mtp.layers.0.mlp.experts.93.gate_proj",
2485
- "mtp.layers.0.mlp.experts.93.up_proj",
2486
- "mtp.layers.0.mlp.experts.94.down_proj",
2487
- "mtp.layers.0.mlp.experts.94.gate_proj",
2488
- "mtp.layers.0.mlp.experts.94.up_proj",
2489
- "mtp.layers.0.mlp.experts.95.down_proj",
2490
- "mtp.layers.0.mlp.experts.95.gate_proj",
2491
- "mtp.layers.0.mlp.experts.95.up_proj",
2492
- "mtp.layers.0.mlp.experts.96.down_proj",
2493
- "mtp.layers.0.mlp.experts.96.gate_proj",
2494
- "mtp.layers.0.mlp.experts.96.up_proj",
2495
- "mtp.layers.0.mlp.experts.97.down_proj",
2496
- "mtp.layers.0.mlp.experts.97.gate_proj",
2497
- "mtp.layers.0.mlp.experts.97.up_proj",
2498
- "mtp.layers.0.mlp.experts.98.down_proj",
2499
- "mtp.layers.0.mlp.experts.98.gate_proj",
2500
- "mtp.layers.0.mlp.experts.98.up_proj",
2501
- "mtp.layers.0.mlp.experts.99.down_proj",
2502
- "mtp.layers.0.mlp.experts.99.gate_proj",
2503
- "mtp.layers.0.mlp.experts.99.up_proj",
2504
  "mtp.layers.0.mlp.gate",
2505
  "mtp.layers.0.mlp.shared_expert.down_proj",
2506
  "mtp.layers.0.mlp.shared_expert.gate_proj",
@@ -2520,7 +983,7 @@
2520
  "kv_cache_quant_config": {},
2521
  "kv_cache_post_rope": false,
2522
  "quant_mode": "eager_mode",
2523
- "version": "0.11.1",
2524
  "export": {
2525
  "kv_cache_group": [],
2526
  "min_kv_scale": 0.0,
 
964
  "model.visual.blocks.9.norm2",
965
  "model.visual.merger.linear_fc1",
966
  "model.visual.merger.linear_fc2",
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
967
  "mtp.layers.0.mlp.gate",
968
  "mtp.layers.0.mlp.shared_expert.down_proj",
969
  "mtp.layers.0.mlp.shared_expert.gate_proj",
 
983
  "kv_cache_quant_config": {},
984
  "kv_cache_post_rope": false,
985
  "quant_mode": "eager_mode",
986
+ "version": "0.12+6b1a3c15e5e",
987
  "export": {
988
  "kv_cache_group": [],
989
  "min_kv_scale": 0.0,
model.safetensors-00001-of-00094.safetensors CHANGED
@@ -1,3 +1,3 @@
1
  version https://git-lfs.github.com/spec/v1
2
- oid sha256:731276a79bd9f10311d7ad5d685529e213870f72ddca0eaa87b27ce595b97a8e
3
  size 2281988744
 
1
  version https://git-lfs.github.com/spec/v1
2
+ oid sha256:c8b83709a460f87dfaf304923909a38de285c670df2e200b119cd9d8c3622783
3
  size 2281988744
model.safetensors-00002-of-00094.safetensors CHANGED
@@ -1,3 +1,3 @@
1
  version https://git-lfs.github.com/spec/v1
2
- oid sha256:d4d02bc4c9bd82d49fde38dbc340c1e01530be3972e28dd21897ae28c98a03a3
3
  size 2281988744
 
1
  version https://git-lfs.github.com/spec/v1
2
+ oid sha256:6a0cfcb889cf594712dcf0ccce0700b8547b01f2bb50511c51a27ef701506b48
3
  size 2281988744
model.safetensors-00003-of-00094.safetensors CHANGED
@@ -1,3 +1,3 @@
1
  version https://git-lfs.github.com/spec/v1
2
- oid sha256:8dd2651ce1b991bcee5e00bc803c9ebc79c9bd506c68f41831526896623755a4
3
  size 2281988744
 
1
  version https://git-lfs.github.com/spec/v1
2
+ oid sha256:eee99d7ebe49f6940fe37750d617bdb9bb6429b237973337389e1173525d0765
3
  size 2281988744
model.safetensors-00004-of-00094.safetensors CHANGED
@@ -1,3 +1,3 @@
1
  version https://git-lfs.github.com/spec/v1
2
- oid sha256:44414825271572c078da84cfb058d548e55a4cccd61974cb715aa2b9460727eb
3
  size 2281986696
 
1
  version https://git-lfs.github.com/spec/v1
2
+ oid sha256:450e0b4eec152e6ab6c3b43df940be47b2895aae72794b7b17df529205a8cceb
3
  size 2281986696
model.safetensors-00005-of-00094.safetensors CHANGED
@@ -1,3 +1,3 @@
1
  version https://git-lfs.github.com/spec/v1
2
- oid sha256:24f3b2f767ed148bfb2fab8b482136ce9c72d2d374626b510bcd3c08bc9b9a4e
3
  size 2281986696
 
1
  version https://git-lfs.github.com/spec/v1
2
+ oid sha256:099820b92ab919c4cc942a672b8644b90a935544f4dc7a44fa781c13e8975181
3
  size 2281986696
model.safetensors-00006-of-00094.safetensors CHANGED
@@ -1,3 +1,3 @@
1
  version https://git-lfs.github.com/spec/v1
2
- oid sha256:18a8d2af5d361ead0506f715baf1ade1471568bafe31254a6b0cee8208c675a5
3
  size 2281988744
 
1
  version https://git-lfs.github.com/spec/v1
2
+ oid sha256:527d1e4f47826ad0a9c960067895966cedb29ac698713434530ec48feaecc4ee
3
  size 2281988744
model.safetensors-00007-of-00094.safetensors CHANGED
@@ -1,3 +1,3 @@
1
  version https://git-lfs.github.com/spec/v1
2
- oid sha256:68dee1361f1aea6c2768aada6417c4292bad0d791a39e12b2e3a2183fcd13f17
3
  size 2281988744
 
1
  version https://git-lfs.github.com/spec/v1
2
+ oid sha256:83329bfdec2680e8c7f79736ac23389550757a6e15a7d8da534f70b398f08a3f
3
  size 2281988744
model.safetensors-00008-of-00094.safetensors CHANGED
@@ -1,3 +1,3 @@
1
  version https://git-lfs.github.com/spec/v1
2
- oid sha256:9670a9cb93faf964c8d0dc586d70afbbbfe12eaa5e2c060f3b7ab637fd111a10
3
  size 2281988744
 
1
  version https://git-lfs.github.com/spec/v1
2
+ oid sha256:12b15372ff5841e5e26badb56ee5645f5acdc35b8b62dcde7db54927163fed6f
3
  size 2281988744
model.safetensors-00009-of-00094.safetensors CHANGED
@@ -1,3 +1,3 @@
1
  version https://git-lfs.github.com/spec/v1
2
- oid sha256:bc24cef08279c9a813e9bb11e89fc20544d0c6306172c91a34ba517e85452b1a
3
  size 2281988744
 
1
  version https://git-lfs.github.com/spec/v1
2
+ oid sha256:d9fae295d14c9f529435ff345a802008fa58d8de3128b84d635869598c83f7bc
3
  size 2281988744
model.safetensors-00010-of-00094.safetensors CHANGED
@@ -1,3 +1,3 @@
1
  version https://git-lfs.github.com/spec/v1
2
- oid sha256:75343cfe7df98c881782eb3abe73e7a565aabdb48315f95cc695134688f49ae2
3
  size 2281988744
 
1
  version https://git-lfs.github.com/spec/v1
2
+ oid sha256:2e36bb40963c16d655c19241a43d4d32bb96db1209fec35ada607ac2ff467775
3
  size 2281988744
model.safetensors-00011-of-00094.safetensors CHANGED
@@ -1,3 +1,3 @@
1
  version https://git-lfs.github.com/spec/v1
2
- oid sha256:c0f406376fc71b38efc552a12751db41a9efbacb6be07d059c3a8ef045c878b1
3
  size 2281988744
 
1
  version https://git-lfs.github.com/spec/v1
2
+ oid sha256:72278c458869953b58eb5db592632fa4090491dc1deb3c9cc0e7d2dd10ce6d02
3
  size 2281988744
model.safetensors-00012-of-00094.safetensors CHANGED
@@ -1,3 +1,3 @@
1
  version https://git-lfs.github.com/spec/v1
2
- oid sha256:2f9bbe03a057819b823141e78825e90170bc59736cb9095501a5360d487146a9
3
  size 2281988744
 
1
  version https://git-lfs.github.com/spec/v1
2
+ oid sha256:fdde75d2d9089a04c8dcb5e2473599e7f886b686c7057f7a1d3054fef932f28d
3
  size 2281988744
model.safetensors-00013-of-00094.safetensors CHANGED
@@ -1,3 +1,3 @@
1
  version https://git-lfs.github.com/spec/v1
2
- oid sha256:97e51e07bb068d2920fbfb3c711c03e50f3da793fc9f79676be4cd420538fbfc
3
  size 2281988744
 
1
  version https://git-lfs.github.com/spec/v1
2
+ oid sha256:9abd1d6d6e857f90dabf458a62f6e0cc88247c844db42e5c84145eb9e7350213
3
  size 2281988744
model.safetensors-00014-of-00094.safetensors CHANGED
@@ -1,3 +1,3 @@
1
  version https://git-lfs.github.com/spec/v1
2
- oid sha256:541c32338baef4835ee786761edfb4f31f2a40dab480347002f5b4ff091fcfd6
3
  size 2281988744
 
1
  version https://git-lfs.github.com/spec/v1
2
+ oid sha256:f18b635663e7c0fa69af09976eaec785cd586ea376229496798682d2d37a735e
3
  size 2281988744
model.safetensors-00015-of-00094.safetensors CHANGED
@@ -1,3 +1,3 @@
1
  version https://git-lfs.github.com/spec/v1
2
- oid sha256:6f9bc64023a302f1556c972b17cdf6096d06bb4e56e73c25f1b235de33b217e5
3
  size 2281988744
 
1
  version https://git-lfs.github.com/spec/v1
2
+ oid sha256:7a0f527d8360d6cf4a13d7b04b432fc2b3dd1a9eb6018a042d1c59ab312249c6
3
  size 2281988744
model.safetensors-00016-of-00094.safetensors CHANGED
@@ -1,3 +1,3 @@
1
  version https://git-lfs.github.com/spec/v1
2
- oid sha256:5201a254a42bc6f497402489eabe87618b3c01346b74b742f5c14449f9335041
3
  size 2281986696
 
1
  version https://git-lfs.github.com/spec/v1
2
+ oid sha256:da89af735bb1cc45cff2c59ca5655966ac18f278601b280b3332d06fa0a1be40
3
  size 2281986696
model.safetensors-00017-of-00094.safetensors CHANGED
@@ -1,3 +1,3 @@
1
  version https://git-lfs.github.com/spec/v1
2
- oid sha256:783b1466cfd55ed76964ec48b9f0cc37a160147d87872ff8fce5148e0d55f36f
3
  size 2281988744
 
1
  version https://git-lfs.github.com/spec/v1
2
+ oid sha256:c7094ac1fe434fce359d2cf8ffe30a37669526c1f78a809886f62bf7f8c2e45d
3
  size 2281988744
model.safetensors-00018-of-00094.safetensors CHANGED
@@ -1,3 +1,3 @@
1
  version https://git-lfs.github.com/spec/v1
2
- oid sha256:fb5d80299268448b5f6635ad11882e3dad5105c08fe1e0866357c8ee07bc0f91
3
  size 2281986696
 
1
  version https://git-lfs.github.com/spec/v1
2
+ oid sha256:a2baa70d163c65047f4f63419529d17a8aed77413c9baad68445898134defc4a
3
  size 2281986696
model.safetensors-00019-of-00094.safetensors CHANGED
@@ -1,3 +1,3 @@
1
  version https://git-lfs.github.com/spec/v1
2
- oid sha256:7b982948442336ba9bf08b0c5ded286b946ffbc5a941aaa77d68d6684585d29f
3
  size 2281986696
 
1
  version https://git-lfs.github.com/spec/v1
2
+ oid sha256:fd6e0c87cb21bfca1c39098ca43578d422ee14a5aa773f8dc24ab1b0bfc401cd
3
  size 2281986696
model.safetensors-00020-of-00094.safetensors CHANGED
@@ -1,3 +1,3 @@
1
  version https://git-lfs.github.com/spec/v1
2
- oid sha256:cd150faf57c8358dfc51c19492703ec7c9ae550e03b8d9a0f8668484cfd532d2
3
  size 2281988744
 
1
  version https://git-lfs.github.com/spec/v1
2
+ oid sha256:f664716af0f3c341c9527e347394a206ad37444d5ae17c0c4fd61f4218be22b9
3
  size 2281988744
model.safetensors-00021-of-00094.safetensors CHANGED
@@ -1,3 +1,3 @@
1
  version https://git-lfs.github.com/spec/v1
2
- oid sha256:a82c9b79b01405d3442615a1d29eaaae03f256f9b4f8b9054ffd93209718fdbf
3
  size 2281988744
 
1
  version https://git-lfs.github.com/spec/v1
2
+ oid sha256:fca84617a337b629ac41745f9d7d11a613cdf202db710455aa3d4a00ae81a221
3
  size 2281988744
model.safetensors-00022-of-00094.safetensors CHANGED
@@ -1,3 +1,3 @@
1
  version https://git-lfs.github.com/spec/v1
2
- oid sha256:721e948133b1e80727cf660e16faf86ac5dc006e5add9311c63fea1b7b54e880
3
  size 2281988744
 
1
  version https://git-lfs.github.com/spec/v1
2
+ oid sha256:f78ac70f346a5e39c6425a914477293763545f1c2e5e36e74133e9daac419cae
3
  size 2281988744
model.safetensors-00023-of-00094.safetensors CHANGED
@@ -1,3 +1,3 @@
1
  version https://git-lfs.github.com/spec/v1
2
- oid sha256:33a546b9433af6efd7aa1ef7e74cc3cff4f5adb24098e278385045df893b2a34
3
  size 2281988744
 
1
  version https://git-lfs.github.com/spec/v1
2
+ oid sha256:f26f5dfecff6fc07c25f55b2e15dcfc928e58905c6b1a139b610251aac15afaf
3
  size 2281988744
model.safetensors-00024-of-00094.safetensors CHANGED
@@ -1,3 +1,3 @@
1
  version https://git-lfs.github.com/spec/v1
2
- oid sha256:94f78c60df7475f6a456d298bbfb7aa50051ed26ce0fbf67d1fbfe6e14943c83
3
  size 2281988744
 
1
  version https://git-lfs.github.com/spec/v1
2
+ oid sha256:440a8928cecc616afb8cd6fd62be75a357f083658a4141bf89a2396ca7e16f44
3
  size 2281988744
model.safetensors-00025-of-00094.safetensors CHANGED
@@ -1,3 +1,3 @@
1
  version https://git-lfs.github.com/spec/v1
2
- oid sha256:e4dea79792544de2fab3bb05bf1d9744c01e037d3f576af253eabeff025905d7
3
  size 2281988744
 
1
  version https://git-lfs.github.com/spec/v1
2
+ oid sha256:eaa666b259aece55bf8d4e01ef9f8f9ef850fd5bf88bcacf32ee8df9ef260810
3
  size 2281988744
model.safetensors-00026-of-00094.safetensors CHANGED
@@ -1,3 +1,3 @@
1
  version https://git-lfs.github.com/spec/v1
2
- oid sha256:09320fabe63119be40f2eb4f23c5422cbec557324de37fd55d0a69497ea186b2
3
  size 2281988744
 
1
  version https://git-lfs.github.com/spec/v1
2
+ oid sha256:d0c3061339b73b5133772742d054fcf7536ab30f1779fd71e73ebe6029e243c9
3
  size 2281988744
model.safetensors-00027-of-00094.safetensors CHANGED
@@ -1,3 +1,3 @@
1
  version https://git-lfs.github.com/spec/v1
2
- oid sha256:e23237accc7b8701d1dda86af6a235270e0e4f66084650e47db175230b56904a
3
  size 2281988744
 
1
  version https://git-lfs.github.com/spec/v1
2
+ oid sha256:33f66d1c9a297e1d78b075eecee423b8f68a52fbc8248a33871ce9391069a841
3
  size 2281988744
model.safetensors-00028-of-00094.safetensors CHANGED
@@ -1,3 +1,3 @@
1
  version https://git-lfs.github.com/spec/v1
2
- oid sha256:e54c1d6692b67372bf09c2dd83edc31b5b7769e12fd260dafb66b706bbf8d78c
3
  size 2281986696
 
1
  version https://git-lfs.github.com/spec/v1
2
+ oid sha256:88e3404ac0c3b9f61562996da644bf2e97990cd2b773dc37bb4dcf507f260552
3
  size 2281986696
model.safetensors-00029-of-00094.safetensors CHANGED
@@ -1,3 +1,3 @@
1
  version https://git-lfs.github.com/spec/v1
2
- oid sha256:9e10214a0d453d9d5b97bbf4fcf55924b72827f396a40df3963fd2d520b26528
3
  size 2281986696
 
1
  version https://git-lfs.github.com/spec/v1
2
+ oid sha256:9eb5f1d5d6880e3d87b1bc130a56b4d71cc4877fa7f819e541b92c2e44f433bb
3
  size 2281986696
model.safetensors-00030-of-00094.safetensors CHANGED
@@ -1,3 +1,3 @@
1
  version https://git-lfs.github.com/spec/v1
2
- oid sha256:301a956039c93058a628241e6938d6f15ac6d058acd3469496bb1846a1dc9153
3
  size 2281988744
 
1
  version https://git-lfs.github.com/spec/v1
2
+ oid sha256:e6c9eae839b05a1d7c76a50d8bed7841a26b5242110455c7271d60a3bcf2f672
3
  size 2281988744
model.safetensors-00031-of-00094.safetensors CHANGED
@@ -1,3 +1,3 @@
1
  version https://git-lfs.github.com/spec/v1
2
- oid sha256:834211613e5b6847e389a651cd1acdf9b4e66398c3a63e5d261a6782d68dd5b2
3
  size 2281986696
 
1
  version https://git-lfs.github.com/spec/v1
2
+ oid sha256:2dc8e6dc5c205dd786fc77cb38dd205cb26967fb79a3a5d83d1878c0cdd6ac97
3
  size 2281986696
model.safetensors-00032-of-00094.safetensors CHANGED
@@ -1,3 +1,3 @@
1
  version https://git-lfs.github.com/spec/v1
2
- oid sha256:c4292fcc7cc13e2a7bc66d624687afdd4ad5555a1682b10dd5f579ea0ce3058f
3
  size 2281988744
 
1
  version https://git-lfs.github.com/spec/v1
2
+ oid sha256:cb058813f5438d63324f6e36b392b5257987c8a26a0ff84ab48308652ff03733
3
  size 2281988744
model.safetensors-00033-of-00094.safetensors CHANGED
@@ -1,3 +1,3 @@
1
  version https://git-lfs.github.com/spec/v1
2
- oid sha256:280347a11f53eb5d6bb7543a3167939c18d387d0ea8b39749c93dac41289b29b
3
  size 2281988744
 
1
  version https://git-lfs.github.com/spec/v1
2
+ oid sha256:b4c96dff6a444fa827db23137d12cf6c278b03b4604994a86be9be0e2c34c225
3
  size 2281988744
model.safetensors-00034-of-00094.safetensors CHANGED
@@ -1,3 +1,3 @@
1
  version https://git-lfs.github.com/spec/v1
2
- oid sha256:67b76be428e41fc9e3e8630a1ca6756ded5fadbd2c6a6f29cc0c481f1f255746
3
  size 2281988744
 
1
  version https://git-lfs.github.com/spec/v1
2
+ oid sha256:b6eb8f4cdf4ce7b9a22077966000b21a4dd12eb7075053afff79bbb6e7ef8810
3
  size 2281988744
model.safetensors-00035-of-00094.safetensors CHANGED
@@ -1,3 +1,3 @@
1
  version https://git-lfs.github.com/spec/v1
2
- oid sha256:9279f5cebc1a62084fe1888ba3a16e557d351dc85ab38f2fdbbf32e74501b266
3
  size 2281986696
 
1
  version https://git-lfs.github.com/spec/v1
2
+ oid sha256:5255cb94f24fbee07952a47146b54b9b23d2c765316df60780533e290d22be8a
3
  size 2281986696
model.safetensors-00036-of-00094.safetensors CHANGED
@@ -1,3 +1,3 @@
1
  version https://git-lfs.github.com/spec/v1
2
- oid sha256:b2881e449314438a6a3d861c6d043c9bd4d09236be75252cbca272b1ac90060e
3
  size 2281988744
 
1
  version https://git-lfs.github.com/spec/v1
2
+ oid sha256:e4162243814c73ae247f9ef0c66ea94f8a0ae7f2b121875f44c22a6aa435ef14
3
  size 2281988744
model.safetensors-00037-of-00094.safetensors CHANGED
@@ -1,3 +1,3 @@
1
  version https://git-lfs.github.com/spec/v1
2
- oid sha256:4789e9031b7c90cee2ba0ddde50191da75670d631b05218a9129769c8cea5ecf
3
  size 2281988744
 
1
  version https://git-lfs.github.com/spec/v1
2
+ oid sha256:baddc08ebd62e8dc6ea2068b43f2076d4a945cd4c94468a17465948e29a2c03e
3
  size 2281988744
model.safetensors-00038-of-00094.safetensors CHANGED
@@ -1,3 +1,3 @@
1
  version https://git-lfs.github.com/spec/v1
2
- oid sha256:dc1650cd885ca50ceb139b8313ef74ccafc7853f6ff875eff585e3a68da288b5
3
  size 2281988744
 
1
  version https://git-lfs.github.com/spec/v1
2
+ oid sha256:ed92df8b3692fe718a18d5764be951cff2e5fc434ba82774e2b979d8848c517b
3
  size 2281988744
model.safetensors-00039-of-00094.safetensors CHANGED
@@ -1,3 +1,3 @@
1
  version https://git-lfs.github.com/spec/v1
2
- oid sha256:a1470954ae770798a7d8c79b408cd6af676fed517809225c824976f4b341f2b7
3
  size 2281988744
 
1
  version https://git-lfs.github.com/spec/v1
2
+ oid sha256:8adf36b774dcf52bd681edaaa63eb7f5dbad9b08c0d85c4ecac584d3c2e59f55
3
  size 2281988744
model.safetensors-00040-of-00094.safetensors CHANGED
@@ -1,3 +1,3 @@
1
  version https://git-lfs.github.com/spec/v1
2
- oid sha256:3a7c0fa491b4f7b20e0cd4c8991c7aeba881285ed92cd4c507cd85c62ea39c59
3
  size 2281988744
 
1
  version https://git-lfs.github.com/spec/v1
2
+ oid sha256:101cf1ddd548260d1f262b976966c313f5357168cb134506b829549a058f4c6f
3
  size 2281988744
model.safetensors-00041-of-00094.safetensors CHANGED
@@ -1,3 +1,3 @@
1
  version https://git-lfs.github.com/spec/v1
2
- oid sha256:2490f81baabf295c96d8809e220f17fdc971842ef9fa60ebb53709bf633010a2
3
  size 2281988744
 
1
  version https://git-lfs.github.com/spec/v1
2
+ oid sha256:38263863acc0f5cb1a0dbb1a196dec9b32a7e9aad1587d637363bab6f02f6b4a
3
  size 2281988744
model.safetensors-00042-of-00094.safetensors CHANGED
@@ -1,3 +1,3 @@
1
  version https://git-lfs.github.com/spec/v1
2
- oid sha256:f23b84a6063c0b8d12fe1a2c9e1ff2c3d9f30c3831265943220d0f271fe2e29b
3
  size 2281988744
 
1
  version https://git-lfs.github.com/spec/v1
2
+ oid sha256:f39cc2ffada952fd8c1747e11fe6fef7b9f0eeb121bfddcb8cbb8920952ea661
3
  size 2281988744
model.safetensors-00043-of-00094.safetensors CHANGED
@@ -1,3 +1,3 @@
1
  version https://git-lfs.github.com/spec/v1
2
- oid sha256:2dd20f9927cdd6f217b3bc16928676714cbd530ff0fbaa759a1223fd01981202
3
  size 2281988744
 
1
  version https://git-lfs.github.com/spec/v1
2
+ oid sha256:8ee4af94802ba5c09da8267fbb3477c366e3cfe36bf17d93cc064c346bedefb9
3
  size 2281988744
model.safetensors-00044-of-00094.safetensors CHANGED
@@ -1,3 +1,3 @@
1
  version https://git-lfs.github.com/spec/v1
2
- oid sha256:fa3437ab9a8705fc6f896b698736f67aff0599463a87250631c8b403a72b14bd
3
  size 2281988744
 
1
  version https://git-lfs.github.com/spec/v1
2
+ oid sha256:0f349e3404a58e255c5fed214f3da64e66a2f06adb9eb184ffd27fe82be61848
3
  size 2281988744
model.safetensors-00045-of-00094.safetensors CHANGED
@@ -1,3 +1,3 @@
1
  version https://git-lfs.github.com/spec/v1
2
- oid sha256:aaf7fc075e979187418d2588a5a9b67bebff0cf37ff2c35a48226386667157ef
3
  size 2281986696
 
1
  version https://git-lfs.github.com/spec/v1
2
+ oid sha256:f02f96eb5e350c40566bd185de3f41ada8a3965a5ce27f8ef65d5230c447ef17
3
  size 2281986696
model.safetensors-00046-of-00094.safetensors CHANGED
@@ -1,3 +1,3 @@
1
  version https://git-lfs.github.com/spec/v1
2
- oid sha256:921e3eaf0f362a60b43a4a5601a3c819c373c15cd6fc47f280cfebf6c5c22d00
3
  size 2281988744
 
1
  version https://git-lfs.github.com/spec/v1
2
+ oid sha256:d4e63a0e095b1e04f36df0c33a6a5a20062d604a8dc0b2e242c1a96f3b487310
3
  size 2281988744
model.safetensors-00047-of-00094.safetensors CHANGED
@@ -1,3 +1,3 @@
1
  version https://git-lfs.github.com/spec/v1
2
- oid sha256:a8dc711fbd3a2088f455cb531b07681d8a0f1cbf4b49fbe2b6dfd52808906cc5
3
  size 2281988744
 
1
  version https://git-lfs.github.com/spec/v1
2
+ oid sha256:a1fb49043d8f74a0ae9e6d9570049fdc5dab5863ba8d4ce425c58ffd6dd07b24
3
  size 2281988744
model.safetensors-00048-of-00094.safetensors CHANGED
@@ -1,3 +1,3 @@
1
  version https://git-lfs.github.com/spec/v1
2
- oid sha256:9819baf987d7dcafd3b22627d3a67800710d4dcafd0e6b59d2f307660c736c54
3
  size 2281988744
 
1
  version https://git-lfs.github.com/spec/v1
2
+ oid sha256:b23e28a48fb60256c015c9d4745b0229c256ab7365e22dd532fa2c8f85c37d3d
3
  size 2281988744