Update quantized scripts.md

README.md

@@ -1,6 +1,6 @@
 ---
 base_model:
-- Qwen/Qwen3.5-397B-A17B
+- Qwen/Qwen3.5-397B-A17B-FP8
 language:
 - en
 library_name: transformers
@@ -14,12 +14,12 @@ license_link: https://huggingface.co/Qwen/Qwen3.5-397B-A17B/blob/main/LICENSE
   - **Input:** Text
   - **Output:** Text
 - **Supported Hardware Microarchitecture:** AMD MI300 MI350/MI355
-- **ROCm**: 7.0
-- **PyTorch**: 2.8.0
-- **Transformers**: 5.2.0
+- **ROCm**: 7.0.0
+- **PyTorch**: 2.9.1
+- **Transformers**: 5.3.0
 - **Operating System(s):** Linux
 - **Inference Engine:** [SGLang](https://docs.sglang.ai/)/[vLLM](https://docs.vllm.ai/en/latest/)
-- **Model Optimizer:** [AMD-Quark](https://quark.docs.amd.com/latest/index.html) (v0.11)
+- **Model Optimizer:** [AMD-Quark](https://quark.docs.amd.com/latest/index.html) (v0.11.1)
   - **Weight quantization:** OCP MXFP4, Static
   - **Activation quantization:** OCP MXFP4, Dynamic
 
@@ -31,13 +31,35 @@ The model was quantized from [Qwen/Qwen3.5-397B-A17B-FP8](https://huggingface.co
 
 **Quantization scripts:**
 ```
-cd Quark/examples/torch/language_modeling/llm_ptq/
-export exclude_layers="lm_head  model.visual.*  mtp.*  *mlp.gate *shared_expert_gate* *.linear_attn.*  *.self_attn.*  *.shared_expert.*"
-python3 quantize_quark.py --model_dir Qwen/Qwen3.5-397B-A17B-FP8 \
-                          --quant_scheme  mxfp4 \
-                          --file2file_quantization \
-                          --exclude_layers $exclude_layers \
-                          --output_dir amd/Qwen3.5-397B-A17B-MXFP4
+import os
+from quark.torch import LLMTemplate, ModelQuantizer
+from quark.common.profiler import GlobalProfiler
+
+# Register qwen3_5_moe template
+qwen3_5_moe_template = LLMTemplate(
+    model_type="qwen3_5_moe",
+    kv_layers_name=["*k_proj", "*v_proj"],
+    q_layer_name="*q_proj"
+)
+LLMTemplate.register_template(qwen3_5_moe_template)
+
+# Configuration
+ckpt_path = "Qwen/Qwen3.5-397B-A17B-FP8"
+output_dir = "amd/Qwen3.5-397B-A17B-MXFP4"
+quant_scheme = "mxfp4"
+exclude_layers = ["lm_head", "model.visual.*", "mtp.*", "*mlp.gate", "*shared_expert_gate*", "*.linear_attn.*", "*.self_attn.*", "*.shared_expert.*"]
+
+# Get quant config from template
+template = LLMTemplate.get("qwen3_5_moe")
+quant_config = template.get_config(scheme=quant_scheme, exclude_layers=exclude_layers)
+
+# Quantize with File-to-file mode
+profiler = GlobalProfiler(output_path=os.path.join(output_dir, "quark_profile.yaml"))
+quantizer = ModelQuantizer(quant_config)
+quantizer.direct_quantize_checkpoint(
+    pretrained_model_path=ckpt_path,
+    save_path=output_dir,
+)
 ```
 For further details or issues, please refer to the AMD-Quark documentation or contact the respective developers.
 
@@ -72,7 +94,7 @@ The model was evaluated on gsm8k benchmarks using the [vllm](https://github.com/
 
 ### Reproduction
 
-The GSM8K results were obtained using the vLLM framework, based on the Docker image `rocm/vllm-dev:nightly_main_20260211`, and vLLM is installed inside the container with fixes applied for model support.
+The GSM8K results were obtained using the vLLM framework, based on the Docker image `rocm/vllm-dev:nightly_main_20260211`, and vLLM is installed inside the container.
 
 #### Evaluating model in a new terminal
 ```